Skip to content
Davide Tampellini edited this page Jun 2, 2015 · 5 revisions

Now that you have fetched the dump files, you have to separate the interesting stuff from the useless one. This is where the magic happens: we have downloaded our dumps, calculated their score, but how can we effectively know if what we're looking at is an ugly debug file or a juicy hash list?

Simple, you have to run the classifier:
dumpscraper classify <arguments>

Arguments could be one of the following one:

  • since Starting date in the format YYYY-MM-DD. If the until argument is not supplied, Dump Scraper will only process this day
  • until Stop date in the format YYYY-MM-DD

This script will look at the score of each dump, decide which type belongs to (trash, hash, or plain) and copy it under the organized folder in the correct section.

organized
  `- hash
  `- plain
  `- trash

We are copying the file instead of moving it because we always want the original information. In this way, if we improve the classifier, you can safely run the classifier again, without losing your precious dump files.

Clone this wiki locally