Skip to content

Latest commit

 

History

History
72 lines (52 loc) · 1.41 KB

README.md

File metadata and controls

72 lines (52 loc) · 1.41 KB
Raw

FOIATool

Tool for journalists, researchers, and concerned citizens to gather documents from NextRequest based FOIA portals. You can use this tool to periodically search for and download documents related to topics of interest.

Installation
python3 -m pip install .
Basic Usage

Generate a config file:

foiatool init

Edit the config file foia/foiatool.toml:

db_path = "foia.db"

[[request_config]]
url = "https://sanfrancisco.nextrequest.com"
user = "<USER NAME>"
password = "<PASSWORD>"
search_terms = ["police"]
document_search_terms = ["budget"]
ignore_ids = ["123"]
download_timeout = 1200
download_nice_seconds = 2
download_path = "downloads"

[[request_config]]
url = "https://another_site.nextrequest.com"
user = "<USER NAME>"
password = "<PASSWORD>"
search_terms = []
document_search_terms = []
ignore_ids = []
download_timeout = 1200
download_nice_seconds = 2
download_path = "downloads"

Paths can be specified as absolute or relative paths.

Run downloader:

foiatool
Example FOIA Portals
Future
  • Support GovQA based portals
  • Enable multi-threaded downloading
  • Provide an easy cross-platform way to schedule the tool
Requirements
  • peewee
  • requests