Skip to content

v1.1.0

Compare
Choose a tag to compare
@tamarasaurus tamarasaurus released this 03 Jul 23:05
· 36 commits to master since this release
  • Add tests for scraper sources #18 #17
    • These are run nightly on Travis CI for every source
    • Can also be run locally (see Testing)
  • Add handling of multiple pages in the runner #21
        // The selector for the next page button
        public nextPageSelector = '.pager-next > a'
    
        // How many pages do you want to scrape
        public pagesToScrape: number = 5
  • Add pagination in the database, api and frontend #23
  • Run the scrapers sequentially to improve stability
  • Add some clearer and prettier logging to the runner #1