Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: Fix timeouts and retry limit exceeded in interim introduction e…
…xample (#2791) I'm not sure if this is the correct fix, but it does fix the problem, and it uses similar logic to the final script on this page, which is working properly. (It also works if you simply comment out the `await page.waitForSelector('.collection-block-item');` line, but I assume that's there for a reason.) Old output: ``` INFO PlaywrightCrawler: All requests from the queue have been processed, the crawler will shut down. INFO PlaywrightCrawler: Final request statistics: {"requestsFinished":1,"requestsFailed":31,"retryHistogram":[1,null,null,31],"requestAvgFailedDurationMillis":30359,"requestAvgFinishedDurationMillis":1056,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":6,"requestTotalDurationMillis":942186,"requestsTotal":32,"crawlerRuntimeMillis":291681} INFO PlaywrightCrawler: Error analysis: {"totalErrors":31,"uniqueErrors":1,"mostCommonErrors":["31x: page.waitForSelector: Timeout 30000ms exceeded. (/Users/webchick/TechAround/fun-with-scraping/src/main.js:8:20)"]} INFO PlaywrightCrawler: Finished! Total 32 requests: 1 succeeded, 31 failed. {"terminal":true} ``` New output: ``` INFO PlaywrightCrawler: All requests from the queue have been processed, the crawler will shut down. INFO PlaywrightCrawler: Final request statistics: {"requestsFinished":32,"requestsFailed":0,"retryHistogram":[32],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":302,"requestsFinishedPerMinute":340,"requestsFailedPerMinute":0,"requestTotalDurationMillis":9677,"requestsTotal":32,"crawlerRuntimeMillis":5644} INFO PlaywrightCrawler: Finished! Total 32 requests: 32 succeeded, 0 failed. {"terminal":true} ``` Closes #2790
- Loading branch information