This repository has been archived by the owner on Jun 10, 2024. It is now read-only.
v0.3.8
New features:
- Now you can use
cancel
to stop an active task of a task withauto_recrawl
enabled. Handler.crawl_config
will now be applied to the task when fetching. (It's applied when the task created before, that means proxy/headers can be changed afterward). See http://docs.pyspider.org/en/latest/apis/self.crawl/#handlercrawl_config
Fix several bugs:
- * Fixed a global config object thread interference issue, which may cause
connect to scheduler rpc error: error(10061, '')
error whenall --run-in=thread
(default in windows platform) - Fix
response.save
lost when fetch failed issue - Fix potential scheduler failure caused by old version of six
- Fix result dump return nothing when using mongodb backend