Skip to content

Scrapy community management

jrault edited this page Dec 21, 2012 · 1 revision

Communication

We will use the scrapy-developers list to:

  • discuss and agree over Scrapy changes required by the HCI project
  • notify changes about Scrapy core that are relevant to the HCI project, and work together with HCI team to migrate any required code to ensure HCI works on latest Scrapy version
Special care will be taken by the Scrapinghub team to ensure the HCI code will always work on latest Scrapy stable version (at least until Scrapy 1.0, scheduled to be released in mid-2012), so that both projects can grow together.

Development & code

We will use GitHub (forks, pull requests) to collaborate on Scrapy changes between the Scrapinghub and HCI team.

Changes to Scrapy and Scrapyd API

Changes to Scrapy and Scrapyd API (that affect HCI project) will be communicated and discussed with HCI team before the (Scrapy) release, optionally submitting patches to HCI crawler to port it to the new API.

Clone this wiki locally