c9.io scraper

Web scraper for archived c9.io projects. Built with WebDriverJS (Selenium for Node.js). Bootstrapped from webdriverjs-recipes.

Prerequisites

Node.js and npm:

brew install node

Chrome and ChromeDriver:

brew cask install google-chrome
brew install chromedriver

Install

Clone repository:

git clone https://github.com/remarkablemark/c9.io-scraper.git
cd c9.io-scraper

If you're using nvm, you can set the node version:

nvm use

Install dependencies:

npm install

Environment Variables

Fill out .env before running the scraper:

USERNAME=user # required
PASSWORD=pass # required
DOWNLOADS_DIR=~/Downloads/ # optional

If DOWNLOADS_DIR isn't set, it defaults to ~/Downloads/.

Scripts

In the project directory, you can run:

npm start

The script logs into your c9.io account, prepares all workspaces for download, and then downloads them.

Downloaded workspaces are recorded in tmp/downloaded_workspaces.txt.

License

MIT