Tools for processing traceroute data from CAIDA's Ark project.
Tools in the ark-tools folder were provided by CAIDA's Young Hyun.
mkvirtualenv ark
pip install -r requirements.txt
cd ark-tools
gem install rb-asfinder-0.10.1.gem rb-wartslib-1.4.2.gem
cd ..
This project also requires a running, local instance of Postgres with a no-password user named ark who owns a geo-enabled database named ark:
createdb -O ark ark
psql -q ark -c "CREATE EXTENSION postgis;"
This script uses data from CAIDA's Ark IPv4 Routed /24 Topology Dataset. The following script will will download all data, for all three monitoring teams, for every day in March of 2014. Caution: This is 87GB of data. The script can be stopped and started without starting over.
./fetch.sh
You will also need to download the following files to the root project directory:
Warning: This script will run for around a half hour per day of data. If you're loading a month of data it could easily take a full day. (Assuming you even have the disk space to hold it all.)
./process.py
cat by_country.sql | psql -q ark
cat by_monitor.sql | psql -q ark
You can pass a trace from the database into parse_trace.py to generate detailed path data in CSV format:
./parse_trace.py "216.66.30.102:6939,216.66.30.101:6939,213.248.67.125:1299,213.155.130.34:1299,157.130.60.13:701,:q,:r,108.51.141.48:701" > nyc_to_dc.csv