Skip to content

Adding More Data Providers

Martin Roberge edited this page Jun 10, 2017 · 3 revisions

We are constantly looking for more stream gage data! Our current list of data providers is shorter than we would like it to be, although we have more than 9,000 sites!

If you know of a dataset that you would like to have added to HyroCloud, let us know! Here is how:

  1. Check to see if we already have it. Look at our list of list of data providers just to double-check that your data isn't already a part of our system.

  2. File a request. Visit our issue list and check to see if anyone else has already requested this dataset. If they have, then add your voice to their request! If nobody has requested this dataset before, then file an issue by clicking on the green 'New issue' button. Let us know as much about the dataset as possible- is there an automated data retrieval system? Can it provide the data in JSON format? Is there a page that describes how to access the data? How do you get a list of gage locations? The more information we have, the easier it is to include the data!

  3. Add the data yourself! This is our favorite method, of course! If you have a little familiarity with coding, then adding a new dataset might be a fun way to become a collaborator! On my page, "Contribute to HydroCloud", I describe how to File an issue, then Fork the Project, then Create a branch, then edit the code locally, commit your changes, and finally to make a pull request. Follow the directions below to add in the new dataset. Once everything works, we'll accept your pull request and merge your new code into the system! Thank you!

How to Add a New Data Provider

There are two types of data that we need from each provider: time series data of stream discharge and a list of stream gage stations.

Stream Gage Stations

HydroCloud maintains a list of every stream gage station that it can access. Ideally, we need the following items for every station in our system:

  • a station name that includes the river ("Salt River at Morgantown")
  • a station ID code that we can use to request data from the new data provider
  • Latitude and Longitude (decimal degrees in WGS84 is preferable but not a deal-breaker!)

Ideally, we will need code that will produce a Comma Separated Value (CSV) file of this list automatically whenever a user requests it. Create a new entry for the new data provider on the providerList, located in src/dataProviders.js. The REST URL that retrieves this data should be added to the siteURL attribute of your new entry in the providerList. The function for parsing the site data into a CSV file should be named something like processXxxxStations(), where X is the provider name. The name of this function should be added to the siteParse attribute of your provider's entry in providerList. Add a link to station-list.html so users can request a copy of the list in CSV format.

Once we have this working, we can add the new lines from the CSV file to our FusionTable of sites.

Requesting Stream Discharge

The next step is to create a system for requesting stream discharge data from the new data provider. To do this, we will need a function that can compose the URL for making the request (dischargeXxxUrl), and another function that parses the returned data into the format that HydroCloud uses (parseXxxxDischarge). These functions get placed in dataProviders.js, and should be added to the new provider's entry in the providerList using the dischargeURL and dischargeParse attributes.

All of this seems complicated to explain, but follow the examples provided, send me an email, and before long your data will be added to the HydroCloud system!

Thank you!!