Updates for August 2020 release

Main new functionality: multi-dimensional regional analyses

README.md

@@ -1,74 +1,76 @@
 # Conveyal Analysis
 
-This is the server component of [Conveyal Analysis](http://conveyal.com/), which allows users to create transportation scenarios and evaluate them in terms of cumulative opportunities accessibility indicators. 
+This is the server component of [Conveyal Analysis](http://conveyal.com/analysis), which allows users to create public transport scenarios and evaluate them in terms of cumulative opportunities accessibility indicators. 
 
-**Please note** that the Conveyal team does not provide technical support for third-party deployments. We provide paid subscriptions to a cloud-based deployment of this system, which performs these complex calculations hundreds of times faster using a compute cluster. This project is open source primarily to ensure transparency and reproducibility in public planning and decision making processes, and in hopes that it may help researchers, students, and potential collaborators to understand and build upon our methodology.
-
-## R<sup>5</sup> Routing Engine: Rapid Realistic Routing on Real-world and Reimagined networks
-
-Conveyal Analysis relies on R<sup>5</sup>, our routing engine for multimodal (transit/bike/walk/car) networks. It is intended primarily for analysis applications (one-to-many trees, travel time matrices, and cumulative opportunities accessibility indicators). R<sup>5</sup> is included as a module in this repository. Prior to June 2020, it was developed in a [separate repository](https://github.com/conveyal/r5).
-
-We refer to the routing method as "realistic" because it works by planning many trips at different departure times in a time window, which better reflects how people use transportation system than planning a single trip at an exact departure time. 
-
-We say "Real-world and Reimagined" networks because R5's networks are built from widely available open OSM and GTFS data describing baseline transportation systems, but R5 includes a system for applying light-weight patches to those networks for immediate, interactive scenario comparison.
-
-## Methodology
-
-For details on the core methods implemented in Conveyal Analysis and R<sup>5</sup>, see:
-
-* [Conway, Byrd, and van der Linden (2017)](http://hdl.handle.net/2286/R.A.218654)
-* [Conway, Byrd, and van Eggermond (2018)](https://www.jtlu.org/index.php/jtlu/article/view/1074)
-* [Conway and Stewart (2019)](https://files.indicatrix.org/Conway-Stewart-2019-Charlie-Fare-Constraints.pdf)
-
-### Citations
-
-The Conveyal team is always eager to see cutting-edge uses of our software, so feel free to send us a copy of any thesis, report, or paper produced using this software. We also ask that any academic publications using this software cite the papers above, where relevant and appropriate.
+**Please note** that Conveyal does not provide technical support for third-party deployments of Analysis. We provide paid subscriptions to a cloud-based deployment of this system which performs these complex calculations hundreds of times faster using a compute cluster. This project is open source primarily to ensure transparency and reproducibility in public planning and decision making processes, and in hopes that it may help researchers, students, and potential collaborators to understand and build upon our methodology.
 
 ## Configuration
 
-It is possible to run a Conveyal Analysis UI and backend locally (e.g. on your laptop), which should produce results identical to those from our hosted platform. However, the computations for more complex analyses may take quite a long time. Extension points in the source code allow the system to be tailored to cloud computing environments to enable faster parallel computation.
-
-### Running Locally
-
+Conveyal Analysis can be run locally (e.g. on your laptop) or on Amazon Web Services EC2 instances, depending on the 
+configuration set in analysis.properties.  With extensions requiring software development skills, it could be modified 
+to run in other cloud-computing environments.
+                                           
 To get started, copy the template configuration (`analysis.properties.tmp`) to `analysis.properties`.  
 
-To run locally, use the default values in the template configuration file. `offline=true` will create a local instance 
-that avoids cloud-based storage, database, or authentication services. By default, analysis-backend will use the `analysis` database in a local MongoDB instance, so you'll also need to install and start a MongoDB instance.
+To run locally, use the default values in the template configuration file. `offline = true` will create a local instance 
+that avoids cloud-based storage, database, or authentication services.  To run regional accessibility analyses (as 
+opposed to single-point isochrone analyses), you will need to set up an AWS S3 bucket and set the value of `results_bucket`.
+By default, analysis-backend will use the `analysis` database in a local MongoDB instance, so you'll also need to 
+install and start a MongoDB instance.
 
-Database configuration variables include:
+To run on the cloud, we use Auth0 for authentication and S3 for storage; configure these services as needed, then set 
+the corresponding variables including:
 
+- `auth0-client-id`: your Auth0 client ID
+- `auth0-secret`: your Auth0 client secret
 - `database-uri`: URI to your Mongo cluster
-- `database-name`: name of the database to use in your Mongo cluster
+- `database-name`: name of project database in your Mongo cluster
+- `frontend-url`: URL of the analysis-ui frontend (see below)
+- `bundle_bucket`: S3 bucket for storing GTFS bundles and built transport networks
+- `grid_bucket`: S3 bucket for storing opportunity dataset grids
+- `results_bucket`: S3 bucket for storing regional analysis results
+
+You will need S3 credentials set up in your environment or in `~/.aws` for an identity that is allowed to access all the 
+buckets above. If you have multiple profiles, you can use the `AWS_PROFILE` variable in the environment to choose which 
+AWS credentials profile will be used.
 
 ## Building and running
 
 Once you have configured `analysis.properties` and started mongo locally, build the application with `mvn package` and 
-start it with `java -Xmx2g -cp target/shaded/vX.Y.Z.jar com.conveyal.analysis.BackendMain`
+start it with `java -Xmx2g -jar target/analysis.jar`
 
-Next, follow the instructions to start the [analysis-ui frontend](https://github.com/conveyal/analysis-ui). Once that 
-is running, you should be able to log in without authentication (using the frontend URL, e.g. http://localhost:3000). 
+Next, follow the instructions to start the [analysis-ui frontend](https://github.com/conveyal/analysis-ui) . Once that 
+is running, you should be able to log in without authentication (using the frontend URL, e.g. http://localhost:9966). 
+Note that the default value of the analysis-backend `frontend-url` variable is a prebuilt copy of the frontend that 
+relies on Conveyal's authentication setup; in general this will not work with local installations, so this value should 
+be ignored.
 
 ## Creating a development environment
 
-In order to do development on the frontend or backend, you'll need to set up a local development environment. We use [IntelliJ IDEA](https://www.jetbrains.com/idea/). The free/community edition is sufficient for working on Conveyal Analysis. Add analysis-backend to IntelliJ as a new project from existing sources. You can then create a run configuration for `com.conveyal.analysis.BackendMain`, which is the main class. You will need to configure the JVM options and properties file mentioned above.
+In order to do development on the frontend, backend, or on [R5](https://github.com/conveyal/r5), which we use for
+performing the analyses, you'll want a local development environment. We use [IntelliJ IDEA](https://www.jetbrains.com/idea/)
+(free/community version is fine) and add analysis-backend as a new project from existing sources. We also typically clone
+R5 with `git`, then use the green plus button in the Maven panel to add R5 as a Maven Project within the same IntelliJ project. 
+Check to make sure that the version of R5 matches the version specified in the analysis-backend `pom.xml`.  
+
+You can then create a run configuration for `com.conveyal.analysis.BackendMain`, which is the main class. You will need to
+configure the options mentioned above.
 
 ## Structured Commit Messages
 
 We use structured commit messages to allow automated tools to determine release version numbers and generate changelogs.
 
 The first line of these messages is in the following format: `<type>(<scope>): <summary>` 
 
-The `(<scope>)` is optional and is often a class name. The `<summary>` should be in the present tense. The type should be one of the following:
+The `(<scope>)` is optional. The `<summary>` should be in the present tense. The type should be one of the following:
 
 - feat: A new feature from the user point of view, not a new feature for the build.
 - fix: A bug fix from the user point of view, not a fix to the build.
 - docs: Changes to the user documentation, or to code comments.
 - style: Formatting, semicolons, brackets, indentation, line breaks. No change to program logic.
 - refactor: Changes to code which do not change behavior, e.g. renaming a variable.
 - test: Adding tests, refactoring tests. No changes to user code.
-- build: Updating build process, scripts, etc. No changes to user code.
-- devops: Changes to code that only affect deployment, logging, etc. No changes to user code.
-- chore: Any other changes causing no changes to user code.
+- chore: Updating build process, scripts, etc. No changes to user code.
 
 The body of the commit message (if any) should begin after one blank line. If the commit meets the definition of a major version change according to semantic versioning (e.g. a change in API visible to an external module), the commit message body should begin with `BREAKING CHANGE: <description>`.
 

docs/fares/index.md

@@ -0,0 +1,12 @@
+# In-routing fare support
+
+Conveyal Analysis supports computing fares within the routing algorithm, which allows the routing algorithm to find not only the fastest path, but all Pareto-optimal tradeoffs between travel time and fare (the cheapest route is always to walk, unless free transit is available, so generally just finding the cheapest route is undesirable). For instance, here are the Pareto-optimal options for travel from Salem, Massachusetts to Copley Square in Boston:
+
+![Example Pareto surface for travel from Salem to Copley. The fastest option is to take the Newburyport/Rockport line followed by the Green Line. Slightly cheaper and quite a bit longer is to take the Newburyport/Rockport line followed by two buses. Much cheaper is an express bus followed by the green line, and cheapest is a bus to the blue line to the orange line. Each subsequent option takes longer than the last.](fareto.png)
+
+Finding cheapest paths is implemented in the McRAPTOR (multi-criteria RAPTOR) router, since this is a multi-objective optimization problem. Correctly finding cheapest paths is a challenging problem, due to discounted transfers. It may make sense to take a more expensive and/or slower option at the start of your trip, if it provides a discounted transfer to a service you will take later. However, since the router does not know what transit routes will be ridden in later rides, this presents a challenge. We introduce a "transfer allowance" that tracks these possible discounts; full details are available in [this IJGIS paper](https://files.indicatrix.org/Conway-Stewart-2019-Charlie-Fare-Constraints.pdf).
+
+Unfortunately, while there is a common data format for transit timetables (GTFS), no such format exists for fares. GTFS does include two different fare specifications (GTFS-fares and GTFS-fares v2), but they are not able to represent complex fare systems. As such, unless and until such a specification becomes available, Conveyal Analysis includes location-specific fare calculators for a number of locations around the world. They have their own documentation:
+
+- [New York](newyork.html)
+- Boston (documentation coming soon)