A web application that displays opinions over time surrounding the police and law enforcement.
-
Map with data layers This feature uses government database json files to draw lines seperating zip codes and police precincts in two seperate data layers, and integrates features below based on user click in Map API. On the zip code layer it displays the zip code, neighborhood, and associated police precincts for user reference and on the precinct layer it allows sentiment mapping by using analysis done on survey responses seperated by date received and zip code association. Map API includes buttons opening seperate pages for detailed statistics analysis and word cloud based on strong negative and positive associations also.
-
Positive and negative word associations
The positive and negative associations of words within open text responses were analyzed using CloudNLP. The top positive and negative associations were displayed along with a wordcloud displaying all results (with size corresponding to prominence in the text and color corresponding to average sentiment). You can view these results for the entirety of SF or a specific precinct.
- Statistics and predictions
Fetches from the Google Surveys API, transforms data for GCP Datastore, and visualizes analysis through the Google Charts API. This feature also runs a decision tree regression model on the survey results to get predicted sentiment scores for users based on their gender, age group, and whether they had a direct experience with the police or not. The program for regression analysis was written in Python with the Flask framework. Using crom jobs and a separate python service on GAE, new prediction results can be automatically generated for new survey responses. There is a statistics page with survey-specific details such as response time and completion, age and gender break down, and sentiment percentages for each precinct.
Some of the tools that will be used to build this portfolio are
- HTML
- CSS
- Javascript
- Java
- Python
The following APIs were used
- Google Maps API
- Google Charts API
- CloudNLP (https://cloud.google.com/natural-language)
- Wordcloud2 (https://www.npmjs.com/package/wordcloud2)
- SciKit-Learn
- Google Surveys API
To deploy the main project
- Install maven (https://maven.apache.org/install.html)
- Navigate to the capstone directory.
- Modify pom.xml to contain the GCP project ID.
- Ensure that GCP Cloud Build (https://console.developers.google.com/apis/api/cloudbuild.googleapis.com) and natural language (https://console.developers.google.com/apis/library/language.googleapis.com) are enabled on the cloud project.
- Run
mvn package appengine:deployto deploy the project live ormvn package appengine:runto run the project on the local server.
Additional instructions for deploying the python service:
- Ensure that you have the requirement.txt file set up with all the relevant dependencies. Do pip freeze to get the dependencies
- Set up a new app.yaml file with the service name, runtime, static files or script specified
- Keep these files along with your source code in a separate directory
- Deploy the separate service using gcloud app deploy --project [PROJECT_ID]
- (Optional) Include a cron.yaml file with the schedule field specified
This repository runs checks on every pull request and commit. You can run these locally from the root directory of the project.
- Java Continuous Integration: Run
mvn package - Java Format: Follow the instructions at https://github.com/google/google-java-format to download the .jar. Place the jar in the top level directory of the repository. You can then run it seperately with instructions from the the formatter README (must use java 11) or run all tests together as shown below.
- JavaScript Lint:
- One time setup: run
npm install - Run
./node_modules/.bin/eslint capstone/src/.
- One time setup: run
- All tests can be run using
./lintor./lint-replaceif you want to automatically fix the errors found
This code is licensed under the Apache 2.0 License.