Lantern

Lantern is an open source tool developed by the Office of the National Coordinator for Health Information Technology (ONC) and the MITRE Corporation that monitors and provides analytics about the availability and adoption of FHIR API service base URLs (endpoints) across healthcare organizations in the United States. It also gathers information about FHIR Capability Statements returned by these endpoints and provides visualizations to show FHIR adoption and patient data availability. For more information, check out the “About Lantern” page on our website, https://www.lantern.healthit.gov.

Index

Running Lantern - Basic Flow
Testing Lantern - Basic Flow
Make Commands
Configure Data Collection Failure System
Configure Backup System
Running Lantern Services Individually
Using Docker Compose
Testing - Details
Contributing
License

Running Lantern - Basic Flow

Note: Before running the commands below, make sure that you either:

have created a hidden file named .env in the root directory of the project containing the environment variables specified in the env.sample file.
have the variables specified in the env.sample file defined as system environment variables.

Setup your environment

To run Lantern, several environment variables need to be set. These are defined within each project's README. Each README defines the variables that must be set on the system vs those whose default values are sufficient.

Clean your environment

This is optional!

If you'd like to start with a clean slate, run:

make clean

This removes all docker images, networks, and local volumes.

Start Lantern

Production Environment

To start up Lantern with a development environment, in your terminal, run:
```
make run_prod
```
This starts all of the following services:
- PostgreSQL - application database
- LanternMQ (RabbitMQ) - the message queue (localhost:15672)
- Capability Querier - queries the endpoints for their capability statements once a day. Kicks off the initial query immediately.
- Capability Receiver - receives the capability statements from the queue, peforms validations and saves the results to fhir_endpoints_info
- Endpoint Manager - sends endpoints to the capability querying queues
Or if you wish to start up Lantern with a production environment, run:
```
make run_prod
```
If you have a clean database or want to update the data in your database
1. Run the following command with the Lantern project running to update your endpoint resource files found in lantern-back-end/resources/prod_resources. This command will automatically query all the endpoint sources listed in EndpointResourceList.json, which can be found in lantern-back-end/resources/prod_resources. It will also query CHPL for it's list of endpoint list sources.
```
 make update_source_data_prod
```
-Note: Google chrome must be installed in order to run this command and run the webscrapers needed for certain list sources.
Run the following command to query NPPES for their endpoint and npi data files and automatically populate the database with this information, as the files are too large to be persisted in our list of resources, as well as populate the database using the data found in lantern-back-end/resources/prod_resources. -Note: The NPPES npidata_pfile and endpoint_pfile are very large and therefore are not persisted in our directory of prod resources, so to add the full NPPES data into the database, you must run this make populatedb_prod command which will query NPPES for their endpoint and npi data files, cut out all the entries in the npi data file that are not organization entries, and automatically add the information to the database before deleting these large NPPES files. It will also add the data found in lantern-back-end/resources/prod_resources to the database.

The populate db prod script expects the resources directory to contain the following files:

CernerEndpointSources.json - JSON file containing endpoint information from Cerner
EpicEndpointSourcesDSTU2.json - JSON file containing DSTU2 endpoint information from Epic
EpicEndpointSourcesR4.json - JSON file containing R4 endpoint information from Epic
1UpEndpointSources.json - JSON file containing endpoint information from 1upHealth
CareEvolutionEndpointSources.json - JSON file containing endpoint information from CareEvolution
LanternEndpointSources.json - JSON file containing endpoint information reported directly to Lantern
linkerMatchesAllowlist and linkerMatchesBlocklist - allowlist and blocklist files used in manually correcting the endpoint to npi organization linker. To manually add/remove endpoint to npi organization links in the database, see endpointmanager README on format for adding links to allowlist and blocklist files

make populatedb_prod

This runs the following tasks inside the endpoint manager container:

the endpoint populator, which iterates over the list of endpoint sources and adds them to the database.
the CHPL querier, which requests health IT product information from CHPL and adds these to the database
the NPPES endpoint populator, which adds endpoint data from the monthly NPPES export to the database.
the NPPES org populator, which adds provider data from the monthly NPPES export to the database.
the data validator, which ensures that the amount of data in the database can successfully be quried in the 23 hour query interval.
the NPPES querier, which queries NPPES for their endpoint and npi data files, and automatically populates the database with this information, cuts out all the entries in the npi data file that are not organization entries, and automatically adds the information to the database before deleting these large NPPES files.

The populate db prod script expects the resources directory to contain the same files as above, besides the endpoint_pfile.csv and npidata_pfile.csv, as these are automatically queried and added to the database within this script.

make populatedb_prod

If you want to requery and rereceive capability statements outside the refresh interval run the following:
```
docker restart lantern-back-end_endpoint_manager_1
```

Development Environment

To start up Lantern with a development environment, in your terminal, run:
```
make run
```
This starts all of the following services:
- PostgreSQL - application database
- LanternMQ (RabbitMQ) - the message queue (localhost:15672)
- Capability Querier - queries the endpoints for their capability statements once a day. Kicks off the initial query immediately.
- Capability Receiver - receives the capability statements from the queue, peforms validations and saves the results to fhir_endpoints_info
- Endpoint Manager - sends endpoints to the capability querying queues

If you have a clean database or want to update the data in your database

Run the following command with the Lantern project running to update your endpoint resource files found in lantern-back-end/resources/prod_resources. This command will automatically query all the endpoint sources listed in EndpointResourceList.json, which can be found in lantern-back-end/resources/prod_resources, and it will also query CHPL for it's list of endpoint list sources. This command will also query NPPES for their endpoint and npi data files, cut out all the entries in the npi data file that are not organization entries, and then create a copy of each file and reduce them to 1000 lines for development resources and save them in lantern-back-end/resources/dev_resources Resources can be moved from lantern-back-end/resources/prod_resources to lantern-back-end/resources/dev_resources to be used in the development environment.

 make update_source_data

-Note: Google chrome must be installed in order to run this command and run the webscrapers needed for certain list sources.

Run the following command to only query the endpoint sources listed in EndpointResourceList.json, which can be found in lantern-back-end/resources/prod_resources, and CHPL for it's list of endpoint list sources. ```bash make update_source_data_prod

-Note: Google chrome must be installed in order to run this command and run the webscrapers needed for certain list sources.

2. Run the following command to begin populating the database using the data found in `lantern-back-end/resources/dev_resources`. You must be running Lantern with a development environment by using the command `make run` to start up Lantern.
-Note: Since you are doing development, use the `dev_resources` directory as it contains less endpoints which reduces unnecessary load on the servers hosting the endpoints we are querying.

The populate db script expects the resources directory to contain the following files:
* **CernerEndpointSources.json** - JSON file containing endpoint information from Cerner
* **EpicEndpointSourcesDSTU2.json** - JSON file containing DSTU2 endpoint information from Epic
* **EpicEndpointSourcesR4.json** - JSON file containing R4 endpoint information from Epic
* **1UpEndpointSources.json** - JSON file containing endpoint information from 1upHealth
* **CareEvolutionEndpointSources.json** - JSON file containing endpoint information from CareEvolution
* **LanternEndpointSources.json** - JSON file containing endpoint information reported directly to Lantern
* **endpoint_pfile.csv** - enpoint_pfile from the data dissemination package downloaded from https://download.cms.gov/nppes/NPI_Files.html
* **npidata_pfile.csv** - npidata_pfile from the data dissemination package downloaded from https://download.cms.gov/nppes/NPI_Files.html 
  * NOTE: This file can take a very long time to load so for development purposes, the load time can be reduced by only using the first 100000 entries. The first 100000 entries can be obtained by running `head -n 100000 npidata_pfile_20050523-20191110.csv >> npidata_pfile.csv`. Alternatively, running `make update_source_data` adds truncated npi files to the `dev_resources` directory as well.
* **linkerMatchesAllowlist and linkerMatchesBlocklist** - allowlist and blocklist files used in manually correcting the endpoint to npi organization linker. To manually add/remove endpoint to npi organization links in the database, see endpointmanager README on format for adding links to allowlist and blocklist files

```bash
make populatedb

This runs the following tasks inside the endpoint manager container:

the endpoint populator, which iterates over the list of endpoint sources and adds them to the database.
the CHPL querier, which requests health IT product information from CHPL and adds these to the database
the NPPES endpoint populator, which adds endpoint data from the monthly NPPES export to the database.
the NPPES org populator, which adds provider data from the monthly NPPES export to the database.
the data validator, which ensures that the amount of data in the database can successfully be quried in the 23 hour query interval.

If you want to requery and rereceive capability statements outside the refresh interval run the following:
```
docker restart lantern-back-end_endpoint_manager_1
```

Stop Lantern

Production Environment

To stop Lantern when running with a production environment, run:

make stop_prod

Development Environment

To stop Lantern when running with a development environment, run:

make stop

Starting Services Behind SSL-Inspecting Proxy

If you are operating behind a proxy that does SSL-Inspection you will have to copy the certificates that are used by the proxy into the following directories:

capabalitiyquerier/certs/
capabilitiyreceiver/certs/
endpointmanager/certs/
lanternmq/certs
shinydashboard/certs/
e2e/certs
db/migration/certs

Testing Lantern - Basic Flow

There are three types of tests for Lantern and three corresponding commands:

test type	command
unit	`make test`
integration	`make test_int`
end-to-end	`make test_e2e`
all tests	`make test_all`

Make Commands

make command	action
`make run`	runs docker-compose for a development environment
`make run_prod`	runs docker-compose for a production environment
`make stop`	runs docker-compose `down` for a development environment
`make stop_prod`	runs docker-compose `down` for a development environment
`make clean`	runs docker-compose `down` with the `--rmi local -v` tags to remove local images and volumes. Runs this for all docker-compose setups. Before running, it confirms with the user that they actually want to clean.
`make clean_remote`	runs docker-compose `down` with the `--rmi all -v` tags to remove all images and volumes. Runs this for all docker-compose setups. Before running, it confirms with the user that they actually want to clean.
`make test`	runs unit tests
`make test_int`	runs integration tests
`make test_e2e`	runs end-to-end tests
`make test_all`	runs all tests and ends if any of the tests fail
`make populatedb`	Should be used with development environment by running `make run` first. Populates the database with the endpoint resource list information and NPPES information found in the `resources/dev_resources` directory.
`make populatedb_prod`	Should be used with production environment by running `make run_prod` first. Populates the database with the endpoint resource list information found in the `resources/prod_resources` directory, and queries NPPES for its latest information and automatically stores it in the database before deleting the files.
`make backup_database`	saves a database backup .sql file in the lantern base directory with name lantern_backup_`<timestamp>`.sql
`make restore_database file=<backup file name>`	restores the backup database that the 'file' parameter is set to
`make migrate_database force_version=<migration version number to force db to>`	Starts the postgres service and runs the next `*.up.sql` migration in the `db/migration/migrations` directory that has not yet been run. Must run this command the number times equal to the number of migrations you want to run. The optional force_version parameter can be included to force the database to a specific migration version before running the next migration. If this parameter is omitted, it runs the next migration that has not yet been run.
`make update_source_data`	Automatically queries the endpoint lists listed in the EndpointResourcesList.json file found in the `resources/prod_resources` directory, including Epic, Cerner, CareEvolution, and 1UpHealth, queries the NPPES npi and endpoint data and stores truncated versions of the files in the `resources/dev_resources` directory, and queries CHPL for its list of endpoint lists and stores the data in a file in the `resources/prod_resources` directory.
`make update_source_data_prod`	Automatically queries the endpoint lists listed in the EndpointResourcesList.json file found in the `resources/prod_resources` directory, including Epic, Cerner, CareEvolution, and 1UpHealth, and queries CHPL for its list of endpoint lists and stores the data in a file in the `resources/prod_resources` directory.
`make lint`	Runs the R and golang linters
`make lint_go`	Runs the golang lintr
`make lint_R`	Runs the R lintr
`make json_export file=<export file name> exportType=<month/30days/all>`	Exports the history of the endpoint data to a JSON file specified by the 'file' parameter. This 'file' parameter must only be a file name with the appropriate `.json` file extension, not a file path. Setting exportType equal to "month" creates the export file using only the last months history data, setting it to "30days" or leaving it blank will create the export file with all the history information from the last 30 days, and setting it to "all" will create an export file using all of the history data Lantern has stored.
`make history_pruning`	Prunes the fhir_endpoint_info_history table to remove duplicate entries
`make create_archive start=<start date> end=<end date> file=<archive file name>`	Creates an archive of the data in the database between the given dates in a JSON format and saves it to the given 'file' name. The dates format is '2021-01-31' (year, month, date). Example: `make create_archive start=2020-06-01 end=2021-06-01 file=archive_file.json`. Note: If the archive period includes any time between the current date and the LANTERN_PRUNING_THRESHOLD, then the given number of updates might be higher than expected because the history pruning algorithm is only run on data older than the threshold.
`make migrate_validations direction=<up/down>`	Runs validation migrations when direction is set to up. If direction is set to down, undos validation migrations
`make migrate_resources direction=<up/down>`	Runs resources migrations when direction is set to up. If direction is set to down, undos resources migrations

Configure Data Collection Failure System

You can configure a data collection failure system using cron and the data_collection_check.sh script located in the scripts directory to send an email notification if the lantern data collection goes down for any reason.

The data_collection_check.sh script runs outside of Lantern and periodically checks to see if data has been written to the fhir_endpoints_info within the last N many minutes, where N is the Lantern query interval. If data has not been written within said interval, or the database is down, then the script sends an alert to the set email address.

To set up the script for this data collection failure system, you must insert the correct information into the following variables located at the beginning of the data_collection_check script. The DB_NAME, DB_USER, and QUERY_INTERVAL variables used in the script should match their corresponding environmental variable (shown in parentheses below) defined in the .env file:

Set the EMAIL variable to the email you want the failure system to send alerts to
Set the DB_NAME variable to name of your database (LANTERN_DBNAME)
Set the DB_USER variable to the name of the database user (LANTERN_DBUSER)
Set the QUERY_INTERVAL variable to the capability querier query interval in minutes (LANTERN_CAPQUERY_QRYINTVL)