-
Notifications
You must be signed in to change notification settings - Fork 13
HAPI Bundles and Export
The two main interactions that mimic-fhir has with HAPI is bundling and exporting. The bundle step sends resources to HAPI and validates on the way. Then to get the resources out the other end, the resources are exported from HAPI.
Bundling resources helps to resolve referential integrity and accelerate the passing of resources to HAPI.
For mimic-fhir the referential integrity is important for these resources:
- Microbiology
- ObservationMicroTest
- ObservationMicroOrg
- ObservationMicroSusc
- Medication
- MedicaitonRequest
- MedicationDispense
- MedicationAdministration
Microbiology and medication resources have parent-child relationships that would fail if resources were passed individually.
Resources from mimic-fhir needed to be organized to pass efficiently to HAPI. The bundles generated were:
Bundle | Resources | Notes |
---|---|---|
patient | Patient, Encounter, Condition, Procedure | |
spec | Specimen | The specimen are sent separately from micro and labs to simplify referencing |
micro | ObservationMicroTest, ObservationMicroOrg, ObservationMicroSusc | |
lab | ObservationLabs | |
med | MedicationRequest, MedicaitonDispense, MedicationAdministration | Med bundle not currently working, medicaiton PR still in the works |
icu_base | EncounterICU, ProcedureICU, MedicationAdministrationICU | May need to separate out medadmin since it is very large and no direct relations |
icu_obs | ObservationChartevents, ObservationDatetimeevents, ObservationOutputevents | Chartevents may be separated out since they are huge and can work to optimize them sent alone |
In the py_mimic_fhir package there are 2 bundle classes that are used to assist in bundling.
- Bundle
- The bundle class creates a transaction bundle
- Lists of resources are passed to the Bundle to be added as
entry
's - Once a bundle is ready, the Bundle.request function can be used to send the bundle to HAPI
- If the bundle is invalid or there are errors in the resources, as OperationOutcome is returned with the issues
- If the bundle is valid and no errors for its resources, the valid Bundle is returned
- Bundler
- The bundler class is used to make mimic specific bundles
- A patient_id is passed into the bundler
- All resources associated with that patient are then created into the bundle groupings specified above
- Once all bundles are generated for a patient, they can be sent individually or all together to HAPI
To handle any errors in the bundles, an ErrBundle class was created. The ErrBundle class reorganizes the failed bundle, sending the issue and fhir ids to a bundle error log. These failed bundles can be rerun from this error log once the source of error has been fixed using the rerun_bundle_from_file function.
To export resources from HAPI the bulk export feature is used. The $export
keyword is used to trigger the export. The steps for export are:
- POST export request
- Example request: http://localhost:8080/fhir/$export?_type&?_typeFilter=?_profile=<profile_url>
-
type
: the resource type -
typeFilter
: the resource type followed by filter paramaters -
_profile
: One of may filter paramaters. Specifies the profile the resource is set to
-
- Response from the server will include a polling location to get the export download location
- Example request: http://localhost:8080/fhir/$export?_type&?_typeFilter=?_profile=<profile_url>
- GET exported resource download location
- Send GET request to polling location received in the initial POST request
- The request to the polling location will return either the resources download location or a message saying not ready yet
- Keep polling until resources are ready
- GET exported resources
- Using the download location URL get the resources
- The resources will come in a binary format (base64)
- Write resource to NDJSON
- Decode resources since they come in a binary format from the server
- Write all resources of the same type to the same NDJSON