This repository includes pipelines to transform data from a FHIR server (like HAPI, GCP FHIR store, or even OpenMRS) using the FHIR format into a data warehouse based on Apache Parquet files, or another FHIR server. There is also a query library in Python to make working with FHIR-based data warehouses simpler.
These tools are intended to be generic and eventually work with any FHIR-based data source and data warehouse. Here is the list of main directories with a brief description of their content:
-
pipelines/ *START HERE*: Batch pipelines to transform data from a FHIR-based source to an analytics-friendly data warehouse or another FHIR store.
-
docker/: Docker configurations for various servers/pipelines.
-
doc/: Documentation for project contributors. See the pipelines README and wiki for usage documentation.
-
utils/: Various artifacts for setting up an initial database, running pipelines, etc.
-
dwh/: Query library for working with distributed FHIR-based data warehouses.
-
bunsen/: A fork of a subset of the Bunsen project which is used to transform FHIR JSON resources to Avro records with SQL-on-FHIR schema.
-
e2e-tests/: Scripts for testing pipelines end-to-end.
NOTE: This was originally started as a collaboration between Google and the OpenMRS community.
This is an open-source project and we welcome contributions. Our development process is also open. We use GitHub issues and review processes. If you are working on a feature or an issue, please comment on that GitHub issue to communicate with other developers. It would also be great to attend our weekly developers' calls on Tuesdays; here are the details:
- When: Tuesdays at 10 AM Eastern Time (Toronto)
- Where: Virtual at https://meet.google.com/ago-nmcc-oed