Cloud Data Ingestion + Health Data

Increase health cost transparency with payor data aggregation

Ingest and unify large scale health payor data to increase healthcare cost transparency for patients
The challenge

Our client has a platform that allows providers to share in-network and out-of-network pricing with patients for laboratory tests at the point of care. Because of the new Transparency in Coverage, payors are now required to publish cost information via machine-readable JSON files. ‍Our client needs to incorporate the data in these files into their platform for the most up-to-date cost and coverage information, for each plan, for each geography, for each provider, and for each CPT code. The primary challenge involves engineering a data processing pipeline to efficiently process and update hundreds of terabytes of data from large one-line files.

Our Solution

We designed a unified data model using Snowflake to support the taxonomy of the payor data. We developed a data ingest pipeline that uses cloud computing to preprocess the files in parallel, then ingest and store into a data warehouse.‍We then developed optimized queries and views to retrieve information. This pipeline enables the client to access valuable healthcare cost data to provide to their customers, enabling healthcare consumers to make informed decisions on their choice of providers, and avoid unexpected medical bills.

More Case Studies ›

Advance Your Mission
Partner with us to advance your life science mission with our unparalleled expertise in biology & data science.