Skip to content

Add pdc calculation to pharmacy models#770

Open
saywurdson wants to merge 32 commits intotuva-health:mainfrom
saywurdson:saywurdson/pdc
Open

Add pdc calculation to pharmacy models#770
saywurdson wants to merge 32 commits intotuva-health:mainfrom
saywurdson:saywurdson/pdc

Conversation

@saywurdson
Copy link
Copy Markdown

@saywurdson saywurdson commented Apr 18, 2025

Describe your changes

This PR adds new capabilities to the Tuva healthcare data framework for calculating medication adherence metrics, specifically the Proportion of Days Covered (PDC) following PQA (Pharmacy Quality Alliance) methodology.

###Models
This PR creates 3 pharmacy models that calculate PDC and associated metrics:

  1. pharmacy__stg_add_ingredient_concepts - Maps pharmacy claims to active ingredients via RxCUI and applies claim-level overlap adjustments
  • Performs recursive adjustment at the claim line level, then ingredient explosion
  • Ensures all ingredients from combination medications maintain synchronized coverage dates
  • Filters invalid data (NULL or zero days_supply) upfront for data quality
  • Preserves claim identifiers (claim_id, claim_line_number, data_source) for traceability
  1. pharmacy__int_calculate_sub_exposures - Groups continuous coverage periods into sub-exposures
  • Detects gaps in coverage using window functions
  • Groups continuous medication exposure periods
  • Aggregates by ingredient_rxcui to handle brand/generic continuity
  1. pharmacy__pdc - Calculates final adherence metrics
  • Applies 30-day persistence window to combine sub-exposures within the same treatment era
  • Calculates total days covered, gap days, and PDC percentage
  • Ingredient-level aggregation: Groups by ingredient_rxcui only, using min(ingredient_name) to handle name variations

How has this been tested?

I created the models in my local tuva instance using duckdb as a database. Code ran with no errors

Reviewer focus

I would like to validate that the logic used to calculate the drug eras/treatment periods makes sense. I think my logic is solid but there may be others who have done this calculation that have better insight.

Checklist before requesting a review

  • I have added at least one Github label to this PR (bug, enhancement, breaking change,...) -- This is an enhancement, not sure how to add the Label
  • My code follows style guidelines
  • (New models) YAML files are categorized by sub folder and models listed in alphabetical order
  • (New models) I have added a config to each new model to enable it for claims and/or clinical data
  • (New models) I have added the variable tuva_last_run to the final output -- added to the final pdc model table
  • (Optional) I have recorded a Loom to explain this PR

Package release checklist

  • I have updated dbt docs
  • I have updated the version number in the dbt_project.yml

(Optional) Gif of how this PR makes you feel

Loom link

Summary by CodeRabbit

  • New Features
    • Added Pharmacy Drug Continuity (PDC) metric calculation to measure medication adherence per drug era across a 30-day window.
    • Added drug exposure grouping to identify continuous medication periods and calculate gap days.
    • Added ingredient-level tracking for pharmacy claims to support detailed medication analysis.
    • Added product-to-ingredient reference mapping for comprehensive medication identification.

@davidshimamoto davidshimamoto added community Issues created by community members enhancement New feature or request labels May 8, 2025
@davidshimamoto
Copy link
Copy Markdown
Contributor

@saywurdson do you have any published documentation on the process or logic that you followed to develop this? This would help me validate the work as well as provide instruction for future maintenance of this algorithm.

@github-project-automation github-project-automation Bot moved this to 👀 Ready for Review in The Tuva Project May 8, 2025
@saywurdson
Copy link
Copy Markdown
Author

saywurdson commented May 9, 2025

Hi @davidshimamoto
I've attached a paper about how the Pharmacy Quality Alliance defines the PDC calculation. There is also an example of SAS code that they used as their logic.
PDC_as_a_Preferred_Method_of_Measuring_Medication_Adherence.pdf. I am attempting to provide a dbt version that achieves the same logical outcome of their SAS example.

I also used some of the logic from the OHDSI drug era table implementation to understand how to consolidate medication records into continuous medicaiton treatment periods while factoring in some real-life medication refill scenarios like gap periods between fills and overlapping fills where a patient gets a refill but still has some medication from the previous fill left. Here is a link to their code: https://github.com/OHDSI/ETL-CMS/blob/master/SQL/create_CDMv5_drug_era_non_stockpile.sql

Please let me know if you need anything else!

P.S. I also have the python code I used to create the seed file for map NDCs/product-level RXCUIs to the active ingredient level RXCUIs to consolidate different medications that have the same active ingredient. I didn't think it fit into this PR but I'm happy to share it with you if it will help with ongoing maintenance

@sarahmcmorgan sarahmcmorgan moved this from 👀 Ready for Review to 🚧 Needs Scope in The Tuva Project May 20, 2025
@jansenpa jansenpa moved this from 🚧 Needs Scope to 👍 Ready to Pull in The Tuva Project Jun 12, 2025
@jansenpa jansenpa removed the status in The Tuva Project Jun 12, 2025
@jansenpa jansenpa moved this to 👍 Ready to Pull in The Tuva Project Jun 12, 2025
@kristinashikhrakarmaitri kristinashikhrakarmaitri moved this from 👍 Ready to Pull to 🏗 In Progress in The Tuva Project Jun 24, 2025
@sarahmcmorgan sarahmcmorgan added the stale No recent updates or commits label Aug 25, 2025
@Bijaya-Maitri Bijaya-Maitri moved this from 🏗 In Progress to ✅ Done in The Tuva Project Sep 18, 2025
@Bijaya-Maitri Bijaya-Maitri moved this from ✅ Done to 👀 Ready for Review in The Tuva Project Sep 18, 2025
@Bijaya-Maitri Bijaya-Maitri moved this from 👀 Ready for Review to 🏗 In Progress in The Tuva Project Sep 22, 2025
@caleb-palmer
Copy link
Copy Markdown

I noticed that PQA is being used to define adherence. Have any considerations been made for how combination medications are handled in the algorithm? PQA requires that a subsequent claim's supply begin on the first day of no supply from any shared active ingredient. This can result in scenarios where a single-ingredient product can affect a subsequent fill date of a different single-ingredient medication due to the presence of a combination medication, such as A >> AB >> B.

This is relevant for quality metric measures such as PQA:PDC-DR and PQA:PDC-ARV.

@Bijaya-Maitri Bijaya-Maitri moved this from 🏗 In Progress to 👀 Ready for Review in The Tuva Project Oct 10, 2025
Refactor SQL logic to adjust overlapping drug exposures and calculate sub-exposures based on adjusted start and end dates.
@coderabbitai

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

Updated SQL script to include therapy grouping key in claim line processing and adjustments.
coderabbitai[bot]

This comment was marked as outdated.

Refactor claim line processing to include ingredient-based therapy keys and improve data quality checks.
Refactor SQL logic to streamline sub-exposure calculations by removing gap detection and group ID assignment steps.
Updated event_date handling and comments in pharmacy__pdc.sql.
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between faf106b and 6a4fe41.

📒 Files selected for processing (2)
  • models/pharmacy/intermediate/pharmacy__int_calculate_sub_exposures.sql (1 hunks)
  • models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • models/pharmacy/intermediate/pharmacy__int_calculate_sub_exposures.sql

Comment thread models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
models/pharmacy/final/pharmacy__pdc.sql (1)

12-17: 30‑day persistence window fix looks correct; consider parameterizing the window.

You retained the padded end_date (no -30) and add +30 at end events. This resolves the earlier cancellation bug and yields correct era bridging at the 30‑day boundary. To improve flexibility, make the 30 a configurable var.

-            -- End events padded by 30 days for the grace period
-            -- This +30 days creates the persistence window
+            -- End events padded by a configurable grace period (default 30 days)
+            -- This creates the persistence window
             select
                 person_id,
                 ingredient_rxcui,
                 ingredient_name,
-                {{ dbt.dateadd('day', 30, 'drug_sub_exposure_end_date') }} as event_date,
+                {{ dbt.dateadd('day', var('pdc_persistence_window_days', 30), 'drug_sub_exposure_end_date') }} as event_date,
                 1 as event_type,
                 null as start_ordinal

Also applies to: 49-56

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6a4fe41 and 2211b76.

📒 Files selected for processing (1)
  • models/pharmacy/final/pharmacy__pdc.sql (1 hunks)

Comment thread models/pharmacy/final/pharmacy__pdc.sql
Comment thread models/pharmacy/final/pharmacy__pdc.sql
Refactor SQL logic for processing claim lines and ingredients, including adjustments to date calculations and grouping logic.
@saywurdson saywurdson marked this pull request as draft October 16, 2025 03:04
@saywurdson saywurdson marked this pull request as ready for review October 16, 2025 13:45
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
models/pharmacy/final/pharmacy__pdc.sql (1)

121-123: Fix macro namespace for cross-database casting (build blocker).

type_float() is not namespaced; use dbt.type_float() to avoid "macro not found" errors.

Apply:

-            then (cast(total_days_exposed as {{ type_float() }}) / cast(era_duration_in_days as {{ type_float() }})) * 100
+            then (
+                cast(total_days_exposed as {{ dbt.type_float() }}) /
+                cast(era_duration_in_days as {{ dbt.type_float() }})
+            ) * 100
🧹 Nitpick comments (1)
models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql (1)

96-123: Nested dateadd expression is correct but brittle; simplify for readability.

The Jinja string concatenation inside dbt.dateadd works but is hard to maintain. Compute adjusted_start_date as you do, then derive adjusted_end_date from that in an outer SELECT to avoid repeating the expression.

Example pattern:

select
  *,
  {{ dbt.dateadd('day', 'days_supply - 1', 'adjusted_start_date') }} as adjusted_end_date
from (
  -- current recursive select computing adjusted_start_date
) s
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2211b76 and 385e6c3.

📒 Files selected for processing (2)
  • models/pharmacy/final/pharmacy__pdc.sql (1 hunks)
  • models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql (1 hunks)
🔇 Additional comments (5)
models/pharmacy/final/pharmacy__pdc.sql (2)

65-86: Era construction logic looks solid (30‑day persistence, pairing, de-dup name).

Correct pairing with padded end events, proper join to earliest closing end, and aggregation avoids splitting by name.

Also applies to: 88-105


1-3: Validate enabling flag aligns with pharmacy models.

The model is gated by brand_generic_enabled (falling back to claims/tuva_marts). Confirm this is the intended flag for pharmacy models.

Would you prefer a dedicated pharmacy_enabled var to decouple from brand/generic gating?

models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql (3)

133-149: Good: claim-level synchronization across combo ingredients.

Taking MAX adjusted_start_date per claim line and recomputing end enforces synchronized coverage for combination fills as required by PQA.

Also applies to: 151-165, 166-185


49-51: Inner join drops unmapped NDCs; confirm intent.

Unmapped products are excluded entirely. If seeds are incomplete, this silently undercounts exposure. Consider LEFT JOIN + filter to logged exceptions, or assert mapping coverage upstream.

I can add an exceptions CTE that surfaces unmapped NDCs with counts by data_source/person for QA if desired.


35-71: Ingredient-level partitioning/order looks right for sequential adjustment.

Partition by person_id, ingredient_rxcui and tie-break ordering ensures deterministic sequencing.

Comment thread models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql Outdated
@saywurdson
Copy link
Copy Markdown
Author

Refactored the entire pdc calculation process. There were a lot of problems with my initial approach and this should more aligned with PQA methodology

Copy link
Copy Markdown

@caleb-palmer caleb-palmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this recursion still needs more refinement.

Since the logic is fundamentally adjusting claims using ingredient relationships, I suggest creating a node graph, where each row is a claim:ingredient, and includes a reference to a parent claim if it shares an ingredient. You could then implement recursion by joining on parent_claim = child_claim, with the recursion beginning with claims that have no "parent". I can mock up an example if that would be helpful.

Comment thread models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql Outdated
Comment thread models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql Outdated
@saywurdson saywurdson marked this pull request as draft October 16, 2025 21:08
@saywurdson saywurdson marked this pull request as ready for review October 19, 2025 00:00
Copy link
Copy Markdown

@caleb-palmer caleb-palmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work! The process you put together appears to be incredibly robust. I will need to spend some time testing it with mock data. Could you provide some context for my questions?

Comment thread models/pharmacy/staging/pharmacy__stg_add_ingredient_concepts.sql Outdated
Comment thread models/pharmacy/intermediate/pharmacy__int_calculate_sub_exposures.sql Outdated
Comment thread models/pharmacy/intermediate/pharmacy__int_calculate_sub_exposures.sql Outdated
Comment thread models/pharmacy/intermediate/pharmacy__int_calculate_sub_exposures.sql Outdated
Comment thread models/pharmacy/intermediate/pharmacy__int_calculate_sub_exposures.sql Outdated
Comment thread models/pharmacy/intermediate/pharmacy__int_calculate_sub_exposures.sql Outdated
Refactor SQL logic to enhance clarity and maintain granularity in claims processing. Adjustments include changing recursive CTEs to LAG for prior exposures and ensuring all ingredients from the same claim line share adjusted dates.
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 7, 2026

Workflow has finished with the following statuses:

  • Linter: success
  • DuckDB Clinical and Claims: failure
  • DuckDB Claims: skipped
  • DuckDB Clinical: skipped
  • BigQuery Clinical and Claims: failure
  • BigQuery Claims: skipped
  • BigQuery Clinical: skipped
  • Databricks Clinical and Claims: failure
  • Databricks Claims: skipped
  • Databricks Clinical: skipped
  • Fabric Clinical and Claims: failure
  • Fabric Claims: skipped
  • Fabric Clinical: skipped
  • Redshift Clinical and Claims: failure
  • Redshift Claims: skipped
  • Redshift Clinical: skipped
  • Snowflake Clinical and Claims: failure
  • Snowflake Claims: skipped
  • Snowflake Clinical: skipped
  • Snowflake Semantic Layer: skipped

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community Issues created by community members enhancement New feature or request stale No recent updates or commits

Projects

Status: 👀 Ready for Review

Development

Successfully merging this pull request may close these issues.

9 participants