Skip to content

Add BigQuery description length CI safeguard#1324

Draft
seamus-mckinsey wants to merge 3 commits intotuva-health:mainfrom
seamus-mckinsey:codex/bigquery-description-length-check
Draft

Add BigQuery description length CI safeguard#1324
seamus-mckinsey wants to merge 3 commits intotuva-health:mainfrom
seamus-mckinsey:codex/bigquery-description-length-check

Conversation

@seamus-mckinsey
Copy link
Copy Markdown
Contributor

@seamus-mckinsey seamus-mckinsey commented Apr 24, 2026

Summary

  • add a lightweight Python scanner for dbt schema YAML column descriptions using PyYAML
  • add a pull_request/push workflow that installs PyYAML and runs the scanner for models, seeds, and snapshots
  • keep the workflow warning-only for now because the repo currently has existing over-limit descriptions

Why

This addresses the BigQuery field-description length regressions surfaced in #1323 by adding a durable CI safeguard so new over-limit metadata does not slip in unnoticed.

Validation

  • python3 -m unittest scripts/test_parse_ci_command.py
  • python3 -m venv /tmp/tuva-pyyaml-check && /tmp/tuva-pyyaml-check/bin/python -m pip install pyyaml
  • /tmp/tuva-pyyaml-check/bin/python -m unittest scripts/test_check_metadata_description_length.py
  • /tmp/tuva-pyyaml-check/bin/python scripts/check_metadata_description_length.py --limit 1024
  • ruby -e 'require "yaml"; Dir[".github/workflows/*.yml"].sort.each { |path| YAML.load_file(path) }'

Current violations

The scanner currently finds 10 existing descriptions over 1024 characters in input-layer YAML files. Once those are cleaned up, the workflow can be made required by removing continue-on-error.

@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 24, 2026

Deploy Preview for thetuvaproject canceled.

Name Link
🔨 Latest commit db37116
🔍 Latest deploy log https://app.netlify.com/projects/thetuvaproject/deploys/69eb396059fdea00072ff7b6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 👀 Ready for Review

Development

Successfully merging this pull request may close these issues.

1 participant