Add explicit units column to annotate multi-unit columns

[zaneselvans]

Overview

• PUDL has a handful of columns which contain quantities with heterogenous units, making them impossible to annotate like we did for simple columns in Add machine-readable units via pint and fix some dtypes #5361
• Most of these are columns that contain fuel quantities, with solid, liquid, and gaseous fuels having different conventional units (short_tons, oil_barrels, and Mcf).
• The FERC EQR contracts and transactions tables also have this problem, both for prices, and for quantities.
• One clear solution for this problem is to add an additional column that JUST contains machine-readable units that annotates each of the heterogeneous units columns.
• Previously this seemed relatively low value, since units were just human-readable, and we already documented the different units for different types of fuels, but now that we have machine readable units, including these annotating columns would facilitate programmatic conversions for aggregating values reported with different units.

Mixed Units Columns:

These are columns that report a physical quantity, heat content, or price/rate whose unit of measure varies row-by-row depending on the fuel, by-product, or product type — rather than a single fixed unit for the whole column. Where PUDL provides a companion column with the explicit unit for each row, it's listed in the "Disambiguating units column" field below.

These are columns that report a physical quantity, heat content, or price/rate whose unit of measure varies row-by-row depending on the fuel, by-product, or product type — rather than a single fixed unit for the whole column. Where PUDL provides a companion column with the explicit unit for each row, it's listed in the "Disambiguating units column" field below.

Fuel quantity / heat-content columns

Unit varies by fuel type: short_ton for solid fuels (coal), oil_barrel for liquid fuels (petroleum), Mcf for gaseous fuels (natural gas).

Table	Column	units column
core_eia923__fuel_receipts_costs	fuel_received_units
core_eia923__fuel_receipts_costs	fuel_mmbtu_per_unit
core_eia923__monthly_boiler_fuel	fuel_consumed_units
core_eia923__monthly_boiler_fuel	fuel_mmbtu_per_unit
core_eia923__monthly_generation_fuel	fuel_consumed_units
core_eia923__monthly_generation_fuel	fuel_consumed_for_electricity_units
core_eia923__monthly_generation_fuel	fuel_mmbtu_per_unit
core_eia923__monthly_generation_fuel_nuclear	fuel_consumed_units
core_eia923__monthly_generation_fuel_nuclear	fuel_consumed_for_electricity_units
core_eia923__monthly_generation_fuel_nuclear	fuel_mmbtu_per_unit
core_eia923__monthly_energy_storage	fuel_consumed_units	fuel_units
core_eia923__monthly_energy_storage	fuel_consumed_for_electricity_units	fuel_units
core_ferc1__yearly_steam_plants_fuel_sched402	fuel_consumed_units	fuel_units
core_ferc1__yearly_steam_plants_fuel_sched402	fuel_mmbtu_per_unit
out_eia923__fuel_receipts_costs (+ _monthly_/_yearly_ variants)	fuel_received_units, fuel_mmbtu_per_unit
out_eia923__boiler_fuel (+ _monthly_/_yearly_ variants)	fuel_consumed_units
out_eia923__generation_fuel_combined (+ _monthly_/_yearly_ variants)	fuel_consumed_units, fuel_consumed_for_electricity_units
out_ferc1__yearly_steam_plants_fuel_sched402	fuel_consumed_units, fuel_mmbtu_per_unit	fuel_units

Combustion by-product columns

Unit varies by by-product type: short_ton (to the nearest 100 short_ton) for solid by-products, MMBtu for steam sales.

Table	Column	units column
_core_eia923__yearly_byproduct_disposition	disposal_landfill_units	byproduct_units
_core_eia923__yearly_byproduct_disposition	disposal_offsite_units	byproduct_units
_core_eia923__yearly_byproduct_disposition	disposal_ponds_units	byproduct_units
_core_eia923__yearly_byproduct_disposition	sold_units	byproduct_units
_core_eia923__yearly_byproduct_disposition	stored_offsite_units	byproduct_units
_core_eia923__yearly_byproduct_disposition	stored_onsite_units	byproduct_units
_core_eia923__yearly_byproduct_disposition	total_disposal_units	byproduct_units
_core_eia923__yearly_byproduct_disposition	used_offsite_units	byproduct_units
_core_eia923__yearly_byproduct_disposition	used_onsite_units	byproduct_units

FERC EQR contract / transaction quantity and price columns

Unit varies by product type (e.g. energy in MWh, capacity in MW, gas in Dth/MMBtu). Note this package is built separately from the main PUDL parquet/ datapackage — it lives in its own ferceqr_parquet_datapackage.json.

Table	Column	units column
core_ferceqr__contracts	quantity	units
core_ferceqr__contracts	rate	rate_units
core_ferceqr__contracts	rate_minimum	rate_units
core_ferceqr__contracts	rate_maximum	rate_units
core_ferceqr__transactions	transaction_quantity
core_ferceqr__transactions	price	rate_units

Notes / exclusions

core_ferceqr__transactions.standardized_quantity / standardized_price are not mixed-unit — they're pre-normalized to MWh (energy/booked-out power) or MW-month (capacity/booked-out power), but only populated for those product types; other product types are left null. Not included above.