Skip to content

Support for schema on the fly to unlock bigquery Write API #192

@psarka

Description

@psarka

This is probably a duplicate of #140, but with an interesting twist.

Bigquery (a gcloud data warehouse) has a WriteAPI, that is a major pain to use. Not only you need to send your data in protobuf format, but you also need to advertise your payload schema in protobuf :-/

Documentation of this capability is terrible, but this example is a good illustration on how it works:

https://github.com/xebia/bigquery-storage-write-api-examples/blob/main/src/bigquery_storage_write_api_examples/examples/pending_type_stream_writer_example.py

If this library supported exporting schema in protobuf, it would be a major improvement and would essentially unlock the write API for normal python users.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions