Skip to content

Google Cloud Storage whitelist buckets by regex#85

Open
damiano1996 wants to merge 3 commits into
dremio:masterfrom
damiano1996:feature/gcs-buckets-regex-filter
Open

Google Cloud Storage whitelist buckets by regex#85
damiano1996 wants to merge 3 commits into
dremio:masterfrom
damiano1996:feature/gcs-buckets-regex-filter

Conversation

@damiano1996

Copy link
Copy Markdown

Google Cloud Storage whitelist buckets by regex

Description

This pull request adds a new feature to Google Cloud Storage plugin that allows for the whitelisting of buckets using regular expressions. This feature can be particularly useful in a data mesh context, where it's common to segregate data domains and products based on specific naming conventions.

Before this change, whitelisting buckets had to be done manually, which was a time-consuming and error-prone task. With this new feature, users can now choose to whitelist buckets by listing their names individually or by defining a regular expression to match multiple buckets simultaneously. This makes the process of whitelisting buckets more efficient and less prone to errors, which is especially important in a data mesh context where data quality and governance are critical.

Examples

Here are some examples of how this new feature could be used:

  • Whitelist all buckets that contain the word "sales": ^sales-.*
  • Whitelist all buckets that belong to a specific domain: ^mydomain-.*
  • Whitelist all buckets that start with a specific date: ^2023-03-.*

Screenshots

The screenshot below shows the classic option:

dremio_01_list

This one shows the new option with the regex:

dremio_02_regex.

@pavloseroAgile

pavloseroAgile commented Jul 9, 2024

Copy link
Copy Markdown

It looks very useful, we really need this feature!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants