BigQuery - Revefi Roles

Grant Service Account Access to Project(s) - version

If you have one (or few) BigQuery projects, Revefi recommends to use the simplied version listed here

For fine control roles setup, please follow these steps.

All three roles are required for advanced roles setup.

Role: RevefiBigQueryJobExecutionRole

Revefi requires a context BigQuery project to connect to your instance of BigQuery. The context project is used as the project for billing (cost attribution to Revefi usage).

Create a role named RevefiBigQueryJobExecutionRole and add the following permissions

bigquery.jobs.create
bigquery.jobs.listAll
bigquery.readsessions.create
bigquery.readsessions.getData

Use the following command to create the role. Replace the <ORGANIZATION_ID> with the organization id.

gcloud iam roles create RevefiBigQueryJobExecutionRole --organization=<ORGANIZATION_ID> --title=RevefiBigQueryJobExecutionRole --description="Revefi role for context project" --permissions=bigquery.jobs.create, bigquery.jobs.listAll, bigquery.readsessions.create, bigquery.readsessions.getData

Role: RevefiBigQueryProjectMetadataRole

Create a role named RevefiBigQueryProjectMetadataRole for all the projects you want Revefi to monitor. Add the following permissions to the role

bigquery.datasets.get
bigquery.datasets.getIamPolicy
bigquery.routines.get
bigquery.routines.list
bigquery.tables.get
bigquery.tables.getIamPolicy
bigquery.tables.list
bigquery.jobs.listAll
datalineage.events.get
datalineage.locations.searchLinks
resourcemanager.projects.get

Use the following command to create the role. Replace the <ORGANIZATION_ID> with the organization id.

gcloud iam roles create RevefiBigQueryProjectMetadataRole  --organization=<ORGANIZATION_ID> --title=RevefiBigQueryProjectMetadataRole --description="Revefi role for project metadata access" --permissions=bigquery.datasets.get, bigquery.datasets.getIamPolicy, bigquery.routines.get, bigquery.routines.list, bigquery.tables.get, bigquery.tables.getIamPolicy, bigquery.tables.list, bigquery.jobs.listAll, datalineage.events.get, datalineage.locations.searchLinks, resourcemanager.projects.get

Role: RevefiBigQueryReservationMetadataRole

Create a role named RevefiBigQueryReservationMetadataRole for the project that manages the reservations. Add the following permissions to the role

bigquery.capacityCommitments.list
bigquery.reservationAssignments.list
bigquery.reservations.list

Use the following command to create the role. Replace the <ORGANIZATION_ID> with the organization id.

gcloud iam roles create RevefiBigQueryProjectMetadataRole --organization=<ORGANIZATION_ID> --title=RevefiBigQueryProjectMetadataRole --description="Revefi role for project reservation access" --permissions=bigquery.capacityCommitments.list, bigquery.reservationAssignments.list, bigquery.reservations.list

Bind Roles to the service account

Bind the roles defined above to your service account. Replace the <ORGANIZATION_ID>, <CONTEXT_PROJECT_ID>, <PROJECT_ID>, <RESERVATION_PROJECT_ID>, <SERVICE_ACCOUNT_EMAIL> placeholders respectively.

Note: Use the service account email created for Revefi.

gcloud projects add-iam-policy-binding <CONTEXT_PROJECT_ID> --member="serviceAccount:<SERVICE_ACCOUNT_EMAIL>" --role="organizations/<ORGANIZATION_ID>/roles/RevefiBigQueryJobExecutionRole"

Note: Run this command for ALL project ids you want Revefi to monitor.

gcloud projects add-iam-policy-binding <PROJECT_ID> --member="serviceAccount:<SERVICE_ACCOUNT_EMAIL>" --role="organizations/<ORGANIZATION_ID>/roles/RevefiBigQueryProjectMetadataRole"

Run this command for the reservation project

gcloud projects add-iam-policy-binding <RESERVATION_PROJECT_ID> --member="serviceAccount:<SERVICE_ACCOUNT_EMAIL>" --role="organizations/<ORGANIZATION_ID>/roles/RevefiBigQueryReservationMetadataRole"