Cloud – Ryan Chapin's Website

List the Roles for a User or Service Account in a Specific GCP Project

Posted on May 27, 2022 by rchapin

If you do not have web console permissions to do so, but have the ability to activate a service account that has the viewer permissions or IAM permissons to list IAM roles in a given project, the following is how you can list the roles for a given user or service account.

gcloud projects get-iam-policy <gcp-project\
--filter="bindings.members:<email-address" \
--flatten="bindings[].members" --format="table(bindings.role)"

→ Continue reading

Using bq load Command to Load logicalType Partitioned Data into a BigQuery Table

Posted on January 27, 2022 by rchapin

Following is the syntax and bq load command that you need to issue if you want to load data in avro file into a partitioned BigQuery table based on avro field defined as a logicalType.

Given the following schema

{
  "type" : "record",
  "name" : "logicalType",
  "namespace" : "com.ryanchapin.tests",
  "fields" : [ {
    "name" : "id",
    "type" : [ "null", "string" ],
    "default" : null
  }, {
    "name" : "value",
    "type" : [ "null", "long" ],
    "default" : null
  },

→ Continue reading

s3cmd ‘ERROR: Test failed: 403 (AccessDenied): Access Denied’ and ‘ERROR: Config: verbosity level ’20’ is not valid’ [SOLVED]

Posted on March 19, 2014November 25, 2021 by rchapin

I’m working on a project that includes sending data via Amazon Simple Storage Service (S3) and was having some problems configuring and using the s3cmd client.

The first thing I discovered about s3cmd is not to trust what it tells you when invoking s3cmd –configure to get things set up to use the bucket.

$ s3cmd -v --configure s3://some-bucket/some-prefix/

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access

→ Continue reading