GCP IAM Authentication and Authorization 101

Just a few days ago, on 14th Dec., Google IAM experienced a 50-minute outage🔥 and resulted in the unavailability of all the user verification services that rely on it worldwide. Youtube alone lost 1.7 million💸 in advertising revenue, while the losses of other websites were utterly incalculable.

Image for post
Image for post
Google outrage influence
  • What is Google IAM?
Image for post
Image for post
GCP Authentication & Authorization, by author

What’s a User

Traditional web systems often have independent user designs, storing data containing basic user information, such as ages, names, birthdays, emails, phone numbers, etc., in the database. The most important data is user passwords, using which users log in to the website after inputting the username.

There are a wider variety of users in the cloud era, though the user definition has not changed much. Google users are no longer limited to Google only. Many websites have integrated Google’s third-party login now, and by obtaining the users’ basic information after user authorization, they enable users to log in directly.

Image for post
Image for post
Medium Google Account Login, by author

It brings much convenience.

  • It is no longer required to build an independent user management system. Relying on a giant like Google and its huge user base, you can save resources and expand the users’ scope.

But don’t forget the disadvantages. Having witnessed the serious consequences caused by Google IAM’s outage, you can imagine how crashing you would be if you are completely dependent on Google’s SRE to fulfill your obligations.🙏

GCP IAM?

Before talking about GCP IAM, you need to understand Kubernetes RBAC. And if not, please read Kubernetes Authentication and Authorization 101.

Image for post
Image for post
IAM definition

IAM (Identity and Access Management), by understanding it, you will know how to answer the following questions on GCP.

  • Who are you?

Identities

The User term on GCP is similar to that of Kubernetes. Identities on GCP can also be categorized into ordinary users and Google Service accounts.

The usual sources of ordinary users are:

  • Google accounts

Google Service Account(GSA), used by APP on GKE or VM running code on your behalf.

  • User-managed GSA: for example, [service-account-name]@[project-id].iam.gserviceaccount.com. And, you can choose the service account name.

GSA uses private/public RSA key-pairs to authenticate with Google.

IAM roles

IAM roles are encapsulations of various GCP resource use permissions. Generally, they can be divided into three categories.

  • Primitive roles: Owner, Editor, Viewer. Roles defined in advance when creating the GCP project and will not be assigned to ordinary users.

Role Bindings

It forms a policy combining identity with roles, and then IAM assigns the policy to different users.

The process is almost the same as the Rolebindings of Kubernetes RBAC, but GCP Rolebindings are not only cluster-wide and namespace-wide, but also have IAM hierarchy as well. From the top to the bottom is organization -> folder -> project -> resource.

Image for post
Image for post
IAM hierarchy

Authentication

After understanding IAM’s definition, let’s try to figure out how to define users and authenticate them.

Usually, when you join the company, you will get your own Gmail mailbox (Here, take Google as an example, not against Microsoft fans) and be added to various Google groups within the company, as shown below.

Image for post
Image for post

(It includes internal and external groups.)

Then you can easily login into the browser. But if you are to operate GKE on the command line and use kubectl, then you are required to use gcloud on the CLI for authentication. Suppose you already have a GCP project and a new GKE cluster.

run gcloud init, and follow the guide prompt, config your CLI step by step.

Or log in first, gcloud auth login, then run gcloud config set to change items by the flag, such as zone, region, project-id. This is more useful when you only want to update one or two items, such as changing to another project in the same region, then you can add the flag-project=xxx.

  • Add your Google credential into kubeconfig so that you can run kubectl against the GKE cluster. Run command gcloud container clusters get-credentials {{cluster-name}}

To make all this work, you need to make sure your google identity has at least container.clusters.get permission.

After all the steps, you can run kubectl in CLI finally. But how?

OpenID Token

The reason why users can directly use kubectl to complete Kubernetes APIServer authentication after passing cloud provider authentication is OpenID Connect.

Simply put, the cloud provider gives an ID Token to your local kubeconfig via OAuth2, then you can complete user authentication with this token in the same way as the APIServer bearer token. For more about the bearer token, refer to Kubernetes Authentication and Authorization 101.

The flow chart is as follows.

Image for post
Image for post
from Kubernetes doc

When you have configured the above gcloud correctly, you can see something similar to the following when you open the local ~/.kube/config file.

When executing the kubectl command, the token in the user corresponding to the cluster in kubeconfig will be automatically put into the HTTP header requested to pass APIServer verification. You can also set the desired token via token flag.

Authorization

It’s time to clear your mind about the GCP authorization mechanism. I believe you are familiar with Kubernetes RBAC.

Except for the identities mentioned above, GCP has its own Roles and Rolebindings.

We can apply IAM configurations to the clusters through YAML by using Google Config Connector. See the example below:

There is one IAMPolicyMember and one IAMPolicy in the YAML.

  • IAMPolicyMember member-test gives the storage.admin role to a GSA(serviceAccount:test@project-test.iam.gserviceaccount.com).

To be noted, the GCP role works on specific resources. Here in the example is the Google cloud project project-test. It can also be other kinds of Google cloud resources, such as BigTable, PubSubTopic, etc.

Custom Role

Sometimes in order to follow the security principle of least privilege, we need to create custom roles since the default roles are not enough.

Bespoke a Custom Role is so easy with Config Connector. For instance, to avoid grant PubSubTopicadmin role, define a custom role with reader and sender permissions, and bind it to the GSA.

External User

I have encountered a special situation where we need to assign permissions to people other than the GCP default owner. To implement it, we need to assign a reasonable GSA to an external user or group.

You can assign a GSA with the predefined role roles/iam.serviceAccountAdmin or
roles/iam.serviceAccountTokenCreator. And the app using this GSA can automatically create related GSA with external user identities and apply it to the cluster, and automatically grant these users the authority to operate on your cluster.

You can fetch all IAM roles by running a gcloud command gcloud iam roles list. I haven’t found a document online containing all the current IAM roles for all the resources on GCP.

How to align IAM with RBAC

After comprehending the fundamental concepts of RBAC and IAM, it’s time to figure out how to use them in the GKE cluster.

  • Is there any conflict?

These questions haunted me for a while at the beginning of my learning Kubernetes and using GCP. By answering them, I quickly absorbed the concepts and took them into practice.

Difference

The essence of either IAM or RBAC is the same, binding the permissions of the resources they manage to respective users.

Image for post
Image for post
RBAC align with IAM, by author

They can cooperate without interference in a cluster, and manage their resources, respectively, requiring users to configure Kubernetes user (ServiceAccount) and Google IAM identity at the same time. Absolutely, it is not easy, requiring more configuration files. So Google proposed a relatively simplified solution.IAM workload identity.

The key is to make a Kubernetes service account act as a Google service account in the format of serviceAccount:project-id.svc.id.goog[k8s-namespace/ksa-name]. ksa-name: Kubernetes Service Account name.

In simple, if you bind a Kubernetes Service Account with a Google Service Account, and then the Pod attached to this Kubernetes Service Account can access the corresponding GCP resources.

Again, it could be set in UI, CLI, and done by Config Connector. I’ll show you the Config Connector way, which is also the YAML way.

The above default Kubernetes Service Account will have the authority to use any GCP resources bind to the IAMServiceAccount.

Remember, there are limitations to workload identity, and here are some important ones.

  • Only one pool per GCP project.

IAMServiceAccountKey

There is another way to use GSA in Kubernetes resources, that is using IAMServiceAccountKey. Take a look at the following example.

  • First, define GSA and IAMPolicyMember.
  • Then, define IAMServiceAccountKey.
  • Use the IAMServiceAccountKey in a Deployment.

This is also a workaround for using client authentication in Kubernetes resources, passing APIServer verification by injecting the key directly into the Deployment. For more details, please refer to Google documentation.

Keep in mind, abusing GSA and IAMServiceAccountKey is a terrible practice. Some best practices about using GSA and its credentials are,

  • Always use IAMServiceAccountKey or token, and never use a secret key directly in the code.

In the end

When using GKE, Authentication and Authorization are hard to understand and use in the beginning especially when it’s entangled with RBAC. But things get better once you have fully understood the concepts and the difference between them.

The critical point is always the resource. Every resource has a set of roles or APIs. When you figure out how to find them quickly and correctly, then you’re good to go.

Thanks for reading!

Live in Stockholm. Love writing, interested in cooking, drawing and reading. Want to travel all around Europe with my cat.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store