Background
I have recently been working on a golang project where I need to authenticate to AWS resources using a GCP workload identity. I think a lot of people would accomplish this by using a static access key and set environment variables like AWS_SECRET_ACCESS_KEY, but this approach is not ideal since it is a static credential that is not easy to rotate. Luckily, there is a way to keep keyless authentication using GCP workload identity to assume an AWS role!
A visualization of my goal for this project looks like this:
How does GCP workload identity function?
GCP hosts a metadata server that has context to which service account is tied to a particular compute instance or pod in GKE. The metadata server is hosted at http://metadata.google.internal and has several endpoints that you can query. An easy way to see this in action is by creating a service account and tying it to a compute instance.
Once this is created, you can curl the email endpoint and see that the service account we attached to the VM is being returned as expected:
aro@tester:~$ curl http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/email -H "Metadata-Flavor: Google"
[email protected]
For the purposes of this project, we want to utilize the token endpoint to receive an access token that can be used for authentication to various services:
aro@tester:~$ curl http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token -H "Metadata-Flavor: Google"
{"access_token":"ya29.c.<omitted for brevity>","expires_in":3288,"token_type":"Bearer"}
This process of querying the metadata server to get an access token is the basis of how workload identity authentication works inside GCP.
More info
If you want to learn more about the endpoints and flows of the GCP metadata server, the following links might be helpful:
Creating the AWS Role
Now that we know how the GCP metadata server works to give us an access token, we need to create an AWS role that federates to our GCP service account identity and allows us to call AWS services. When creating the AWS role, we add a policy that essentially says “ask accounts.google.com to verify the authentication of this token for this unique ID”
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "accounts.google.com"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"accounts.google.com:sub": "115513827103198327698"
}
}
}
]
}
For the purposes of this experiment, I give the role S3 read only access:
The code, and bringing it all together
After setting up the GCP service account, AWS role, and VM to run the experiment on, I was ready to write my code. To validate that the credentials automatically refresh in the AWS SDK, I created an S3 bucket and put a test file inside it. My code just lists the contents of this bucket every 10 seconds. This is a pretty simple test, but making an authenticated call like this will validate the authentication path and the credential refresh.
The code:
package main
import (
"context"
"fmt"
"log"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/credentials/stscreds"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/aws/aws-sdk-go-v2/service/sts"
"golang.org/x/oauth2/google"
"google.golang.org/api/idtoken"
"google.golang.org/api/option"
)
// creating a struct to satisfy the IdentityTokenRetriever interface
type gcpCredRetreiver struct {
Audience string
}
func main() {
roleArn := "arn:aws:iam::<accountID>:role/gcp-bucket-lister"
bucketName := "gcp-to-aws-auth-bucket"
g := gcpCredRetreiver{Audience: "aro-labs.com"}
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
panic(err)
}
cfg.Region = "us-east-1"
creds := stscreds.NewWebIdentityRoleProvider(sts.NewFromConfig(cfg), roleArn, g)
cfg.Credentials = aws.NewCredentialsCache(creds)
s3client := s3.NewFromConfig(cfg)
params := &s3.ListObjectsV2Input{
Bucket: &bucketName,
}
for {
objects, err := s3client.ListObjectsV2(context.Background(), params)
if err != nil {
log.Fatal("can't list objects\n", err)
}
log.Print("listing bucket:")
for _, obj := range objects.Contents {
fmt.Println(*obj.Key)
}
time.Sleep(10 * time.Second)
}
}
func (g gcpCredRetreiver) GetIdentityToken() ([]byte, error) {
log.Print("GCP creds updater called")
ctx := context.Background()
// This is exactly the same as calling the metadata url
// curl http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token -H "Metadata-Flavor: Google"
credentials, err := google.FindDefaultCredentials(ctx)
if err != nil {
return nil, fmt.Errorf("failed to generate default credentials: %w", err)
}
ts, err := idtoken.NewTokenSource(ctx, g.Audience, option.WithCredentials(credentials))
if err != nil {
return nil, fmt.Errorf("failed to create NewTokenSource in GCP: %w", err)
}
// Get the ID token.
// Once you've obtained the ID token, you can use it to make an authenticated call
t, err := ts.Token()
if err != nil {
return nil, fmt.Errorf("failed to receive GCP token: %w", err)
}
return []byte(t.AccessToken), nil
}
The main bit of “magic” happens with the gcpCredRetriever struct and method:
type gcpCredRetreiver struct {
Audience string
}
func (g gcpCredRetreiver) GetIdentityToken() ([]byte, error) {
log.Print("GCP creds updater called")
ctx := context.Background()
// This is exactly the same as calling the metadata url
// curl http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token -H "Metadata-Flavor: Google"
credentials, err := google.FindDefaultCredentials(ctx)
if err != nil {
return nil, fmt.Errorf("failed to generate default credentials: %w", err)
}
ts, err := idtoken.NewTokenSource(ctx, g.Audience, option.WithCredentials(credentials))
if err != nil {
return nil, fmt.Errorf("failed to create NewTokenSource in GCP: %w", err)
}
// Get the ID token.
// Once you've obtained the ID token, you can use it to make an authenticated call
t, err := ts.Token()
if err != nil {
return nil, fmt.Errorf("failed to receive GCP token: %w", err)
}
return []byte(t.AccessToken), nil
}
The method GetIdentityToken() is what satisfies the IdentityTokenRetriever interface. When we initialize the AWS client we are passing our custom gcpCredRetriever and the AWS SDK knows to call GetIdentityToken() whenever the current token is expired:
//g is our gcpCredRetriever object
creds := stscreds.NewWebIdentityRoleProvider(sts.NewFromConfig(cfg), roleArn, g)
cfg.Credentials = aws.NewCredentialsCache(creds)
s3client := s3.NewFromConfig(cfg)
This means that as long as we provide the GetIdentityToken() method and return an access token in a byte slice, we can use our custom code to get the token from the GCP metadata server!
Results
After running this code, we can see that the token refresh happens automatically after almost exactly 1 hour:
2024/01/17 17:09:01 GCP creds updater called
2024/01/17 17:09:01 listing bucket:
test.txt
2024/01/17 17:09:11 listing bucket:
test.txt
2024/01/17 17:09:21 listing bucket:
test.txt
2024/01/17 17:09:32 listing bucket:
test.txt
< ~ 1 hour of logs omitted >
2024/01/17 18:08:37 listing bucket:
test.txt
2024/01/17 18:08:47 listing bucket:
test.txt
2024/01/17 18:08:57 listing bucket:
test.txt
2024/01/17 18:09:07 GCP creds updater called <-- Credential refresh happening transparently!
2024/01/17 18:09:08 listing bucket:
test.txt
2024/01/17 18:09:18 listing bucket:
test.txt
Final Thoughts
This example was made in golang, but this process could work for any language. While creating this experiment I was surprised there aren’t more examples of how to facilitate this cross-cloud authentication so I hope this example helps. Feel free to leave a comment if you have any questions!