Using El Carro Operator on AWS

Introduction

Google recently released a Kubernetes operator for Oracle, El Carro.

The project includes good examples of how it works on GCP and on your local computer (using minikube). Given it is a portable implementation, we wanted to give it a try on AWS deploying an Oracle XE database.

The changes from the original instructions are mainly to replace GCP services with the equivalent in AWS:

• Amazon Elastic Container Registry (ECR) instead of GCP Container Registry (GKR) to store the Docker image built for the database
• AWS CodeBuild instead of Google Cloud Build to execute the steps needed to create that image and push it to the repository
• Amazon Elastic Kubernetes Service (EKS) instead of Google Kubernetes Engine (GKE) to run the k8s cluster
• Amazon Elastic Block Store (EBS) Container Storage Interfase (CSI) driver instead of GCP Compute Engine persistent disk CSI Driver to handle persistent volumes inside k8s (ebs.csi.aws.com vs pd.csi.storage.gke.io provisioner), where data will be stored

This paper follows the steps performed using AWS services to deploy an Oracle XE database on k8s using the El Carro operator.

Note that this is just an example covering similar steps described in the 18c XE quickstart guide for GCP; it may not be the only way to make this work in AWS. Also, the names for the resources we create in AWS are the same as described in the GCP guide. We were looking to make it easy to follow and compare, but you may want to customize that to meet your needs—for example, GCLOUD as the container database name, gke as the dbDomain, csi-gce-pd storage class name, etc.

Preparation

I used an AWS account still in its first year with enough free services quota for most of the compute and storage needs of this test.

The first difference I found is that while you can run this test on GCP using Cloud Shell, the equivalent AWS CloudShell doesn’t support Docker containers. Looking to save time, we used a small EC2 instance (t2.micro) with the Amazon Linux 2 AMI image, where docker can be installed easily.

All steps below were executed as the ec2-user from within the EC2 instance:

1) Download the El Carro software:

$ wget https://github.com/GoogleCloudPlatform/elcarro-oracle-operator/releases/download/v0.0.0-alpha/release-artifacts.tar.gz
$ tar -xzvf release-artifacts.tar.gz

That will create the directory v0.0.0-alpha in your working directory.

2) Install git and docker:

$ sudo amazon-linux-extras install docker
$ sudo yum update -y
$ sudo service docker start
$ sudo usermod -a -G docker ec2-user
$ reboot
$ sudo yum install git

3) Configure the AWS client:

$ aws configure

Enter the following data from your account, which is available in your AWS console:

AWS Access Key ID [None]: <your AK id>
AWS Secret Access Key [None]: <your SAK>
Default region name [None]: <your region>

Prepare environment variables for config values that will be needed frequently: Zone and Account ID:

$ export ZONE=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/\(.*\)[a-z]/\1/') ; echo $ZONE
$ export ACC_ID=$(aws sts get-caller-identity --query "Account") ; echo $ACC_ID

Creating a Containerized Oracle Database Image

The following steps create a docker container image and pushes it into an AWS ECR repository, automating the process using AWS CodeBuild,

1) Create a service role, needed for CodeBuild as described here:

$ vi create-service-role.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "codebuild.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

$ vi put-role-policy.json
$ cat put-role-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CloudWatchLogsPolicy",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "CodeCommitPolicy",
      "Effect": "Allow",
      "Action": [
        "codecommit:GitPull"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "S3GetObjectPolicy",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "S3PutObjectPolicy",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "S3BucketIdentity",
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketAcl",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Action": [
        "ecr:BatchCheckLayerAvailability",
        "ecr:CompleteLayerUpload",
        "ecr:GetAuthorizationToken",
        "ecr:InitiateLayerUpload",
        "ecr:PutImage",
        "ecr:UploadLayerPart"
      ],
      "Resource": "*",
      "Effect": "Allow"
   }
  ]
}

$ aws iam create-role --role-name CodeBuildServiceRole --assume-role-policy-document file://create-service-role.json
$ aws iam put-role-policy --role-name CodeBuildServiceRole --policy-name CodeBuildServiceRolePolicy --policy-document file://put-role-policy.json

2) Create an ECR repository. We are naming it elcarro:

$ aws ecr create-repository \
>     --repository-name elcarro \
>     --image-scanning-configuration scanOnPush=true \
>     --region ${ZONE}

3) Create the build instructions file to create the container image and push it into the registry. We are also including in that directory the scripts used by those steps—Dockerfile and shell scripts called from there:

$ mkdir cloudbuild 
$ cp v0.0.0-alpha/dbimage/* cloudbuild/ 
$ cd cloudbuild/ 
[cloudbuild]$ rm cloudbuild.yaml cloudbuild-18c-xe.yaml image_build.sh 
[cloudbuild]$ vi buildspec.yml 
[cloudbuild]$ cat buildspec.yml
version: 0.2

phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
  build:
    commands:
      - echo Build started on `date`
      - echo Building the Docker image...          
      - docker build --no-cache --build-arg=DB_VERSION=18c --build-arg=CDB_NAME=GCLOUD --build-arg=CHARACTER_SET=$_CHARACTER_SET --build-arg=EDITION=xe -t $IMAGE_REPO_NAME:$IMAGE_TAG .
      - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG 
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the Docker image...
      - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG

4) Prepare a zip with all those files:

[cloudbuild]$ zip ../elCarroImage.zip *

5) Create an S3 bucket and copy the zip file:

$ aws s3 mb s3://codebuild-${ZONE}-${ACC_ID}-input-bucket --region $ZONE
$ aws s3 --region $ZONE cp elCarroImage.zip s3://codebuild-${ZONE}-${ACC_ID}-input-bucket --acl public-read
upload: ./elCarroImage.zip to s3://codebuild-<ZONE>-<ACC_ID>-input-bucket/elCarroImage.zip

6) Prepare a build JSON file for this project – replace the strings <$ZONE> and <$ACC_ID> with their actual values, as this is a static file:

$ cat elcarro-ECR.json
{
  "name": "elCarro",
  "source": {
    "type": "S3",
    "location": "codebuild-<$ZONE>-<$ACC_ID>-input-bucket/elCarroImage.zip"
  },
  "artifacts": {
    "type": "NO_ARTIFACTS"
  },
  "environment": {
    "type": "LINUX_CONTAINER",
    "image": "aws/codebuild/standard:4.0",
    "computeType": "BUILD_GENERAL1_SMALL",
    "environmentVariables": [
      {
        "name": "AWS_DEFAULT_REGION",
        "value": "<$ZONE>"
      },
      {
        "name": "AWS_ACCOUNT_ID",
        "value": "<$ACC_ID>"
      },
      {
        "name": "IMAGE_REPO_NAME",
        "value": "elcarro"
      },
      {
        "name": "IMAGE_TAG",
        "value": "latest"
      }
    ],
    "privilegedMode": true
  },
  "serviceRole": "arn:aws:iam::<$ACC_ID>:role/CodeBuildServiceRole"

7) Create the build project:

$ aws codebuild create-project --cli-input-json file://elcarro-ECR.json

8) Finally, start the build:

$ aws codebuild start-build --project-name elCarro

To check progress from the CLI:

$ aws codebuild list-builds
{
    "ids": [
        "elCarro:59ef7722-06d6-4a1d-b2d5-14d967e9e4df"
    ]
}

We can see build details using the latest ID from the above output:

$ aws codebuild batch-get-builds --ids <elCarro:59ef7722-06d6-4a1d-b2d5-14d967e9e4df>

The build takes around 20 minutes. Once completed successfully, we can see the image in our ECR repo:

$ aws ecr list-images --repository-name elcarro
{
    "imageIds": [
        {
            "imageTag": "latest",
            "imageDigest": "sha256:7a2bd504abdf7c959332601b1deef98dda19418a252b510b604779a6143ec809"
        }
    ]
}

Create a Kubernetes cluster

We need to install the CLI utils first:

$ curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.20.4/2021-04-12/bin/linux/amd64/kubectl
$ chmod +x ./kubectl
$ mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$PATH:$HOME/bin
$ echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
$ kubectl version --short --client
Client Version: v1.20.4-eks-6b7464
   
$ curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
$ sudo mv /tmp/eksctl /usr/local/bin
$ eksctl version
0.55.0

Using bash variables for frequently used customized values will provide clarity to upcoming steps, in addition to ACC_ID and ZONE that were already set previously:

export PATH_TO_EL_CARRO_AWS=/home/ec2-user/elcarro-aws
export PATH_TO_EL_CARRO_GCP=/home/ec2-user/v0.0.0-alpha
export DBNAME=GCLOUD
export CRD_NS=db
export CLUSTER_NAME=gkecluster

We need to create a new ssh key for this test: myekskeys. This will be stored in the account, so there’s no need to repeat it if you deploy a second test in the future:

$ aws ec2 create-key-pair --region ${ZONE} --key-name myekskeys

Next, we’ll create a local file with the content of the private key, using output from the previous command:

$ vi myekskeys.priv

We are ready to create the cluster. It will take about 25 minutes to complete:

$ eksctl create cluster \
    --name $CLUSTER_NAME \
    --region $ZONE \
    --with-oidc \
    --ssh-access \
    --ssh-public-key myekskeys \
    --managed

These are basic checks to monitor the progress and resources as they are created:

aws eks list-clusters
aws eks describe-cluster --name $CLUSTER_NAME
eksctl get addons --cluster $CLUSTER_NAME
kubectl get nodes --show-labels --all-namespaces
kubectl get pods -o wide --all-namespaces --show-labels
kubectl get svc --all-namespaces
kubectl describe pod --all-namespaces
kubectl get events --sort-by=.metadata.creationTimestamp

Add the EBS CSI driver

We need to use the AWS-EBS CSI driver:

$ cd ${PATH_TO_EL_CARRO_AWS}
$ git clone https://github.com/kubernetes-sigs/aws-ebs-csi-driver.git

Configuration files in El Carro project are set to use GCP storage; we need to adjust a few to use the AWS EBS driver. We will create new configuration files in the directory elcarro-aws/deploy/csi based on the original ones:

$ mkdir -p deploy/csi
$ cp ${PATH_TO_EL_CARRO_AWS}/deploy/csi/gce_pd_storage_class.yaml deploy/csi/aws_pd_storage_class.yaml
$ cp ${PATH_TO_EL_CARRO_AWS}/deploy/csi/gce_pd_volume_snapshot_class.yaml deploy/csi/aws_pd_volume_snapshot_class.yaml

$ vi deploy/csi/aws_pd_storage_class.yaml
$ cat deploy/csi/aws_pd_storage_class.yaml
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: csi-gce-pd
# provisioner: pd.csi.storage.gke.io
provisioner: ebs.csi.aws.com
parameters:
  type: pd-standard
volumeBindingMode: WaitForFirstConsumer

$ vi deploy/csi/aws_pd_volume_snapshot_class.yaml
$ cat deploy/csi/aws_pd_volume_snapshot_class.yaml
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: csi-gce-pd-snapshot-class
# driver: pd.csi.storage.gke.io
driver: ebs.csi.aws.com
deletionPolicy: Delete
$

The below policy is required only one time in your account, so there’s no need to re-execute this if you’re repeating cluster creation:

$ export OIDC_ID=$(aws eks describe-cluster --name $CLUSTER_NAME --query "cluster.identity.oidc.issuer" --output text | rev | cut -d"/" -f1 | rev)

$ aws iam list-open-id-connect-providers | grep $OIDC_ID

$ aws iam create-policy \
    --policy-name AmazonEKS_EBS_CSI_Driver_Policy \
    --policy-document file://${PATH_TO_EL_CARRO_AWS}/aws-ebs-csi-driver/docs/example-iam-policy.json

The following steps are mandatory every time you need to configure EBS CSI:

$ eksctl create iamserviceaccount \
    --name ebs-csi-controller-sa \
    --namespace kube-system \
    --cluster $CLUSTER_NAME \
    --attach-policy-arn arn:aws:iam::<${ACC_ID}>:policy/AmazonEKS_EBS_CSI_Driver_Policy \
    --approve \
    --override-existing-serviceaccounts

$ export ROLE_ARN=$(aws cloudformation describe-stacks --stack-name eksctl-${CLUSTER_NAME}-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa --query='Stacks[].Outputs[?OutputKey==`Role1`].OutputValue' --output text)
$ echo $ROLE_ARN
    
$ kubectl apply -k "github.com/kubernetes-sigs/aws-ebs-csi-driver/deploy/kubernetes/overlays/stable/?ref=master"

This is in AWS docs but seems to not be needed. It may have been necessary for an older release, so we’re keeping it just to be consistent with official docs:

$ kubectl annotate serviceaccount ebs-csi-controller-sa \
    -n kube-system \
    eks.amazonaws.com/role-arn=$ROLE_ARN

We are ready to add our storage class into the cluster: 

$ kubectl create -f ${PATH_TO_EL_CARRO_AWS}/deploy/csi/aws_pd_storage_class.yaml

Now we’re completing the storage configuration and installing the volume snapshot class:

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml

And now we add the storage class into the cluster:

$ kubectl create -f ${PATH_TO_EL_CARRO_AWS}/deploy/csi/aws_pd_volume_snapshot_class.yaml

At this point, the cluster is ready to deploy the operator. 

A few basic sanity checks of our k8s cluster can be performed to compare later in case of any problem appears:

$ kubectl get storageclass --all-namespaces
$ kubectl describe storageclass
$ kubectl get pods -o wide --all-namespaces
$ kubectl get events --sort-by=.metadata.creationTimestamp

Deploy El Carro operator and create the DB

We will use the code provided by the GCP project:

$ kubectl apply -f ${PATH_TO_EL_CARRO_GCP}/operator.yaml

Now we are ready to create an Oracle 18c XE database. 

We took the original YAML file for this and updated it to use the proper URL for the DB image—the one we created a few steps ago. There’s no need to adjust the storageClass name, as we deployed to the cluster with the same name used by the GCP example but pointing to the correct AWS driver:

$ vi ${PATH_TO_EL_CARRO_AWS}/samples/v1alpha1_instance_18c_XE_express.yaml
$ cat ${PATH_TO_EL_CARRO_AWS}/samples/v1alpha1_instance_18c_XE_express.yaml
apiVersion: oracle.db.anthosapis.com/v1alpha1
kind: Instance
metadata:
  name: mydb
spec:
  type: Oracle
  version: "18c"
  edition: Express
  dbDomain: "gke"
  disks:
  - name: DataDisk
    storageClass: csi-gce-pd
  - name: LogDisk
    storageClass: csi-gce-pd
  services:
    Backup: true
    Monitoring: false
    Logging: false
    HA/DR: false
  images:
    # Replace below with the actual URIs hosting the service agent images.
    # service: "gcr.io/${PROJECT_ID}/oracle-database-images/oracle-18c-xe-seeded-${DB}"
    service: "<${ACC_ID}>.dkr.ecr.us-east-2.amazonaws.com/elcarro:latest"
  sourceCidrRanges: [0.0.0.0/0]
  # Oracle SID character limit is 8, anything > gets truncated by Oracle
  cdbName: GCLOUD

Now we continue with the steps described in the GCP guide to deploy the 18c XE database:

$ export CRD_NS=db
$ kubectl create ns ${CRD_NS} 
$ kubectl get ns ${CRD_NS}
  
$ kubectl apply -f ${PATH_TO_EL_CARRO_AWS}/samples/v1alpha1_instance_18c_XE_express.yaml -n ${CRD_NS}

To monitor the progress of the deployment, we can check every component. The most relevant information is on the mydb-sts-0 pod:

kubectl get instances -n $CRD_NS
kubectl get events --sort-by=.metadata.creationTimestamp
kubectl get nodes
kubectl get pods -n $CRD_NS
kubectl get pvc -n $CRD_NS
kubectl get svc -n $CRD_NS
kubectl logs mydb-agent-deployment-<id> -c config-agent -n $CRD_NS
kubectl logs mydb-agent-deployment-<id> -c oracle-monitoring -n $CRD_NS
kubectl logs mydb-sts-0 -c oracledb -n $CRD_NS
kubectl logs mydb-sts-0 -c dbinit -n $CRD_NS
kubectl logs mydb-sts-0 -c dbdaemon -n $CRD_NS
kubectl logs mydb-sts-0 -c alert-log-sidecar -n $CRD_NS
kubectl logs mydb-sts-0 -c listener-log-sidecar -n $CRD_NS

kubectl describe pods mydb-sts-0 -n $CRD_NS
kubectl describe pods mydb-agent-deployment-<id> -n $CRD_NS

kubectl describe node ip-<our-ip>.compute.internal

After a few minutes, the database instance (CDB) is created:

$ kubectl get pods -n $CRD_NS
NAME                                     READY   STATUS    RESTARTS   AGE
mydb-agent-deployment-6f7748f88b-2wlp9   1/1     Running   0          11m
mydb-sts-0                               4/4     Running   0          11m

$ kubectl get instances -n $CRD_NS
NAME   DB ENGINE   VERSION   EDITION   ENDPOINT      URL                                                                           DB NAMES   BACKUP ID   READYSTATUS   READYREASON      DBREADYSTATUS   DBREADYREASON
mydb   Oracle      18c       Express   mydb-svc.db   af8b37a22dc304391b51335848c9f7ff-678637856.us-east-2.elb.amazonaws.com:6021                          True          CreateComplete   True            CreateComplete

The last step is to deploy the database (PDB):

$ kubectl apply -f ${PATH_TO_EL_CARRO_GCP}/samples/v1alpha1_database_pdb1_express.yaml -n ${CRD_NS}
database.oracle.db.anthosapis.com/pdb1 created

After this completes, we can see instance status now includes the PDB:

$ kubectl get instances -n $CRD_NS
NAME   DB ENGINE   VERSION   EDITION   ENDPOINT      URL                                                                           DB NAMES   BACKUP ID   READYSTATUS   READYREASON      DBREADYSTATUS   DBREADYREASON
mydb   Oracle      18c       Express   mydb-svc.db   af8b37a22dc304391b51335848c9f7ff-678637856.us-east-2.elb.amazonaws.com:6021   ["pdb1"]               True          CreateComplete   True            CreateComplete

We can connect to the database by either the TNS entry (af8b37a22dc304391b51335848c9f7ff-678637856.us-east-2.elb.amazonaws.com:6021/pdb1.gke) or directly to the container:

$ kubectl  exec -it -n db mydb-sts-0 -c oracledb -- bash -i
$ . oraenv <<<GCLOUD
bash-4.2$ sqlplus / as sysdba

SQL*Plus: Release 18.0.0.0.0 - Production on Wed Jul 7 21:03:28 2021
Version 18.4.0.0.0

Copyright (c) 1982, 2018, Oracle.  All rights reserved.


Connected to:
Oracle Database 18c Express Edition Release 18.0.0.0.0 - Production
Version 18.4.0.0.0

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB1                           READ WRITE NO

SQL> set lines 180 pages 180
SQL> select * from v$session_connect_info;
...
12 rows selected.

SQL> exit
Disconnected from Oracle Database 18c Express Edition Release 18.0.0.0.0 - Production
Version 18.4.0.0.0
bash-4.2$

Next steps

This setup is just the first step to provision an Oracle database into a k8s cluster on AWS EKS.

It is ready to use the features implemented by the operator, as backup using snapshots or RMAN, restore from a backup, or exporting and importing data using Data Pump. 

We will explore that outside of GCP in upcoming blog posts.

Deleting resources after testing

After our tests are finished, we should delete the cluster to avoid charges if we don’t plan to continue using it:

$ kubectl get svc --all-namespaces

$ kubectl delete svc mydb-svc -n ${CRD_NS} 

$ eksctl delete cluster --name $CLUSTER_NAME

Even if the output from the last command shows it completed successfully, check if that is true by inspecting the detailed output using the command below, as we found a couple of times “The following resource(s) failed to delete: [InternetGateway, SubnetPublicUSEAST2A, VPC, VPCGatewayAttachment, SubnetPublicUSEAST2B]. “ and had to delete those resources manually using the AWS console:

$ eksctl utils describe-stacks --region=us-east-2 --cluster=$CLUSTER_NAME

Conclusion

Since El Carro is based on open standards and technologies (such as Kubernetes), porting it to another cloud provider such as AWS isn’t that difficult. Though some minor changes are required, overall it’s not much of a departure from the GCP process. Maybe additional curated quickstart guides for other cloud providers will be created based on experiences like this, which could ease its adoption outside of GCP.