The OCIDtab—a Solution to Make OCI CLI Scripting Easier

This article is about working effectively with multiple OCI (Oracle Cloud Infrastructure) environments via OCI CLI (Command Line Interface) and encourages the use of an environment variable file called OCIDtab, however before we look into it’s working the next couple of sections would give an understanding of OCID and the significance of using it in CLI. 

What is an OCID?

OCID is an Oracle Cloud Identifier for any OCI Cloud resource created and is unique for each resource.  Pronounced “oh-sid”.

For more detailed information on OCID, please refer to the Oracle Documentation.

OCIDs generally include the resource type and for some resources, the region in a readable format.  Plus a UUID.  The fact that all resources have OCIDs for unique identification is one of the nice attributes of OCI.

For example, a tenancy would have an OCID like this:

ocid1.tenancy.oc1..bbbaaaaaba3pv6wkcr4jqae5f44n2b2m2yt2j6rx32uzr4h25vqstifabcad

An instance would have an OCID like this:

ocid1.instance.oc1.phx.xylv4ljrlsfiqw6ytytb43vyypt4pkodawglp3wqxjqofakrwvou52gr

OCI CLI is to be used to manage OCI resources without the need of logging in to the Web Console, the OCI CLI will use the same IAM policies as required for the web console, in fact, the OCI CLI is an extension of the web console, one could have an OCI CLI only IAM user. There are some excellent Pythian Blogs for OCI CLI hands on if you need to get started with using OCI CLI and creating resources in OCI 

https://blog.pythian.com/installing-oci-command-line-utilities-in-linux-and-windows/ 

https://blog.pythian.com/oracle-database-cloud-service/

OCI CLI  needs to be looked at as an alternative for the Web Console (and REST APIs), for performing certain activities such as downloading a backup from object storage or monitoring the state of the instance.  The OCI CLI does not require the need of Ansible playbooks or Terraform state and config files. Ansible and Terraform can still be leveraged once the environment is stable the OCI administration activities are repeatable. The use of OCI CLI via shell/python scripts is also one of the common use cases followed for managing OCI

Why OCIDtab?

For many Oracle DBA's it is an extremely common activity to use the oratab file and the oraenv utility to set the required environment variables for the database environment. As you start working on Oracle Cloud Infrastructure (OCI) and want to make your steps documented and repeatable (by leveraging the OCI CLI instead of the web console) you may need to prepare many scripts referencing the OCID values. When working for a customer to schedule many OCI CLI scripts I realized there is no equivalent of setting variables for OCID values, so I started to create a small script for properly setting the necessary  OCID values so I can repeatedly use them as needed.

Pre-requisites

There are two prerequisites to set up the OCIDtab files

1) OCI CLI

OCI CLI to be set up in the Environment where CLI commands would be run

In case you have not set up your OCI CLI and do not have a working profile, you can use the Oracle Quick Start Guide as a reference.

https://docs.cloud.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm

A Sample OCI Profile (from the “config” file) would be as below, note the text in the square brackets [ ] is the profile name, a logical name that represents the environment is to be provided here. A single config file can support multiple profiles if required.

[DEV-PROFILE]
user=ocid1.user.oc1..aaaaaaaarvdjfksfghxkfhgkxhgkxhgznvusgvnuvifvuvipaneydyxhdu5t
fingerprint=ab:cd:ef:gh:ij:kl:mn:op:qr:st:uv:wx:yz:12:24:56
key_file=/home/abhilash/.ssh/mkey.pem
tenancy=ocid1.tenancy.oc1..aaaaaaaavf7vnjnvifnvifnvinvirnvi4refrwefewfbriehwfw2nccwis8c
region=us-ashburn-1

2) JSON Query—jq

As all OCI CLI output is in JSON format, there is often the need to filter out the OCID values from the output using the jq utility.  On most Linux distributions, jq can be installed as

# sudo yum install jq

How to generate OCIDtab?

The ocidenv script is a straightforward tool to populate the required environment variables, The ocidenv script creates an environment variable hosting file like ${HOME}/.<envname>-ocidtab which includes variables and OCID values for some of the more commonly used OCI resources such as regions, availability domains, etc.

The script works best when used for a single profile as DEV-PROFILE and I would recommend creating distinct OCIDtab files such as.DEV-ocidtab , .UAT-ocidtab , .PROD-ocidtab to set the required OCID values needed for their corresponding environments. The OCIDtab needs to be generated from a ocidenv.sh script , the source code of the ocidenv is available here

https://github.com/abhilash-8/ocidenv

The ocidenv.sh script is used to generate the OCIDtab files, example shown below

$ ocidenv.sh <config_file_path> <oci_profile> <compartment_name> <vcn_name>

An example to generate OCIDtab files is shown below

$ ./ocidenv.sh $HOME/.oci/config DEV-PROFILE devlabcomp1 devlabvcn1
$ cat .DEV-ocidtab
TENANCY_OCID=ocid1.tenancy.oc1..aaaaaaaavf7vnjnvifnvifnvinvirnvi4refrwefewfbriehwfw2nccwis8c
VCN_OCID=ocid1.vcn.oc1.iad.amaaaaaakyapamdwsfawknfknfaknfaknSlkaweisefscmcmeaf2ndoyqg4j
COMP_OCID=ocid1.compartment.oc1..aaaaaaaaivnijvfhgbhghbfvnksmlsxmjgnlkhergeadmlsfidamcawijf2j
CONFIG_PROFILE=DEV
HOME_REGION=us-ashburn-1
TARGET_REGION=us-ashburn-1
AVD_OCID_1=lhaE:US-ASHBURN-AD-1
AVD_OCID_2=lhaE:US-ASHBURN-AD-2
AVD_OCID_3=lhaE:US-ASHBURN-AD-3

The following examples would help explain further the use of the OCIDtab files, it can be noted that by only changing OCIDtab environment files the $COMP_OCID and  $CONFIG_PROFILE are different values for the respective environments.

To list all regions subscribed to in the DEV Environment:

$ . ~/.DEV-ocidtab
$ oci iam region-subscription list --all --output table --profile $CONFIG_PROFILE
+----------------+------------+---------------+--------+
| is-home-region | region-key | region-name   | status |
+----------------+------------+---------------+--------+
| True           | IAD        | us-ashburn-1  | READY  |
+----------------+------------+---------------+--------+

If you want to list all object storage buckets in the UAT Environment, you can easily change to the UAT OCIDtab:

$ . ~/.UAT-ocidtab
$ oci os bucket list --compartment-id $COMP_OCID --query "data [*].{\"name\":\"name\"}" --output table --profile $CONFIG_PROFILE
+---------------------------------------+
| name                                  |
+---------------------------------------+
| app-server-backups                    |
| media-files                           |
+---------------------------------------+

And to list all compute instances in the production environment you can change to the PROD OCIDtab:

$ . ~/.PROD-ocidtab
$ oci compute instance list -c $COMP_OCID --query "data [*].{\"display-name\":\"display-name\",\"availability-domain\":\"availability-domain\",\"fault-domain\":\"fault-domain\",shape:shape,\"lifecycle-state\":\"lifecycle-state\"}" --output table --profile $CONFIG_PROFILE
+----------------------+--------------------+----------------+-----------------+------------------+
| availability-domain  | display-name       | fault-domain   | lifecycle-state | shape            |
+----------------------+--------------------+----------------+-----------------+------------------+
| lhaE:US-ASHBURN-AD-1 | app-prod-1         | FAULT-DOMAIN-1 | RUNNING         | VM.Standard2.2   |
| lhaE:US-ASHBURN-AD-1 | app-prod-2         | FAULT-DOMAIN-2 | RUNNING         | VM.Standard2.2   |
| lhaE:US-ASHBURN-AD-1 | web-prod-1         | FAULT-DOMAIN-2 | RUNNING         | VM.Standard.E2.1 |
| lhaE:US-ASHBURN-AD-1 | web-prod-2         | FAULT-DOMAIN-3 | RUNNING         | VM.Standard.E2.1 |
+----------------------+--------------------+----------------+-----------------+------------------+

Working further one would need to keep adding many OCID values for Object Storage, Block Volumes, etc.. to name a few, this OCIDtab file can always be regenerated with new OCI resources would help eliminate any manual errors in generating the ocid values for a particular profile and can also be sourced to run OCI CLI commands or to any shell/python scripts.

It is also worth noting that there is an Oracle provided CLI Configuration file called oci_cli_rc file which is documented here, the difference between the OCIDtab file and the oci_cli_rc file is that the purpose of the ocidenv script is to set the custom OCID variables pertaining to the environment being worked upon such as ( .DEV-ocidtab .UAT-ocidtab .PROD-ocidtab ) whereas the oci_cli_rc sets various parameter options for CLI operations such as ( ls for list , rm for delete object storage operations, get_top_5_results for top 5 results, --ad for --availability-domain, etc ) , the oci_cli_rc still would require --profile to identify the required profile being worked upon and does not include any OCID variables, this is where the OCIDtab files give the CLI user the additional flexibility to clearly source the required environment needed to work upon.