Share this
by Simon Pane on Sep 16, 2015 12:00:00 AM
Getting Started with Oracle Cloud Backups - Amazon Web Services (AWS)
This is part 2 of a 3 part series on "Getting Started with Oracle Cloud Backups".- Part 1 covers setting up RMAN to backup directly to the new Oracle Cloud Database Backup Service (ODBS).
- Part 2 covers setting up RMAN to backup directly to the cloud using Amazon Web Services (AWS) Simple Storage Service (S3).
- Part 3 compares and contrasts the two services.
Background
As mentioned in part 1 of this series, it's actually really easy to get started with writing Oracle backups directly to the cloud. So regardless of the motivation, it's certainly possible to go from zero to fully backed-up to a cloud based service in just a few hours, hence meeting any requirements to implement off-site backups extremely quickly. Of course, there are several dependencies including the database size, internet upload speed (and cost), and change control formalities. Recovery speed is most certainly going to be slow compared to recovering from local on-premise storage, but this service probably isn't meant to replace your on-premise backup strategy, but rather compliment it or possibly replace your off-site tape-vaulting vendor service (at least for Oracle database backups). Regardless, recovery time and RTO must of course also be taken into consideration. Also, while the Amazon Web Services metered usage costs are extremely competitive, directly integrating with RMAN does require the Oracle Secure Backup Cloud Module, which is a licensed option. However, Amazon does also provide some additional unique features with their S3 storage: namely object versioning, object life-cycle rules, the ability to pick the storage region with more granularity (i.e. a specific region within the United States) and "Cross-Region Replication". This article provides a quick start for getting up and running with Oracle RMAN backups to AWS S3 using the Oracle Secure Backup (OSB) Cloud Module for users new to the service and cloud backups.Backing up to Amazon Web Services (AWS)
Backing up to the AWS Simple Storage Service or S3 isn't new. I first experimented with this in June 2011 and hadn't really touched it since. Yet, fundamentally it hasn't really changed at all since that time. From a very high level the process is:- Create an AWS S3 account and setup the necessary credentials.
- Install an AWS specific Oracle Secure Backup library into your Oracle Home.
- Run an RMAN backup using the SBT_TAPE device type.
Creating an AWS S3 Account
Creating the necessary AWS S3 account is almost trivially easy. If you or your organization does not already have an "Amazon Web Services account", start by signing up for the Free Trial. The free trial gives you 5GB of storage (along with about 20 other services) for a full year. For almost all organizations the 5GB of storage probably won't be sufficient. However, Amazon does provide a simple to use billing calculator to try to estimate your monthly costs based on usage. Amazon also provides other benefits, such as being able to choose what data centers or geographic regions your data is stored in. Further details can be found in their FAQ. After filling in the necessary information and creating an account (if your organization does not already have one), the next step is to set up a security credential. In the top right corner of your S3 console navigate to "Security Credentials":
Installing the "Oracle Secure Backup Cloud Module for Amazon S3"
Installing the necessary "Oracle Secure Backup Cloud Module for Amazon S3" into your Oracle home is very easy. From OTN download an installer Java JAR file and copy and extract the zip to your database server. When run, the installer will determine the proper database version and OS platform, as well as download the appropriate library file to your Oracle home or other specified directory. Installation requires a few mandatory arguments, namely:- The AWS credentials created previously
- Your OTN username (but not your Oracle account password)
- The location for the library file it will download. Usually use $ORACLE_HOME/lib
- The location for the secure wallet file which stores the AWS credentials
This determines the appropriate Oracle Secure Backup library file for your OS platform and database version and downloads it to the specified location (recommended $ORACLE_HOME/lib) and creates a config file and the wallet file:$ java -jar osbws_install.jar \ > -AWSID AKI***************QA \ > -AWSKey no/MD*******************************upxK \ > -otnUser pane@pythian.com \ > -walletDir $ORACLE_HOME/dbs/osbws_wallet \ > -libDir $ORACLE_HOME/lib Oracle Secure Backup Web Service Install Tool, build 2015-06-22 AWS credentials are valid. S3 user already registered. Registration ID: 17d*****-0***-4***-8***-41e******ccc S3 Logging Bucket: oracle-log-pane-1 Validating log bucket location ... Validating license file ... Oracle Secure Backup Web Service wallet created in directory /u01/app/oracle/product/12.1.0/dbhome_2/dbs/osbws_wallet. Oracle Secure Backup Web Service initialization file /u01/app/oracle/product/12.1.0/dbhome_2/dbs/osbwsCDB121.ora created. Downloading Oracle Secure Backup Web Service Software Library from file osbws_linux64.zip. Downloaded 27151475 bytes in 40 seconds. Transfer rate was 678786 bytes/second. Download complete.
At this point we're ready to backup directly to the AWS S3 cloud.$ ls -ltr $ORACLE_HOME/lib | tail -1 -rw-r--r--. 1 oracle oinstall 86629108 Sep 3 09:05 libosbws.so $ ls -ltr $ORACLE_HOME/dbs | tail -1 -rw-r--r--. 1 oracle oinstall 204 Sep 3 09:04 osbwsCDB121.ora $ cat $ORACLE_HOME/dbs/osbwsCDB121.ora OSB_WS_HOST=https://s3.amazonaws.com OSB_WS_LICENSE_ID=17d*****-0***-4***-8***-41e******ccc OSB_WS_WALLET='location=file:/u01/app/oracle/product/12.1.0/dbhome_2/dbs/osbws_wallet CREDENTIAL_ALIAS=pane_aws' $ ls -l $ORACLE_HOME/dbs/osbws_wallet total 12 -rw-r--r--. 1 oracle oinstall 10228 Sep 3 09:04 cwallet.sso
Using with RMAN
Sending the RMAN backup pieces to the AWS S3 storage is as simple as backing via the normal RMAN commands but to the SBT_TAPE device. Of course the new OSB library file and configuration file to use must be specified. For example we can backup in a single run block without over-riding any of our existing RMAN configuration:
backup device type SBT_TAPE tablespace users;
RMAN> run {
2> allocate channel aws_s3 type sbt
3> parms='SBT_LIBRARY=libosbws.so,SBT_PARMS=(OSB_WS_PFILE=/u01/app/oracle/product/12.1.0/dbhome_2/dbs/osbwsCDB121.ora)';
4> backup tablespace users;
5> }
allocated channel: aws_s3
channel aws_s3: SID=21 device type=SBT_TAPE
channel aws_s3: Oracle Secure Backup Web Services Library VER=3.15.7.14
Starting backup at 03-SEP-15
channel aws_s3: starting full datafile backup set
channel aws_s3: specifying datafile(s) in backup set
input datafile file number=00006 name=/u01/app/oracle/oradata/CDB121/users01.dbf
channel aws_s3: starting piece 1 at 03-SEP-15
channel aws_s3: finished piece 1 at 03-SEP-15
piece handle=0aqg7f8h_1_1 tag=TAG20150903T095737 comment=API Version 2.0,MMS Version 3.15.7.14
channel aws_s3: backup set complete, elapsed time: 00:00:15
Finished backup at 03-SEP-15
Starting Control File and SPFILE Autobackup at 03-SEP-15
piece handle=c-3847224663-20150903-01 comment=API Version 2.0,MMS Version 3.15.7.14
Finished Control File and SPFILE Autobackup at 03-SEP-15
released channel: aws_s3
RMAN>
And to verify:
Notice that the first backup was a local one run earlier and shows a local backup piece (file). The second shows that the media was "s3.amazonaws.com". The "oracle-data-pane-1" is the "bucket" or logical container automatically created within the Amazon S3. If we want to make the backup command automatically use the AWS S3 SBT_TAPE it's trivial to do using the RMAN CONFIGURE command:RMAN> list backup of tablespace users; List of Backup Sets =================== ------- ---- -- ---------- ----------- ------------ --------------- 6 Full 1.34M DISK 00:00:00 03-SEP-15 BP Key: 6 Status: AVAILABLE Compressed: NO Tag: TAG20150903T094342 Piece Name: /u01/app/oracle/product/12.1.0/dbhome_2/dbs/06qg7eee_1_1 List of Datafiles in backup set 6 File LV Type Ckp SCN Ckp Time Name ---- -- ---- ---------- --------- ---- 6 Full 1366827 03-SEP-15 /u01/app/oracle/oradata/CDB121/users01.dbf BS Key Type LV Size Device Type Elapsed Time Completion Time ------- ---- -- ---------- ----------- ------------ --------------- 8 Full 1.50M SBT_TAPE 00:00:02 03-SEP-15 BP Key: 8 Status: AVAILABLE Compressed: NO Tag: TAG20150903T095453 Handle: 08qg7f3d_1_1 Media: s3.amazonaws.com/oracle-data-pane-1 List of Datafiles in backup set 8 File LV Type Ckp SCN Ckp Time Name ---- -- ---- ---------- --------- ---- 6 Full 1367173 03-SEP-15 /u01/app/oracle/oradata/CDB121/users01.dbf RMAN>
And really that's all there is to it. Of course you can perform more advanced RMAN commands such as allocating multiple channels, etc. And we're free to perform any combination of local backups and/or cloud backups to the SBT_TAPE device type:RMAN> configure channel device type sbt parms='SBT_LIBRARY=libosbws.so, 2> SBT_PARMS=(OSB_WS_PFILE=/u01/app/oracle/product/12.1.0/dbhome_2/dbs/osbwsCDB121.ora)'; using target database control file instead of recovery catalog new RMAN configuration parameters: CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' PARMS 'SBT_LIBRARY=libosbws.so,SBT_PARMS=(OSB_WS_PFILE=/u01/app/oracle/product/12.1.0/dbhome_2/dbs/osbwsCDB121.ora)'; new RMAN configuration parameters are successfully stored RMAN> backup device type sbt tablespace users; Starting backup at 03-SEP-15 released channel: ORA_DISK_1 allocated channel: ORA_SBT_TAPE_1 channel ORA_SBT_TAPE_1: SID=16 device type=SBT_TAPE channel ORA_SBT_TAPE_1: Oracle Secure Backup Web Services Library VER=3.15.7.14 channel ORA_SBT_TAPE_1: starting full datafile backup set channel ORA_SBT_TAPE_1: specifying datafile(s) in backup set input datafile file number=00006 name=/u01/app/oracle/oradata/CDB121/users01.dbf channel ORA_SBT_TAPE_1: starting piece 1 at 03-SEP-15 channel ORA_SBT_TAPE_1: finished piece 1 at 03-SEP-15 piece handle=0eqg7ft3_1_1 tag=TAG20150903T100834 comment=API Version 2.0,MMS Version 3.15.7.14 channel ORA_SBT_TAPE_1: backup set complete, elapsed time: 00:00:15 Finished backup at 03-SEP-15 Starting Control File and SPFILE Autobackup at 03-SEP-15 piece handle=c-3847224663-20150903-03 comment=API Version 2.0,MMS Version 3.15.7.14 Finished Control File and SPFILE Autobackup at 03-SEP-15 RMAN>
And obviously restoring works exactly the same way. IMPORTANT: Note here that none of the backups are encrypted or compressed. Not having to encrypt backups is the first major functional difference from the Oracle Database Backup Service (OSDB) where encryption in mandatory. Either encryption and/or compression can be used. Normally, backup encryption requires the Oracle Advanced Security Option however one exemption to that is when backing up using Oracle Secure Backup. Similarly, "basic" RMAN backup compression is included with the database without any additional licenses. However the additional compression options (such as "HIGH", "MEDIUM", or "LOW") usually do require the Oracle Advanced Compression option but are included as "Special-use licensing" of the Oracle Secure Backup product. Compressing prior to backup is generally highly recommended. While this will consume local CPU cycles, it will minimize transfer time through the internet and S3 space used.RMAN> list backup summary; List of Backups =============== Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag ------- -- -- - ----------- --------------- ------- ------- ---------- --- 1 B F A SBT_TAPE 01-SEP-15 1 1 NO TAG20150901T123222 2 B F A SBT_TAPE 01-SEP-15 1 1 NO TAG20150901T123222 3 B F A SBT_TAPE 01-SEP-15 1 1 NO TAG20150901T123222 4 B F A SBT_TAPE 01-SEP-15 1 1 NO TAG20150901T123222 5 B F A SBT_TAPE 01-SEP-15 1 1 NO TAG20150901T150814 6 B F A DISK 03-SEP-15 1 1 NO TAG20150903T094342 7 B F A DISK 03-SEP-15 1 1 NO TAG20150903T094343 8 B F A SBT_TAPE 03-SEP-15 1 1 NO TAG20150903T095453 9 B F A SBT_TAPE 03-SEP-15 1 1 NO TAG20150903T095737 10 B F A SBT_TAPE 03-SEP-15 1 1 NO TAG20150903T095752 11 B F A DISK 03-SEP-15 1 1 NO TAG20150903T100555 12 B F A DISK 03-SEP-15 1 1 NO TAG20150903T100557 13 B F A SBT_TAPE 03-SEP-15 1 1 NO TAG20150903T100834 14 B F A SBT_TAPE 03-SEP-15 1 1 NO TAG20150903T100850
Advanced Configuration
Generally, the base parameters should be sufficient. However, if further customization is required there are a number of optional parameters that can be added to the configuration/initialization file (in this example "$ORACLE_HOME/dbs/osbwsCDB121.ora"). An easy way to look for the available parameters including hidden parameters is to search through the OSB Cloud Module library file. For example:$ strings $ORACLE_HOME/lib/libosbws.so |grep OSB_ OSB_WS_PFILE OSB_WS_HOST OSB_WS_PROXY OSB_WS_BUCKET OSB_WS_LOCATION OSB_WS_CHUNK_SIZE OSB_WS_LICENSE_ID OSB_WS_LICENSE_MAX_SESSIONS OSB_WS_WALLET OSB_WS_VIRTUAL_HOST OSB_WS_PRIVATE_CLOUD _OSB_WS_ALLOCBUF_DISABLE _OSB_WS_ACCESSLOG_RETENTION_TIME _OSB_WS_SEND_BUF_SIZE _OSB_WS_RECV_BUF_SIZE _OSB_WS_SDU_SIZE _OSB_WS_TDU_SIZE _OSB_WS_REUSE_CONNECTION _OSB_WS_SESSION_RETENTION_TIME _OSB_WS_CHECKER_RETENTION_TIME _OSB_WS_PURGE_LIMIT _OSB_WS_CLEANER _OSB_WS_CLEANER_RETENTION_TIME _OSB_WS_RUN_CLEANER _OSB_WS_ACCESSLOG_CLEANER _OSB_WS_ACCESSLOG _OSB_WS_CONNECT_TIMEOUT _OSB_WS_RESPONSE_TIMEOUT _OSB_WS_SEND_TIMEOUT _OSB_WS_RETRY_WAIT_TIME _OSB_WS_UPLOAD_DELAY _OSB_WS_EVENT _OSB_WS_TRACE_LEVEL _OSB_WS_VALIDATE_CHUNK _OSB_WS_100_CONTINUE _OSB_WS_DEFERRED_DELETE _OSB_WS_AUTH_SCHEME _OSB_WS_LOG_BUCKET _OSB_WS_REDIRECT _OSB_WS_BUFFER_WRITE _OSB_WS_BUFFER_READ _OSB_WS_USE_IPV6 _OSB_WS_NO_SSL _OSB_WS_VALIDATE_CERT
Secure Transfer
Similar to how backups using the OSB cloud module do not require encryption, they also transfer the data without SSL security by default. This is apparent from a simple Linux netstat command while the backup is running:However we can easily remedy this by changing the URL in the configuration/initialization file to use the "https" address:$ netstat | grep http tcp 0 299300 ORALINUX.localdom:46151 s3-1-w.amazonaws.c:http ESTABLISHED
After adjusting, starting a new RMAN session (critical) and re-running the same backup command now shows a secure SSL (or HTTPS) connection:$ head -1 $ORACLE_HOME/dbs/osbwsCDB121.ora OSB_WS_HOST=https://s3.amazonaws.com
One would think that the undocumented parameter "_OSB_WS_NO_SSL" (which is a boolean accepting only TRUE or FALSE as possible values) might also effect which connection is used though experimentation showed no effect.$ netstat | grep http tcp 0 299300 ORALINUX.localdom:34048 s3-1-w.amazonaws.:https ESTABLISHED tcp 0 0 ORALINUX.localdom:25700 s3-1-w.amazonaws.:https TIME_WAIT
Viewing Usage
Viewing the files stored and data usage on the AWS S3 at first is not as intuitive as one might think. From the AWS dashboard selecting S3 shows the "buckets" (logical storage containers) and allows you to drill down into each to see the actual files stored:
Conclusion
Just like with the Oracle Database Backup Service, getting going with the Oracle Secure Backup Cloud Module and Amazon Web Services is very simple. It really can be implemented quickly and easily allowing for RMAN backups to be written directly to off-site storage almost immediately. However unlike OSDB, the AWS OSB module is not secure by default. With Oracle's service, the data must be encrypted and the transfer is secured by default and hence they can live up to their claim that your data is secure in flight and at rest. However with the AWS OSB module, by default the reverse is true. Though of course, both HTTPS data transfer and backup encryption can be enabled if desired. On a positive note, the ability to generate AWS ID and Key values (or "credentials") specific for each backup configuration can be considered a security advantage. Apart from the security differences, functionally the two services are almost identical as would be expected. After all, the OSDB module appears to simply be a clone of the OSBWS module introduced about half a decade ago. The one big caveat of the AWS OSB module is the licensing cost. The Oracle Secure Backup Cloud module is licensed per RMAN channel. Channel based licenses can be shared among multiple databases however. And overall the S3 storage service is more functional and mature than Oracle's newer public cloud service. AWS includes some nice features with respect to geographic data placement and replication. The bottom line is that the Oracle Secure Backup Cloud Module is a great technical solution for implementing off-site Oracle RMAN backups (likely to compliment your on-premise backups) and specifically putting them onto the AWS S3. Functionally it seems great. However, the additional licensing costs is likely the road block for most small to mid-sized organizations.Additional References
- Oracle Secure Backup
- OSB & AWS White paper
- Oracle Secure Backup Cloud Module for Amazon S3 Download
- My Oracle Support: "OSB Cloud Module - FAQ (Doc ID 740226.1)"
No Comments Yet
Let us know what you think