Securing Oracle monitoring and backup scripts

by Simon Pane on Jan 15, 2016 12:00:00 AM

Background

Almost every DBA writes and uses various custom scripts to monitor and backup their Oracle databases. However, finding the optimal and most secure way to connect to the database is often not prioritized. The short summary is that having your script put the username/password (credentials) in any sort of variables or command arguments is the "bad way". Using an "Oracle External Password Store" (SEPS) or Oracle Wallet is the "better way". Yet this technology which has been around since Oracle 10gR2 and which does not require the Advanced Security Option is often not used.

Common Mistakes

Many DBAs will store credentials in their actual scripts. Sometimes obfuscating the actual password through some custom mechanism or script. For example, the following is a simplified version of an RMAN backup script found at a client site:

#!/bin/bash
 export DB_USER=backup_user
 export DB_PASS=`~/.secure_pwd_extractor`
 
 $ORACLE_HOME/bin/rman << EOF
  connect target $DB_USER/$DB_PASS
  shutdown immediate
  startup mount
  backup database;
  alter database open;
 EOF

The client thought that it was somewhat secure as the actual password wasn't used as a command line argument to RMAN and was stored in a Linux "hidden file" (starts with a period), which was protected by properly setting OS permissions. However, it dynamically extracted the password from a plain text file (based on the DB_USER environment variable). Another key problem was the fact that the environment variable was exported and hence was part of the environmental profile under which the database was started. The exported environment variables in this case can be a little bit of a security risk in a couple of ways: First of all, the complete operating environment including the exported environment variables under which the database is running are recorded by the listener when service is registered. Hence, they are visible in a listener " services" command with " set displaymode verbose":

Secondly, they may be recorded in OS process files. For example, the pmon process' operating environment or even the RMAN process' while running:

But most significantly the credentials can be extracted by anyone with access to the script file and/or the underlying credentials file.

A Better Approach

A better way to store database credentials for monitoring or backup scripts is to use a "Secure External Password Store" (SEPS) which relies on having the necessary credentials securely stored in an Oracle Wallet file. Typically a DBA might create their own plain text password listing file for use by scripts and batch jobs. Usually with three columns: 1) DB_NAME; 2) DB_USERNAME; 3) DB_PASSWORD. The Oracle Wallet is structured exactly the same way except:

The file is protected with 3DES encryption.
The DB_NAME is really an OracleNET Service Name meaning you can have multiple aliases for the same database.
The passwords are never exposed.
A separate Wallet password is required to manipulate the file's contents.
Control on whether the Oracle Wallet file is tied to the local machine or whether it can be copied to and used on other machines.

The advantages of this approach include:

No clear text password in any scripts or files.
No possible exposure of passwords by the listener or process operating environments.
Control on whether the Oracle Wallet file can be copied and used on another machine.

The last point is actually a complex one. A Wallet can be created as an " auto_login" wallet (done by default). To secure it to only work on the local server, it can be changed to " auto_login_local". However, there are various issues, limitations, and 12c bugs with the additional functionality that Oracle provides. A separate article goes into this in detail.

Setup

Setting up a "Secure External Password Store" and Oracle Wallet is actually quite quick and easy: 1) Adjust the sqlnet.ora file to point to an Oracle Wallet location. For example, add the following to the sqlnet.ora file (assuming that the specified directory exists):

WALLET_LOCATION =
  (SOURCE =
  (METHOD = FILE)
  (METHOD_DATA =
  (DIRECTORY = /u01/app/oracle/wallet)
  )
  )
 SQLNET.WALLET_OVERRIDE = TRUE

2) Create the Oracle Wallet files and add a credential. Two files will actually be created in the specified directory:

ewallet.p12 - the actual password protected Wallet file.
cwallet.sso - an additional file for auto-login credentials.

This can be done as either two separate commands or all in a single command:

$ mkstore -wrl "/u01/app/oracle/wallet" -create
 Oracle Secret Store Tool : Version 12.1.0.2
 Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.
 
 Enter password:
 Enter password again:
 
 $ mkstore -wrl "/u01/app/oracle/wallet" -createCredential ORCL scott
 Oracle Secret Store Tool : Version 12.1.0.2
 Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.
 
 Your secret/Password is missing in the command line
 Enter your secret/Password:
 Re-enter your secret/Password:
 Enter wallet password:
 Create credential oracle.security.client.connect_string1

Or as a single command:

$ mkstore -wrl "/u01/app/oracle/wallet" -create -createCredential ORCL scott
 Oracle Secret Store Tool : Version 12.1.0.2
 Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.
 
 Enter password:
 Enter password again:
 Your secret/Password is missing in the command line
 Enter your secret/Password:
 Re-enter your secret/Password:
 Create credential oracle.security.client.connect_string1
 $

Notice that the Wallet is secured by a password. And then the SCOTT credentials are stored within the Wallet. The Wallet password is required to manipulate contents - not for scripts to access the stored credentials. The first parameter after the " -createCredential" argument is an OracleNET Service Name. Just like with any database connection, here we can specify an OracleNET Service Name (from the tnsnames.ora file), or a full connection string, or an EZconnect string. Hence, we could add a second and third connection to the same database as:

$ mkstore -wrl "/u01/app/oracle/wallet" -createCredential localhost:1521/ORCL monitoring_user
 Oracle Secret Store Tool : Version 12.1.0.2
 Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.
 
 Your secret/Password is missing in the command line
 Enter your secret/Password:
 Re-enter your secret/Password:
 Enter wallet password:
 Create credential oracle.security.client.connect_string2
 
 $ mkstore -wrl "/u01/app/oracle/wallet" -createCredential 127.0.0.1:1521/ORCL batch_reporting
 Oracle Secret Store Tool : Version 12.1.0.2
 Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.
 
 Your secret/Password is missing in the command line
 Enter your secret/Password:
 Re-enter your secret/Password:
 Enter wallet password:
 Create credential oracle.security.client.connect_string3
 
 $

And to list the contents of the Oracle Wallet:

$ mkstore -wrl "/u01/app/oracle/wallet" -listCredential
 Oracle Secret Store Tool : Version 12.1.0.2
 Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.
 
 Enter wallet password:
 List credential (index: connect_string username)
 3: 127.0.0.1:1521/ORCL batch_reporting
 2: localhost:1521/ORCL monitoring_user
 1: ORCL scott
 $

Now any of the three can be used (from the same OS account: "oracle") depending on which OracleNET Service Name is referenced:

$ sqlplus /@ORCL
 
 SQL*Plus: Release 12.1.0.2.0 Production on Wed Jan 13 08:59:12 2016
 
 Copyright (c) 1982, 2014, Oracle. All rights reserved.
 
 Last Successful login time: Wed Jan 13 2016 08:56:56 -07:00
 
 Connected to:
 Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
 With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
 
 SQL> show user
 USER is "SCOTT"
 SQL>

$ sqlplus /@localhost:1521/ORCL
 
 SQL*Plus: Release 12.1.0.2.0 Production on Wed Jan 13 08:59:41 2016
 
 Copyright (c) 1982, 2014, Oracle. All rights reserved.
 
 Last Successful login time: Wed Jan 13 2016 08:57:25 -07:00
 
 Connected to:
 Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
 With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
 
 SQL> show user
 USER is "MONITORING_USER"
 SQL>

$ sqlplus /@127.0.0.1:1521/ORCL
 
 SQL*Plus: Release 12.1.0.2.0 Production on Wed Jan 13 09:00:14 2016
 
 Copyright (c) 1982, 2014, Oracle. All rights reserved.
 
 Last Successful login time: Wed Jan 13 2016 08:43:44 -07:00
 
 Connected to:
 Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
 With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
 
 SQL> show user
 USER is "BATCH_REPORTING"
 SQL>

$ sqlplus / as sysdba
 
 SQL*Plus: Release 12.1.0.2.0 Production on Wed Jan 13 09:01:12 2016
 
 Copyright (c) 1982, 2014, Oracle. All rights reserved.
 
 
 Connected to:
 Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
 With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
 
 SQL> show user
 USER is "SYS"
 SQL>

However, one might challenge the fact that the Oracle Wallet file itself that must be secured just as a plain text password file would need to be and that the risks of either being obtained by a "bad actor" are the same. That being said, there are still some benefits of the SEPS method:

Passwords are never maintained in plain text in scripts or (hidden) password files.
No risk of password exposure at the operating system process list.
No risk of password exposure in operating system process environmental files.
No risk of exposure from any bad actor with access to the script.
No risk of password exposure in OS backups maintained by sysadmins or backup vendors (though the files themselves may still be usable).

Regardless, using the SEPS and Oracle Wallet shouldn't make anything less secure. One could argue that security and risk is equal, but definitely not worse. However they are a few operational disadvantages:

The OracleNET Service Name forms the primary key of the entries in the Wallet and hence must be unique. So if another user credential is required for the same DB, an OracleNET alias will be required (as trying to add another user using the same OracleNET Service Name will generate the "Credential already exists" error based on the OracleNET Service Name not being unique).
Doesn't work after connecting to SQLPLUS using the "CONNECT" command.
Scripts are now dependent on listener availability for establishing a TNS connection instead of a BEQ. Though a workaround may be to connect directly to a DISPATCHER port.

Advantages Over OS Authentication

Using a SEPS and an Oracle Wallet may seem functionally similar to just using "OS Authentication". However it does pose a few differences or advantages. Most DBAs operate and implement scripts under the "Oracle software owner" account which is typically called "oracle" on Unix or Linux systems. And hence most are able to connect to the database using a SYSDBA connection. So one solution would be to use a dedicated OS user account specifically for monitoring or database backup scripts. And then rely on OS authentication for database connections. However this is often not done. And if running scripts from a remote server or centralized monitoring server then the REMOTE_OS_AUTHENT=TRUE parameter would have to be set which poses other security risks. Instead, using a SEPS allows for non-OS authenticated connections to a dedicated and minimally privileged database account even from the software owner (Oracle) account locally, or from any other account remotely.

Preventing the Secure Password Store Wallet File from Being Moved to Another Host

Oracle suggests that we can add additional security by tying the Oracle Wallet file to a specific host. See MOS document 1114599.1. However, this poses some restrictions and bugs with 12.1.0.2 specifically. As this discussion is complex, this follow-up article has been created. A good workaround (and a key point from the follow-up article) is to simply use an EZconnect connection in the Oracle Wallet file and to specify localhost or 127.0.0.1.

Summarizing Best Practices

Continue to properly secure the directory and file permissions of the Wallet files as you would a plain text password file. Further, why not make the entire wallet directory a hidden directory starting with a period. Remember that If the wallet file can be copied to another server then potentially the credentials within it can continue to be used. See the follow-up article for suggested techniques for securing access further.
Have scripts connect to the database using dedicated purpose based user accounts with minimal privileges (don't use SYS or SYSTEM).
Use OracleNET Service Name aliases (not duplicate entries, but aliases to an existing entry) in the tnsnames.ora file to allow multiple credentials for the same database.
For additional security add Wallet entries based on EZconnect strings using localhost or 127.0.0.1 instead of relying on OracleNET Service Names from the tnsnames.ora file.

Discover more about our expertise in the world of Oracle.

No Comments Yet

Let us know what you think

Securing Oracle monitoring and backup scripts

Background

Common Mistakes

A Better Approach

Setup

Advantages Over OS Authentication

Other Questions

Preventing the Secure Password Store Wallet File from Being Moved to Another Host

Summarizing Best Practices

No Comments Yet

Subscribe by email

Share this

Securing Oracle monitoring and backup scripts

Background

Common Mistakes

A Better Approach

Setup

Advantages Over OS Authentication

Other Questions

Preventing the Secure Password Store Wallet File from Being Moved to Another Host

Summarizing Best Practices

Share this

No Comments Yet

Subscribe by email