Oracle Direct NFS: How to Start

First of all, if you are using NFS to store Oracle database data files, I strongly advise you to enable Oracle Direct NFS (DNFS) to access these files. I will leave the long explanation on why you should do it out of this blog post. However, I’ll mention that the main reason is performance. (Google it or wait for my following blog posts for more information.) DNFS removes a serialization point over the traditional “kernelized” NFS. However, if you are not using NFS, DNFS is probably not of interest to you, and you should stop reading here. But, if you want to set it up, let’s go! :)

The only legitimate reason I can see for not using DNFS is a bug that doesn’t allow you to use it. If this is the case, I would advise you to resolve the problem (probably by applying a patch from Oracle Support) and enable DNFS.

How to set up

Reference: Step by Step – Configure Direct NFS Client (DNFS) on Linux (11g) [ID 762374.1]
One of DNFS’s greatest features is an easy way to enable/disable it and a relatively easy way to check if it’s enabled or not. It is worth mentioning that DNFS is available from Oracle 11G only. If your system is configured using NFS (see Oracle Support notes on how to configure it), use the following two commands to enable/disable DNFS.

• Enable

ln -s $ORACLE_HOME/lib/libnfsodm11.so $ORACLE_HOME/lib/libodm11.so

• Disable

ln -s $ORACLE_HOME/lib/libodmd11.so $ORACLE_HOME/lib/libodm11.so

Of course, you should stop all DBs that use the ORACLE_HOME, make a backup of the initial symbol link, etc. I’ll leave the precaution measures to you. The simple fact is that the two commands above are the only commands  you should know to enable/disable Oracle Direct NFS.

You would ask: How does Oracle know what NFS server to connect to, what IP to be used, etc? In the simplest configuration, Oracle retrieves all the information it needs from the /etc/mtab Linux configuration file. This file is populated by Linux each time you mount an NFS volume. The static volumes’ mount configuration is stored in an /etc/fstab file.

How to check

This is where the story gets a bit tricky. The Oracle Support note mentioned above [ID 762374.1] states the following:

alert log shows Direct NFS client is enabled by checking the below message
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0

However, I came across several cases where the message was in an alert.log file, but DNFS still wasn’t enabled.
I found that the easiest and most reliable way to verify if DNFS is enabled and working as expected is to check Oracle Instances process’ opened files list. Find a process that has opened at least one data file and run lsof Linux command against the process. I usually check the DBWR foreground process as it has some data files opened right from the instance start up time.

The following is an example of how an output should look like if the DNFS is in use:

-bash-3.2$ ps -ef | grep dbw
oracle   23126     1  0 06:53 ?        00:00:00 ora_dbw0_MYINST1
oracle   23128     1  0 06:53 ?        00:00:00 ora_dbw1_MYINST1
oracle   23246 22377  0 07:04 pts/0    00:00:00 grep dbw
-bash-3.2$ /usr/sbin/lsof -p 23126
COMMAND   PID   USER   FD   TYPE DEVICE      SIZE       NODE NAME
oracle  23126 oracle  cwd    DIR  104,3      4096   14811513 /u02/app/oracle/product/11.2.0.1/dbs
oracle  23126 oracle  rtd    DIR  104,3      4096          2 /
oracle  23126 oracle  txt    REG  104,3 210824714   15892486 /u02/app/oracle/product/11.2.0.1/bin/oracle
...
oracle 23126 oracle 32u IPv4 322388 TCP mydbhost:64553->mynfshost:nfs (ESTABLISHED) oracle 23126 oracle 33u IPv4 322389 TCP mydbhost:21549->mynfshost:nfs (ESTABLISHED) ...

Pay attention to the last two rows I highlighted. You should see just TCP/IP sockets opened by the process towards NFS server. There should not be any data files listed. If you see any of the data files (located on NFS) in the output, then the instance doesn’t use DNFS. BTW: I would appreciate if you provided an output with data files listed in the comments section below this blog post as I would like readers to see an example. Unfortunately, I don’t have a good (bad) example handy as all instances I have access to use DNFS today :).

At this point, I would like to end my very first DNFS-related blog post. I have a lot more to say about DNFS, but I’ll leave it for future blog posts. The following is just comments and questions I have about DNFS:

• NFS server must be configured to accept NFS clients’ requests coming from a non-standard port range.
• How to load balance DNFS traffic if you have several NICs (oranfstab file)?
• Use init.ora parameters (events 19392, 19394, 19396) to troubleshoot DNFS configuration.
• Don’t use the same NFS volumes names on different NFS servers (DNFS can’t work it out).
• Performance difference between kNFS & DNFS (testing results). Is there any real benefits using DNFS over kNFS?
• Good references list for DNFS information (MOS notes, Blog posts, etc.)
• Some other details you want here :) …

Let’s hope that, with all your support and a bit of free time, I will blog about the topics mentioned :)

Stay tuned folks,
Yury