Share this
by Gleb Otochkin on Aug 1, 2017 12:00:00 AM
Not long ago, Oracle adopted the Docker engine as one of the accepted platforms for its products and published a set of scripts and examples in the Oracle GitHub repository. This includes sets for rapid deployment of Oracle databases, Weblogic and for a number of other products. I tried some of the published docker implementations including Oracle database using my sandbox and it worked pretty well for me. Among the published containers, I didn't find an example for Oracle Data Integrator (ODI) and decided to make one just for myself. I created it and found it to be useful when you need just to test one or another ODI scenario and wipe everything out after the tests. I've decided to share my scripts with you and describe how it had been made. Just before publishing the post, I checked the Oracle git repository for docker and found a new “OracleDataIntegrator” template there. So, now we have the proper version from Oracle but even having the “official” deployment, you may find my version useful and I hope it may help you to understand the Docker engine. To build my version, I used the Oracle developed images for Oracle Linux, Oracle Java, and Oracle Database as a basis for ODI making only minor modifications to have the final product better suited to my needs. You may find that other solutions are better and perhaps more adapted for your needs, but please keep in mind that my example is purely educational and not designed for any production use. First we need the Docker engine on your machine. Depending on your platform, you need to download an appropriate engine from the Docker site and install it. The installation is pretty straightforward and easy. If you are on Linux you may need to add your user to the "docker" group. For example, I am adding user oracle to the group to be able to run docker containers and have access to the docker registry from the Oracle account:[root@vm129-132 ~]# gpasswd -a oracle docker
Adding user oracle to group docker
[root@vm129-132 ~]#
And make sure the docker service is up and running:
[root@vm129-132 ~]# systemctl status docker.service
? docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/docker.service.d
??docker-sysconfig.conf
Active: active (running) since Tue 2017-06-06 12:29:35 EDT; 1 months 24 days ago
Then you need to clone or download the Oracle git repository for docker containers to be able to build necessary base images before starting with the ODI. On your system with docker, you either run "git clone https://github.com/oracle/docker-images.git" or go to the
https://github.com/oracle/docker-images in your browser, download and unzip the full archive. There you will be able to find scripts to build docker images with all the different sets of software. Here is my listing after cloning it from the Oracle git repository:
[oracle@vm129-132 docker-images]$ ls -l
total 68
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 ContainerCloud
drwxr-xr-x. 6 oracle oinstall 4096 May 11 16:42 GlassFish
drwx------. 2 root root 16384 May 11 16:54 lost+found
drwxr-xr-x. 2 oracle oinstall 4096 May 11 16:42 MySQL
drwxr-xr-x. 7 oracle oinstall 4096 May 11 16:42 NoSQL
drwxr-xr-x. 5 oracle oinstall 4096 May 11 16:42 OpenJDK
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 OracleCoherence
drwxr-xr-x. 5 oracle oinstall 4096 Jul 18 14:40 OracleDatabase
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 OracleHTTPServer
drwxr-xr-x. 6 oracle oinstall 4096 May 11 16:42 OracleJava
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 OracleTSAM
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 OracleTuxedo
drwxr-xr-x. 5 oracle oinstall 4096 May 11 16:42 OracleWebLogic
-rw-r--r--. 1 oracle oinstall 1588 Jul 17 09:10 README.md
(The listing represents the state as of May 2017 and may look different now) Let's continue and go step by step preparing our images. To understand the process, let me recall what we need for ODI in standalone mode. We need a Linux box with installed JDK or JRE and Oracle database as a repository. Also, if we plan to use the ODI Studio, it makes sense to have either X window or a VNC server installed on the box. We start with building a Linux image with Java JRE or JDK for our ODI. Oracle provides "OracleJava" docker configuration where we can build an image with Java 7 or 8:
[oracle@vm129-132 docker-images]$ ll OracleJava
total 20
drwxr-xr-x. 2 oracle oinstall 4096 May 11 16:42 java-7
drwxr-xr-x. 2 oracle oinstall 4096 May 12 10:56 java-8
-rw-r--r--. 1 oracle oinstall 1886 May 11 16:42 README.md
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 windows-java-7
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 windows-java-8
I used Oracle JDK 8 instead of a server JRE distribution. To make that happen, I've slightly modified the Dockerfile in the "OracleJava/java-8" directory replacing the server JRE distribution by JDK. It is optional and you may choose to keep JRE instead. In my case, the original string in the file was replaced from:
ENV JAVA_PKG=server-jre-8u*-linux-x64.tar.gz
to:
ENV JAVA_PKG=jdk-8u*-linux-x64.tar.gz
After that, I downloaded the JDK from the
Oracle OTN site, put it in the folder and ran the build.sh script. The script prepares an image with Oracle Linux 7 in minimal configuration with the Oracle JDK 8 installed.
[oracle@vm129-132 java-8]$ ll
total 181204
-rwxr-xr-x. 1 oracle oinstall 47 May 11 16:42 build.sh
-rw-r--r--. 1 oracle oinstall 644 May 11 16:42 Dockerfile
-rw-r--r--. 1 oracle oinstall 185540433 May 12 10:43 jdk-8u131-linux-x64.tar.gz
-rw-r--r--. 1 oracle oinstall 263 May 11 16:42 server-jre-8u101-linux-x64.tar.gz.download
[oracle@vm129-132 java-8]$ cp Dockerfile Dockerfile.orig
[oracle@vm129-132 java-8]$ vi Dockerfile
[oracle@vm129-132 java-8]$ ./build.sh
Sending build context to Docker daemon 185.5 MB
Step 1 : FROM oraclelinux:7-slim
7-slim: Pulling from library/oraclelinux
...............
Successfully built 381e0684cea2
[oracle@vm129-132 java-8]$
[oracle@vm129-132 java-8]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/serverjre 8 381e0684cea2 10 weeks ago 490.6 MB
oraclelinux 7-slim 442ebf722584 3 months ago 114.4 MB
The next step is optional, but it makes life a bit easier. We need a number of tools to be installed on top of our minimal installation. In Docker, you can create a container, modify it and save as another image using command "commit". The beauty of this that you are not really doubling your space consumption by those incremental images. Docker will just add those extra changes you've made as a separate volume behind the scenes and will use original image plus your changes for the newly created image. You may think about it as a set of "snapshots". So, I created a container with name "orajrevnc" from the "oracle/serverjre:8" image, installed additional packages including some diagnostic, configuration and other useful packages like a VNC server, vi editor, and others.
[oracle@vm129-132 java-8]$ docker run --name serverjrevnc -p 5901:5901 -ti oracle/serverjre:8
bash-4.2#
bash-4.2# yum -y install vim
bash-4.2# yum -y install net-tools
bash-4.2# yum -y install telnet
bash-4.2# yum -y install strace
bash-4.2# yum -y install gcc
bash-4.2# yum -y install xterm
.....
After that, I used the container to commit it as a new image and saved it as "oracle/serverjrevnc:8".
[oracle@vm129-132 java-8]$ docker commit serverjrevnc oracle/serverjrevnc:8
sha256:ac5b4d85fccc5427c92e65a6c3b1c06e3e8d04ffbe7725bcca1759a2165353d7
[oracle@vm129-132 java-8]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/serverjrevnc 8 ac5b4d85fccc 3 minutes ago 1.661 GB
oracle/serverjre 8 381e0684cea2 34 minutes ago 490.6 MB
oraclelinux 7-slim 442ebf722584 3 weeks ago 114.4 MB
[oracle@vm129-132 java-8]$
The container "orajrevnc" can be deleted now :
[oracle@vm129-132 java-8]$ docker stop orajrevnc
[oracle@vm129-132 java-8]$ docker rm orajrevnc
Now we have a basic Oracle Linux image with the java installed and all necessary tools and utilities we need. The image can be used for the next step as a basis for our Oracle Database image. The database will serve as a repository for our ODI. We go to the folder "docker-images/OracleDatabase/dockerfiles/ " where we replace a line "FROM oraclelinux:7-slim" by "FROM oracle/serverjrevnc:8", download the 12.2.0.1 EE database software from
"Oracle site" and build the image for Oracle Database 12.2.0.1.
[oracle@vm129-132 java-8]$ cd ../../OracleDatabase/dockerfiles/
[oracle@vm129-132 dockerfiles]$ ll
total 16
drwxr-xr-x. 2 oracle oinstall 4096 May 11 16:42 11.2.0.2
drwxr-xr-x. 2 oracle oinstall 4096 Jul 18 14:18 12.1.0.2
drwxr-xr-x. 2 oracle oinstall 4096 Jul 25 14:37 12.2.0.1
-rwxr-xr-x. 1 oracle oinstall 3975 May 11 16:42 buildDockerImage.sh
[oracle@vm129-132 dockerfiles]$ vi 12.2.0.1/Dockerfile.ee
[oracle@vm129-132 dockerfiles]$ ll 12.2.0.1/*.zip
total 3372836
-rw-r--r--. 1 oracle oinstall 3453696911 May 12 09:26 linuxx64_12201_database.zip
[oracle@vm129-132 dockerfiles]$ ./buildDockerImage.sh -v 12.2.0.1 -e
Checking if required packages are present and valid...
......
Build completed in 906 seconds.
[oracle@vm129-132 dockerfiles]$
Here is the new list of images we have after building the Oracle Database image.
[oracle@vm129-132 dockerfiles]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/database 12.2.0.1-ee 91aaea30a651 28 minutes ago 16.18 GB
oracle/serverjrevnc 8 2d7be7d163dc 50 minutes ago 1.543 GB
oracle/serverjre 8 c2c247029798 About an hour ago 490.5 MB
oraclelinux 7-slim 08a01cc7be97 5 weeks ago 114.4 MB
Having all necessary images prepared we can start building our ODI. I have prepared the scripts to build the Docker image and published them on
"https://github.com/gotochkin/docker-images". You can either download or clone them using the git software. Let's have a look at the scripts and discuss what they do:
[oracle@vm129-132 ~]$ ls -l docker-images/ODI/dockerfiles/
total 8
drwxr-xr-x. 2 oracle oinstall 4096 Jul 27 14:18 12.2.1.2.6
-rwxr-xr-x. 1 oracle oinstall 3195 Jul 27 11:54 buildDockerImage.sh
[oracle@vm129-132 ~]$ ls -l docker-images/ODI/dockerfiles/12.2.1.2.6/
total 2390372
-rw-r--r--. 1 oracle oinstall 277 May 9 14:42 Checksum.standalone
-rw-r--r--. 1 oracle oinstall 8572 Jul 17 12:43 createOdiDomainForStandaloneAgent.py
-rw-r--r--. 1 oracle oinstall 4270 Jul 26 10:41 Dockerfile.standalone
-rw-r--r--. 1 oracle oinstall 1289 May 9 14:49 install.file
-rw-r--r--. 1 oracle oinstall 6477 Jul 14 13:03 oracledi.gz
-rw-r--r--. 1 oracle oinstall 55 May 8 13:57 oraInst.loc
-rw-rw-r--. 1 oracle oinstall 2695 May 15 15:08 rcuResponseFile.properties
-rw-r--r--. 1 oracle oinstall 7920 Jul 27 14:18 README.md
-rwxr-xr-x. 1 oracle oinstall 5332 Jul 14 15:51 runOracle.sh
-rwxr-xr-x. 1 oracle oinstall 4406 Jul 27 11:08 startAgent.sh
[oracle@vm129-132 ~]$
-- buildDockerImage.sh - A script to build the ODI image. It takes parameter -v as the version for the ODI (so far only 12.2.1.2.6) and -t to tell that we are going to configure ODI Agent in standalone mode. -- Checksum.standalone - to verify checksum for the downloaded installation files which you will need to put into the docker-images/ODI/dockerfiles/12.2.1.2.6 directory -- createOdiDomainForStandaloneAgent.py - A python script to create a domain for an ODI standalone agent. The original script had been taken from the
"Oracle tutorial" and slightly modified for our needs. -- Dockerfile.standalone - The instructions/script for Docker how to build the image for the ODI standalone agent. -- install.file - A response file for ODI silent installation. -- oracledi.gz - Gzipped files with ODI connection properties, it will be uncompressed into the $HOME/.odi directory. -- oraInst.loc - The configuration for Oracle Inventory. -- rcuResponseFile.properties - A parameter file for Repository Creation Utility. -- README.md - An instruction how to build an image. -- runOracle.sh - A startup script for the Oracle database which is going to be used as a repository. -- startAgent.sh - A script to configure and start the ODI agent. We need to download ODI installation files fmw_12.2.1.2.6_odi_Disk1_1of2.zip and fmw_12.2.1.2.6_odi_Disk1_2of2.zip files from
"Oracle OTN site" and put them to the docker-images/ODI/dockerfiles/12.2.1.2.6 folder. At last, everything is ready and we can build the image using the buildDockerImage.sh script. Of course, you can build it without the script since it is only a wrapper for "docker build" command. The script just makes it bit easier.
[oracle@vm129-132 dockerfiles]$ ./buildDockerImage.sh -v 12.2.1.2.6 -t
..............
Successfully built 239bdf178fbe
ODI Docker Image for 'standalone' version 12.2.1.2.6 is ready to be extended:
--> oracle/odi:12.2.1.2.6-standalone
Build completed in 1032 seconds.
We can see the built image on the list:
[oracle@vm129-132 dockerfiles]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/odi 12.2.1.2.6-standalone 239bdf178fbe 24 seconds ago 23.73 GB
oracle/database 12.2.0.1-ee 91aaea30a651 About an hour ago 16.18 GB
oracle/serverjrevnc 8 2d7be7d163dc About an hour ago 1.543 GB
oracle/serverjre 8 c2c247029798 About an hour ago 490.5 MB
oraclelinux 7-slim 08a01cc7be97 5 weeks ago 114.4 MB
We are ready to create our container with ODI. When we create the container it will do several steps which can be in general listed as: -- Create a container with Oracle Linux 7 with JDK and other supplemental packages. -- Create or start an Oracle database. -- Create an ODI repository, if it is not created already. -- Configure an ODI agent in the repository or adjust hostname for the agent if it has been already configured in the repository. -- Create a Weblogic domain for standalone ODI agent. -- Start the agent. We have an option to create a fresh repository database every time when we deploy a new container using the command:
[oracle@vm129-132 dockerfiles]$ docker run --name oditest -p 1521:1521 -p 5500:5500 -p 5901:5901 -p 5902:5902 --env ORACLE_BASE=/opt/oracle --env ORACLE_HOME=/opt/oracle/product/12.2.0.1/dbhome_1 oracle/odi:12.2.1.2.6-standalone
And the database will be created inside docker file system (docker FS). It is convenient when you want to have a fresh repository every time but it takes a time to create a new database and, as result, your deployment will be delayed. Or we can define a volume for database files out of the docker file system in which case you may reuse database for your containers again and again which can save some time during deployment.
[oracle@vm129-132 dockerfiles]$ docker run --name oditest -p 1521:1521 -p 5500:5500 -p 5901:5901 -p 5902:5902 -v /home/oracle/docker-images/OracleDatabase/oradata:/opt/oracle/oradata --env ORACLE_BASE=/opt/oracle --env ORACLE_HOME=/opt/oracle/product/12.2.0.1/dbhome_1 oracle/odi:12.2.1.2.6-standalone
Just be aware that if you want to use more than one container with the same Oracle database the scripts should be adapted. In the current implementation a new deployment will try to use the same repository. After executing the command you will see the log of creation and in the end, you get the container with a running standalone ODI agent.
[oracle@vm129-132 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
71969452f5b3 oracle/odi:12.2.1.2.6-standalone "/u01/app/oracle/star" 4 days ago Up 2 days 0.0.0.0:1521->1521/tcp, 0.0.0.0:5500->5500/tcp, 0.0.0.0:5901-5902->5901-5902/tcp, 5903/tcp oditest
[oracle@vm129-132 ~]$
Inside the container you can start a vnc server and run an ODI studio.
[oracle@vm129-132 ~]$ docker exec -ti oditest bash
[oracle@71969452f5b3 Middleware]$ vncserver
You will require a password to access your desktops.
Password:
After starting the ODI studio you get the usual questions like whether you want to import any settings or to allow send data to Oracle about ODI studio usage. You don't have anything to import from the previous installation since this is the first one, so, you can ignore and say "no". The studio is eventually up and you need to connect to the repository. When you push the button to connect to the repository, you will be asked if you want to store the credentials in a wallet.
You have an option to refuse and use the pre-created connection and saved credentials.
Ideally this will make the usage bit easier and convenient. Of course, for any kind of production development it is strongly recommended to use a wallet and a proper password. If you're someone making your first steps in Docker, I hope this article has been helpful. In my opinion, Docker can be extremely useful for test deployments. The persistent database files makes the deployment easy and quick. I have some reservations about using Docker for any production deployments of Oracle databases, but that discussion deserves a dedicated post. Stay tuned.
No Comments Yet
Let us know what you think