Category Archives: Docker

Smaller Oracle Docker images

One of the important challenges with Docker is to get used to the image layers and the layered file system. It quickly happens that you unintentionally have too much data in an intermediate layer. Either log files, installation software or login credentials. Whereby the first two “only” blow up the Docker image unnecessarily, while the last point can be a major security vulnerability. It also happens to me when I build Docker images for Oracle Unified Directory. See my blog post on Oracle Unified Directory on Docker.

Problem

Each instruction in the Dockerfile adds a layer to the image, and you need to remember to clean up any artifacts you don’t need before moving on to the next layer. If you do use COPY or ADD in particular a clean up is not possible. Every credential file or software package which is copied during build will remain. Later attempts to remove the intermediate files will only result in the corresponding files no longer being visible in the next layers.

Although there is a way to work around this by using the new build parameter --squash. Squash does merge newly built layers into a single new layer. But --squash is only available in the latest Docker releases. In older releases not at all or at best as experimental feature. Beside this it also has some other downside eg. losing the history or intermediate layers, issue with ONBUILD command etc. Squash does also not help if you specify your credentials as build arguments via ARG.

So why not make small, secure and clean Docker image at first place.

Idea

Rather than put the software packages or credential files to the Docker build context and using COPY or ADD, we will download them in a RUN command using curl. But from where? Oracle software can not be downloaded unattended without credentials. And we do not want to set up a web server just for software, secrets, credentials, etc.

Hei, you are using Docker. Setting up a local web server is a pice of cake. 🙂

  • Put your software, credentials, etc in a dedicated folder
  • Run a Docker container with an Apache HTTP server and make sure it has access to the folder mentioned before
  • Change your Dockerfile to download the software or credentials for the HTTP server
  • Make sure that you docker build can get access to the intermediate HTTP server

Solution

Let’s see how I did use a local HTTP sever to set up small Oracle Unified Directory Docker images.

HTTP Server

Create a folder with all required Oracle software, patch’s etc. But make sure, that you Docker can use this folder as Docker volume.

ls -alh /Data/vm/docker/volumes/orarepo
total 5009896
drwxr-xr-x 11 oracle staff 352B 26 Mär 23:52 .
drwxr-xr-x 4 oracle staff 128B 26 Mär 22:50 ..
-rw-r--r-- 1 oracle staff 1,5G 26 Mär 23:03 p26269885_122130_Generic.zip
-rw-r--r-- 1 oracle staff 404M 26 Mär 22:55 p26270957_122130_Generic.zip
-rw-r--r-- 1 oracle staff 94M 26 Mär 22:49 p26540481_111230_Generic.zip
-rw-r--r-- 1 oracle staff 157M 26 Mär 22:45 p26724938_111170_Linux-x86-64.zip
-rw-r--r-- 1 oracle staff 56M 26 Mär 22:55 p27217121_904_Linux-x86-64.zip
-rw-r--r-- 1 oracle staff 52M 26 Mär 22:56 p27217289_180162_Linux-x86-64.zip
-rw-r--r-- 1 oracle staff 1,3M 26 Mär 22:44 p27438258_122130_Generic.zip
-rw-r--r-- 1 oracle staff 56M 26 Mär 22:54 p27478886_100000_Linux-x86-64.zip
-rw-r--r-- 1 oracle staff 52M 26 Mär 22:53 p27638647_180162_Linux-x86-64.zip

Get your revered HTTP server. For this case I do use the official Apache HTTP server-based on alpine linux. This image is way smaller and more than enough for our purpose.

docker pull httpd:alpine

alpine: Pulling from library/httpd
605ce1bd3f31: Pull complete
6e4ededbced2: Pull complete
03b3c72c9962: Pull complete
bf08478b6930: Pull complete
222d70b58166: Pull complete
Digest: sha256:80d69271825a27c41f41609707095a1cdec381d22f772511ae6e30156c2b788f
Status: Downloaded newer image for httpd:alpine

Start a container for the HTTP server. Define a hostname, volume and external http port. For more information on configuration, see the official httpd Docker image.

docker run -dit --hostname orarepo --name orarepo \
-p 8080:80 \
-v /Data/vm/docker/volumes/orarepo:/usr/local/apache2/htdocs/ \
httpd:alpine

Get the IP address with docker inspect of the orarepo for later use.

orarepo_ip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' orarepo)

A test via curl on command line curl http://localhost:8080 or with your favorite browser will show the files mentioned above. Be aware this test does access the HTTP container from your host network via exposed port 8080.

Dockerfile

Now we just have to adopt the Dockerfile to make sure it does get the software from the HTTP server orarepo. See excerpt from my Dockerfile.

...
RUN curl -f http://orarepo/p26270957_122130_Generic.zip \
-o /tmp/download/p26270957_122130_Generic.zip && \
...

To be more flexible and allow both local files as well download via curl I did extend my RUN command with an extra file check [ -s ... ]. In this case it first check’s if the file is available and not zero. If the file is not available it will use curl to download the file from orarepo. See excerpt from my Dockerfile.

...
COPY p26270957_122130_Generic.zip* /tmp/download/
...
RUN [ -s "/tmp/download/p26270957_122130_Generic.zip" ] || \
curl -f http://orarepo/p26270957_122130_Generic.zip \
-o /tmp/download/p26270957_122130_Generic.zip && \
...

If the OUD software package p26270957_122130_Generic.zip is part of the build context it will be copied to the image and used to build and setup OUD. In case it is not part of the build context the file check will fail and start to use curl.

Build using COPY

Let’s build the Docker image with the software package copied during build. Check the build context.

ls -alh

total 858168
drwxr-xr-x 9 oracle staff 288B 28 Mär 09:31 .
drwxr-xr-x 6 oracle staff 192B 19 Mär 14:19 ..
-rw-r--r-- 1 oracle staff 4,9K 27 Mär 21:40 Dockerfile
-rw-r--r-- 1 oracle staff 225B 19 Mär 11:18 install.rsp
-rw-r--r-- 1 oracle staff 63B 19 Mär 10:57 oraInst.loc
-rw-r--r-- 1 oracle staff 404M 28 Mär 09:31 p26270957_122130_Generic.zip
-rw-r--r-- 1 oracle staff 754B 12 Mär 14:18 p26270957_122130_Generic.zip.download
drwxr-xr-x 6 oracle staff 192B 20 Mär 11:46 scripts

And run docker build.

docker build -t oracle/oud:12.2.1.3.0-copy .

Build using curl

Now let’s build the docker image using curl and not the local software package. For this p26270957_122130_Generic.zip has to be removed from the build context. Additionally the Docker build requires the IP of the orarepo, which is used to download the software image.
Check the build context.

rm p26270957_122130_Generic.zip
ls -alh

total 858168
drwxr-xr-x 9 oracle staff 288B 28 Mär 09:31 .
drwxr-xr-x 6 oracle staff 192B 19 Mär 14:19 ..
-rw-r--r-- 1 oracle staff 4,9K 27 Mär 21:40 Dockerfile
-rw-r--r-- 1 oracle staff 225B 19 Mär 11:18 install.rsp
-rw-r--r-- 1 oracle staff 63B 19 Mär 10:57 oraInst.loc
-rw-r--r-- 1 oracle staff 754B 12 Mär 14:18 p26270957_122130_Generic.zip.download
drwxr-xr-x 6 oracle staff 192B 20 Mär 11:46 scripts

And run docker build with --add-host. Add host does use the variable defined above for the IP address of the orarepo.

docker build --add-host=orarepo:${orarepo_ip} -t oracle/oud:12.2.1.3.0-curl .

The Docker images

When we compare the two image we see, that they differ by around 400MB. More or less the size of the OUD software package.

docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/oud 12.2.1.3.0-curl f7c80fa69db3 2 minutes ago 754MB
oracle/oud 12.2.1.3.0-copy a5e1751d534d 7 minutes ago 1.18GB

Docker history for the image oracle/oud:12.2.1.3.0-copy does also show the size of the COPY layer.

docker history oracle/oud:12.2.1.3.0-copy
IMAGE CREATED CREATED BY SIZE COMMENT
...
07614f386e0f 16 minutes ago /bin/sh -c #(nop) COPY multi:b8206d7811ce917… 424MB
832c9d0bf308 34 hours ago /bin/sh -c #(nop) COPY multi:58a01d5459f0ac6… 20kB
9c9531205281 34 hours ago /bin/sh -c groupadd --gid 1000 oracle && … 8.29MB
96278dfe7c12 34 hours ago /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:…
...

Conclusion

With little effort it is possible to create secure and especially small Docker images. Creating a HTTP server to share software or credentials during build is a piece of cake. Reducing the docker image by 400MB is nice. Depending on the Oracle software this will be even more. Although the downside is, that this does not work for automated builds on Docker Hub. but I’m still working on that 🙂

References

Below you find a few references related to the topics discussed in this post:

Oracle Unified Directory on Docker

A bit a while ago I’ve started to use Docker for miscellaneous purposes. Not really an early adopter, but I still hope I caught the train just in time. 🙂 In one of my customer project, I did have to set up a couple of OUD instance to develop and test the transition from Oracle Directory Server Enterprise Edition (ODSEE) to Oracle Unified Directory (OUD). This did include more engineering and troubleshooting work as initially planned. So I eventually got to set up my OUD instances in docker containers rather than in dedicated Virtualbox VM. Unfortunately Oracle does not provide any Docker images or build templates for Oracle Unified Directory. Indeed they do have a bunch of official Docker configurations, images, and examples on GitHub for a couple of Oracle products. But just not for Oracle unified Directory. Ok, there is a issue requesting such a Docker image see Issue #656. Well… challenge accepted. I did build my own OUD GitHub Repository for OUD Docker deployments.

My GitHub repository oehrlis/docker-oud does contain the Docker build files to facilitate installation, configuration, and environment setup for Docker DevOps users. The project allows you to create two different types of docker images.

  • Standalone Oracle Unified Directory 12.2.1.3.0 to setup and run Oracle Unified Directory. This is the smaller image with only the OUD binaries used to set up and run an OUD directory or proxy server. Administration has to be done via dsconfig, ldapmodify or any other regular LDAP command line or GUI tools.
  • Collocated Oracle Unified Directory 12.2.1.3.0 and Oracle Fusion Middleware Infrastructure 12.2.1.3.0. A rather big docker image to setup and run an Oracle Unified Directory Server Manager (OUDSM). My intention was to use this docker image primarily for OUDSM. Nevertheless, it can also be used to build an OUD directory or proxy server which is operated in a WLS domain. So in Collocated Mode (Under Same Domain) or Non-Collocated Mode (Under Separate Domains).

To setup my Docker OUD images, I’ve tried to follow a few best practice, rules (Oracle’s golden rules for contributing to oracle/docker-images) as well hints by my workmates (Philipp Salvisberg and others).

  • Always aim to produce the smallest possible image. I did not push this to the maximum and start to remove unused components in the Oracle binaries. Oracle Fusion Middleware Infrastructure is currently outrageous large to only running OUDSM.
  • Separate persistent data from the image / container and put it on a volume. Or at least let the user decide, if he want to put it on a volume or not.
  • No public distribution of Docker images containing Oracle software. That’s a legal requirement. My docker build scripts do provide a couple of possibilities to install the software.
  • Allow flexible configuration via –build-arg or -e but provide useful default values.
  • Use Oracle Linux as the base image and install only as much as you need.
  • And much more…

Build Docker Images

The Docker images have to be build manually based on oehrlis/docker-oud from GitHub. To assist in building the images, you can use the scripts/buildDockerImage.sh script. See below for instructions and usage. The buildDockerImage.sh script is just a utility shell script to setup the docker build command and is an easy way for beginners to get started. Expert users are welcome to directly call docker build with their preferred set of parameters.

Usage of buildDockerImage.sh:

buildDockerImage.sh [-hv] [-t TYPE] [-o DOCKER_BUILD_OPTION]
-h Usage (this message)
-v Enable verbose mode
-t TYPE OUD image and installation type to build.
Possible types are:
OUD : Standalone Oracle Unified Directory Server
OUDSM : Collocated Oracle Unified Directory Server.
Default is type is OUD.
-o DOCKER_BUILD_OPTION Passes on Docker build option

Logfile : buildDockerImage.log

Due to license restrictions from Oracle, the Docker images containing Oracle software can not provided on a public Docker repository (see [OTN Developer License Terms](http://www.oracle.com/technetwork/licenses/standard-license-152015.html)). This is the reason why you have to build the images yourself and downloaded the required software prior image build. Alternatively it is possible to specify MOS credentials in scripts/.netrc or via build arguments. Using MOS download during image build will lead into smaller images, since the software will not be part of an intermediate container.

Obtaining Product Distributions

The software can either be downloaded from My Oracle Support, Oracle Technology Network (OTN) or Oracle Software Delivery Cloud (OSDC). The following steps will refer to the MOS software download to simplify the build process.

The following software is required for the Oracle Unified Directory Docker image:

  • Oracle Java Development Kit (JDK) 1.8 (1.8u152) Patch 26595894 for the OUD and OUDSM image
  • Oracle Unified Directory 12.2.1.3.0 Patch 26270957 for the OUD and OUDSM image
  • Oracle Fusion Middleware Infrastructure 12.2.1.3.0 Patch 26269885 just for OUDSM image

Manual Download Software

Simplest method to build the OUD or OUDSM image is to manually download the required software. However this will lead to bigger docker images, since the software is copied during build, which temporary blow up the container file-system. But its more safe because you do not have to store any MOS credentials. If you’ve enabled Docker experimental features, you could work around this and squash Squash newly built layers with docker build parameter --squash.

The corresponding links and checksum can be found in *.download files in the software folder. Alternatively the direct Oracle Support Download Links:

Copy all files to the software folder.

cp p26595894_180152_Linux-x86-64.zip docker-oud/software
cp p26270957_122130_Generic.zip docker-oud/software
cp p26269885_122130_Generic.zip docker-oud/software

Build the docker image either by using docker build or buildDockerImage.sh.

docker build -t oehrlis/oud -f Dockerfile.oud .
docker build -t oehrlis/oudsm -f Dockerfile.oudsm .

scripts/buildDockerImage.sh -v -t OUD
scripts/buildDockerImage.sh -v -t OUDSM

Automatic download with .netrc

The advantage of an automatic software download during build is the reduced image size. No additional image layers are created for the software and the final docker image is about 3GB smaller. But the setup script’s setup_oud.sh, setup_oud.sh and setup_oudsm.sh requires MOS credentials to download the software with using curl. Curl does read the credentials from the .netrc file in scripts folder. The .netrc file will be copied to /opt/docker/bin/.netrc, but it will be removed at the end of the build.

Create a .netrc file with the credentials for login.oracle.com.

echo "machine login.oracle.com login $MOS_USER password $MOS_PASSWORD" >docker-oud/scripts/.netrc

Build the docker image either by using docker build or buildDockerImage.sh.

docker build -t oehrlis/oud -f Dockerfile.oud .
docker build -t oehrlis/oudsm -f Dockerfile.oudsm .

scripts/buildDockerImage.sh -v -t OUD
scripts/buildDockerImage.sh -v -t OUDSM

Although this method has some security issues. The credentials will always remains in the intermediate layer. It is recommended to use a different approach discussed in the new blog post Smaller Oracle Docker images.

Automatic download with Build Arguments

This method is similar to the automatic download with .netrc file. Instead of manually creating a .netrc file it will created based on build parameters. Also with this method the .netrc file is deleted at the end.

Build the docker image with MOS credentials as arguments using docker build or buildDockerImage.sh.

docker build --build-arg MOS_USER=$MOS_USER \
--build-arg MOS_PASSWORD=$MOS_PASSWORD \
-t oehrlis/oud -f Dockerfile.oud .

scripts/buildDockerImage.sh -v -t OUD \

-o "--build-arg MOS_PASSWORD=$MOS_PASSWORD --build-arg MOS_USER=$MOS_USER"

The time taken to build the OUD or OUDSM image will depend on your internet speed. In any case it shouldn’t be more than a couple of minutes. Although this method has as well some security issues. The credentials will always remains in the intermediate layer. It is recommended to use a different approach discussed in the new blog post Smaller Oracle Docker images.

Next Steps

You are now the happy owner of OUD Docker images with a standalone and / or collocated Oracle Directory Server installations. The next step is to start using these Docker images to run your OUD containers and deploy different kind of OUD and OUDSM configurations. I’ll provide how to build the containers as well some “behind the seance” information in my upcoming blog posts about OUD on Docker. Stay tuned.

Files and References

Below you find a few references related to Oracle Unified Directory on Docker: