Category Archives: Linux

Oracle Unified Directory to go on Raspberry Pi Zero

Recently I ran out of movies on one of my longer train rides. Coincidentally, I had my Raspberry Pi Zero with me and thought, “There’s Java running on it, right?”. Doesn’t Oracle Unified Directory also require a JVM? OK, I guess Raspberry Pi or ARM wasn’t in focus when Oracle defined the certified platforms of Unified Directory. But hey, I don’t want to set up a production environment, I just need a small project for a long train ride…

The aim is to setup an Raspberry Pi in OTG Mode, install Java and Oracle Unified Directory and configure a small Directory Server, available whenever you need an OUD instance :-). First of all, yes, it works. But before we begin, a few things we need

  • Raspberry Pi Zero I do use a Zero 1.3 without WiFi.
  • USB OTG host cable Dedicated cable supporting USB On-The-Go (OTG). Regular USB cables usually do not support OTG. See Wikipedia On-The-Go (OTG).
  • Raspbian-Image I do recommend the latest Raspbian Stretch Lite. See Raspbian.
  • Oracle JDK 8 for ARM I do use Oracle JDK 8 Update 144 for ARM 32Bit VFP HardFP MOS Patch 26512975. Other Java version are available on MOS Note 1439822.1.
  • Oracle Unified Directory 12.2.1.3 Available through Oracle Technology Network, Oracle Software Delivery Cloud or as My Oracle Support patch 26270957. See also OUD 12.2.1.3 documentation or MOS Note 2300623.1.
  • Temorary System to install OUD Although unified directory does work on ARM, the OUI installer does not. Due to this OUD first have to be “installed” on a supported system. But later more.
  • Environment Scripts for OUD This is optional but quite handy when working on OUD environments. See blog post Environment Scripts for OUD.

In the following chapters I’ll now go through the different steps to setup the OUD “on the go” device. I work primarily on MacOS. Therefore, the individual steps are related to this operating system, but can be easily adapted to other operating systems. Depending on your individual environment, you may skip one or the other step. Shall we get started?

Setup Raspberry Pi

Install Raspian OS

After download the latest release of Raspbian Stretch Lite, we have to create the SD card to setup the OS on raspberry pi. I usually prefer to do this via commandline. Other methods are decribed on www.raspberrypi.org.

PlugIn the SD card and identify the disk via diskutil list. My SD Card is identified disk2. Your output may look different.

soe@gaia:~/ [ic12102] diskutil list
...

/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FDisk_partition_scheme *32.0 GB disk2
1: Windows_NTFS SD Card 32.0 GB disk2s1

Unmount the disk

soe@gaia:~/ [ic12102] diskutil unmountDisk /dev/disk2
Unmount of all volumes on disk2 was successful

Copy the Raspian image to the SD card.

soe@gaia:~/ sudo dd bs=1m \
if=/Data/ISO-Images/2017-09-07-raspbian-stretch-lite.img \
of=/dev/rdisk2 conv=sync

1768+1 records in
1769+0 records out
1854930944 bytes transferred in 73.667297 secs (25179843 bytes/sec)

soe@gaia:~/ [ic12102] diskutil list
...

/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FDisk_partition_scheme *32.0 GB disk2
1: Windows_FAT_32 boot 43.8 MB disk2s1
2: Linux 1.8 GB disk2s2

That’s it, the OS basically has been setup. But before we plug the SD card into the Raspberry Pi we first have to configure the OTG mode.

Configure OTG Mode

Configuring the OTG mode is straight forward, since the latest Rasbian OS does provide all. Eg. modules, kernel, etc. You just have to adjust the boot configuration.

Update cmdline.txt and add the g_ether module. You have to add modules-load=dwc2,g_ether after rootwait and before quiet. If you use vi to edit cmdline.txt your fine. But if you do use an other editor make sure you do not change the file suffix or add extra lines or line breaks to the file cmdline.txt. Everything must be on one line.
Change to the boot directory on the SD Card. Mounted as /Volumes/boot on my Mac.

soe@gaia:~/ [ic12102] cd /Volumes/boot

Update cmdline.txt

soe@gaia:/Volumes/boot/ [ic12102] vi cmdline.txt

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=11eccc69-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait modules-load=dwc2,g_ether quiet init=/usr/lib/raspi-config/init_resize.sh

Update config.txt and add dtoverlay=dwc2 at the end of the file.

soe@gaia:/Volumes/boot/ [ic12102] vi config.txt

As last task, make sure to create an empty file named ssh in the boot folder. This tells Raspian to configure and start the ssh daemon at first system boot. Unmount the SD card and the basic OS setup is finished.

soe@gaia:/Volumes/boot/ [ic12102] touch ssh
soe@gaia:/Volumes/boot/ [ic12102] cd
soe@gaia:~/ [ic12102] diskutil unmountDisk /dev/disk2
Unmount of all volumes on disk2 was successful

Now put the SD card back in your Raspberry Pi Zero and plug in the USB cable. The first system boot will take slightly longer, since the filesystem is getting extended to the maximum size of the SD card. To be on the safe side, wait up to 5 minutes and then try to login via ssh.

soe@gaia:~/ [ic12102] ssh pi@raspberrypi.local

The Raspberry Pi Zero is now ready as headless server in OTG mode.

Setup Environment

General Configuration

This step is not really mandatory, nevertheless I do prefer to adjust a few configuration settings on my pi. First of all upgrade OS to the latest release using apt-get.

pi@raspberrypi:~ $ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

By default the Raspberry Pi hostname is set to raspberrypi. If you do have several Raspberry Pi’s it makes sense to assign names. In my case I do set the hostname to oud2go by changing /etc/hostname and /etc/hosts. In both files you have to replace raspberry with the new name.

pi@raspberrypi:~ $ sudo vi /etc/hostname
pi@raspberrypi:~ $ sudo vi /etc/hosts
pi@raspberrypi:~ $ sudo reboot

sudo: unable to resolve host raspberrypi: Connection timed out
Connection to raspberrypi.local closed by remote host.
Connection to raspberrypi.local closed.

As soon the Pi is back it’s now available by its new name.

soe@gaia:~/ [ic12102] ssh pi@oud2go.local
The authenticity of host 'oud2go.local (fe80::6554:bfdd:7283:3fa9%bridge100)' can't be established.
ECDSA key fingerprint is SHA256:E7WxvWlYDOi0RLNJxEu7rrmA9PH+GlwEJsz0OdHSgCY.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '
oud2go.local,fe80::6554:bfdd:7283:3fa9%bridge100' (ECDSA) to the list of known hosts.
pi@oud2go.local'
s password:
Linux oud2go 4.9.41+ #1023 Tue Aug 8 15:47:12 BST 2017 armv6l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Sep 7 16:22:54 2017 from fe80::acde:48ff:fe00:3364%usb0

SSH is enabled and the default password for the 'pi' user has not been changed.
This is a security risk - please login as the 'pi' user and type 'passwd' to set a new password.
pi@oud2go.local:~ $

It is a good moment to adjust the time zone from UTC to an appropriate for your Raspberry Pi’s. This can either be done using raspi-config or dpkg-reconfigure.

root@oud2go:~# dpkg-reconfigure tzdata

Current default time zone: 'Europe/Zurich'
Local time is now: Mon Oct 30 19:55:21 CET 2017.
Universal Time is now: Mon Oct 30 18:55:21 UTC 2017.

Change the softlinks for localtime will also do the job.

root@oud2go:~# ln -s -f /usr/share/zoneinfo/Europe/Zurich /etc/localtime

Oracle User

In the oracle context it is common practice to create a dedicated user and group. To keep it simple and clear I name it oracle. Indeed I did set up the environment as described in post about the OUD Base environment.

root@oud2go:~# groupadd --gid 1010 oinstall
root@oud2go:~# useradd --create-home --gid oinstall --shell /bin/bash \
--groups oinstall oracle

To install the OUD software, instance and scripts I do use a limited OFA directory structure. See also my Blog Post on OUD Base.

root@oud2go:~# mkdir -p /u00 /u01
root@oud2go:~# mkdir -p /u00/app/oracle
root@oud2go:~# mkdir -p /u00/app/oracle/etc /u00/app/oracle/local
root@oud2go:~# mkdir -p /u00/app/oracle/product /u00/app/oracle/software

root@oud2go:~# chmod a+xr /u00 /u01
root@oud2go:~# chown oracle:oinstall -R /u00 /u01

The newly created user should be allowed to use sudo similar the pi. For this a new sudoers file has to be created.

root@oud2go:~# echo "oracle ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/020_oracle-nopasswd
root@oud2go:~# chmod 440 /etc/sudoers.d/020_oracle-nopasswd

As the last custom configuration I usually distribute the ssh key’s to allow login without password authentication. You only have to include your public key in the file authorized_keys. Lets create the required .ssh user directory for root, pi and the user oracle.

root@oud2go:~# mkdir .ssh
root@oud2go:~# vi .ssh/authorized_keys
root@oud2go:~# chmod 600 .ssh/authorized_keys
root@oud2go:~# chmod 700 .ssh/
root@oud2go:~# cp -r .ssh /home/oracle
root@oud2go:~# cp -r .ssh /home/pi
root@oud2go:~# chown -R pi:pi /home/pi/.ssh
root@oud2go:~# chown -R oracle:oinstall /home/oracle/.ssh

One more thing. Until now all user still have some default password. It’s more than appropriate to change the passwords for the user root, pi and oracle.

root@oud2go:~# passwd root
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
root@oud2go:~# passwd pi
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
root@oud2go:~# passwd oracle
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

Install OUD Base

Install the OUD Base environment scripts according to blog post OUD Base. First we have to get the install scripts using curl.

oracle@oud2go:~ $ cd /u00/app/oracle
oracle@oud2go:/u00/app/oracle $ curl --cookie-jar /tmp/cookie-jar.txt \
--location-trusted "https://github.com/oehrlis/oudbase/raw/master/build/oudbase_install.sh" \
-o oudbase_install.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 147 100 147 0 0 141 0 0:00:01 0:00:01 --:--:-- 141
100 25556 100 25556 0 0 16704 0 0:00:01 0:00:01 --:--:-- 60273
oracle@oud2go:/u00/app/oracle $ chmod 755 oudbase_install.sh

The installation is straight forward. Just run oudbase_install.sh and specify the ORACLE_BASE directory. More options are available via oudbase_install.sh -h.

oracle@oud2go:/u00/app/oracle/ [oud_pi] ./oudbase_install.sh -v -b /u00/app/oracle
2017-11-13_21:13:23 START: Start of oudbase_install.sh (Version 0.1) with -v -b /u00/app/oracle
2017-11-13_21:13:23 INFO : processing commandline parameter
2017-11-13_21:13:23 Using the following variable for installation
2017-11-13_21:13:23 ORACLE_BASE = /u00/app/oracle
2017-11-13_21:13:23 OUD_BASE = /u00/app/oracle
2017-11-13_21:13:23 OUD_DATA = /u00/app/oracle
2017-11-13_21:13:23 ORACLE_INSTANCE_BASE = /u00/app/oracle/instances
2017-11-13_21:13:23 ORACLE_HOME_BASE = /u00/app/oracle/middleware
2017-11-13_21:13:23 OUD_BACKUP_BASE = /u00/app/oracle/backup
2017-11-13_21:13:23 SCRIPT_FQN = /u00/app/oracle/oudbase_install.sh
2017-11-13_21:13:23 Installing OUD Environment
2017-11-13_21:13:23 Create required directories in ORACLE_BASE=/u00/app/oracle
2017-11-13_21:13:23 Create Directory /u00/app/oracle/local/log
2017-11-13_21:13:23 Create Directory /u00/app/oracle/local/etc
2017-11-13_21:13:23 Create Directory /u00/app/oracle/local
2017-11-13_21:13:23 Create Directory /u00/app/oracle/backup
2017-11-13_21:13:23 Create Directory /u00/app/oracle/instances
2017-11-13_21:13:23 Extracting file into /u00/app/oracle/local
bin/
bin/oud_backup.sh
bin/oud_export.sh
bin/oud_status.sh
bin/oudenv.sh
config/
certificates/
doc/
doc/README.md
etc/
etc/oud._DEFAULT_.conf
etc/oudenv.conf
etc/oudtab
lib/
log/
templates/
templates/.bash_profile
templates/cron.d/
templates/etc/
templates/ldif/
templates/logrotate.d/
templates/logrotate.d/oud
templates/ldif/oud_pi_init.ldif
templates/etc/install.rsp
templates/etc/oraInst.loc
templates/etc/oud_instance.service
templates/etc/wls_oudsm.service
templates/cron.d/oud
2017-11-13_21:13:23 Store customization for OUD_DATA (/u00/app/oracle)
2017-11-13_21:13:23 Store customization for OUD_BASE (/u00/app/oracle)
2017-11-13_21:13:23 Store customization for ORACLE_BASE (/u00/app/oracle)
2017-11-13_21:13:23 Please manual adjust your .bash_profile to load / source
2017-11-13_21:13:23 your OUD Environment
2017-11-13_21:13:23 END : of oudbase_install.sh

To start using OUD Base you have to update your .profile file with the following lines.

# Check OUD_BASE and load if necessary
if [ "${OUD_BASE}" = "" ]
then
if [ -f "${HOME}/.OUD_BASE" ]
then
. "${HOME}/.OUD_BASE"
else
echo "ERROR: Could not load ${HOME}/.OUD_BASE"
fi
fi

# define an oudenv alias
alias oud=". $(find $OUD_BASE -name oudenv.sh)"

# source oud environment
. $(find $OUD_BASE -name oudenv.sh)

Install Oracle Software

Install Java

Since a while, Oracle does also provide Java for Raspberry Pi respectively ARM. See Oracle Java on Raspberry Pi. For OUD it’s recommend to use Oracle Java 8 rather than OpenJDK. You can download either download OracleJDK on Java SE Development Kit 8 Downloads or via My Oracle Support. I do prefer the download via My Oracle Support, since this method allows the use of wget or curl.

Create a .netrc file for curl.

oracle@oud2go:~/ [oud_pi] cd /u00/app/oracle/software
oracle@oud2go:/u00/app/oracle/software/ [oud_pi]
oracle@oud2go:/u00/app/oracle/software/ [oud_pi] echo "machine login.oracle.com login password " >.netrc

Download JDK from My Oracle Support:

oracle@oud2go:/u00/app/oracle/software/ [oud_pi] export JAVA_URL="https://updates.oracle.com/Orion/Services/download/p26512975_180144_Linux_VFP.zip?aru=21442384&patch_file=p26512975_180144_Linux_VFP.zip"
oracle@oud2go:/u00/app/oracle/software/ [oud_pi] export JAVA_PKG="p26512975_180144_Linux_VFP.zip"
oracle@oud2go:/u00/app/oracle/software/ [oud_pi] curl --netrc-file .netrc --cookie-jar cookie-jar.txt --location-trusted $JAVA_URL -o $JAVA_PKG
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
100 1959 0 1959 0 0 820 0 --:--:-- 0:00:02 --:--:-- 1839
0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0
100 77.6M 100 77.6M 0 0 3262k 0 0:00:24 0:00:24 --:--:-- 3578k

Install the JDK as root:

oracle@oud2go:/u00/app/oracle/software/ [oud_pi] sudo su -
root@oud2go:~# unzip -p /u00/app/oracle/software/$JAVA_PKG *tar* |tar zvx -C /usr/java

# set the JAVA alternatives directories and links
root@oud2go:~# export JAVA_DIR=$(ls -1 -d /usr/java/*)
root@oud2go:~# ln -s $JAVA_DIR /usr/java/latest
root@oud2go:~# ln -s $JAVA_DIR /usr/java/default

root@oud2go:~# update-alternatives --install /usr/bin/java java $JAVA_DIR/bin/java 20000
update-alternatives: using /usr/java/jdk1.8.0_144/bin/java to provide /usr/bin/java (java) in auto mode

root@oud2go:~# update-alternatives --install /usr/bin/javac javac $JAVA_DIR/bin/javac 20000
update-alternatives: using /usr/java/jdk1.8.0_144/bin/javac to provide /usr/bin/javac (javac) in auto mode

root@oud2go:~# update-alternatives --install /usr/bin/jar jar $JAVA_DIR/bin/jar 20000
update-alternatives: using /usr/java/jdk1.8.0_144/bin/jar to provide /usr/bin/jar (jar) in auto mode

root@oud2go:~# which java
/usr/bin/java

root@oud2go:~# java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) Client VM (build 25.144-b01, mixed mode)

Install OUD

In principle, one should be able to install OUD directly on the Respberry Pi. OUD is unpacked and installed directly with java. But the Oracle Universal Installer or at least a small part of the installation does not work on the ARM platform. Due to this you have to install OUD on an other OS and move the installation directory onto your Raspberry Pi. In my case I do use my MacBook Pro to temporarily install OUD. Alternatively you may also copy the OUD installation from my OUD docker image. But that’s an other story. I’ll post on this topic in a couple of days.

Prepare the download installation path, variables .netca file.

soe@gaia:~/ [ic12102] export DOWNLOAD=/tmp/download
soe@gaia:~/ [ic12102] mkdir -p $DOWNLOAD
soe@gaia:~/ [ic12102] chmod 777 $DOWNLOAD

soe@gaia:~/ [ic12102] export FMW_OUD_URL="https://updates.oracle.com/Orion/Services/download/p26270957_122130_Generic.zip?aru=21504981&patch_file=p26270957_122130_Generic.zip"
soe@gaia:~/ [ic12102] export FMW_OUD_PKG="p26270957_122130_Generic.zip"
soe@gaia:~/ [ic12102] export FMW_OUD_JAR=fmw_12.2.1.3.0_oud.jar

soe@gaia:~/ [ic12102] echo "machine login.oracle.com login password " >/tmp/download/.netrc

soe@gaia:~/ [ic12102] curl --netrc-file /tmp/download/.netrc --cookie-jar \
 /tmp/download/cookie-jar.txt  --location-trusted $FMW_OUD_URL \
 -o $DOWNLOAD/$FMW_OUD_PKG

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
100 1927 0 1927 0 0 963 0 --:--:-- 0:00:02 --:--:-- 1715
0 0 0 0 0 0 0 0 --:--:-- 0:00:05 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:00:06 --:--:-- 0
100 404M 100 404M 0 0 1847k 0 0:03:44 0:03:44 --:--:-- 1689k

Create a bunch of local Directories in ORACLE_BASE

soe@gaia:~/ [ic12102] export ORACLE_BASE=/u00/app/oracle
soe@gaia:~/ [ic12102] mkdir -p $ORACLE_BASE/etc $ORACLE_BASE/product

To install OUD in silent mode, we need a response file. For OUD this is a simple and straight forward text file.

soe@gaia:~/ [ic12102] echo "[ENGINE]" > $ORACLE_BASE/etc/install.rsp
soe@gaia:~/ [ic12102] echo "Response File Version=1.0.0.0.0" >> $ORACLE_BASE/etc/install.rsp
soe@gaia:~/ [ic12102] echo "[GENERIC]" >> $ORACLE_BASE/etc/install.rsp
soe@gaia:~/ [ic12102] echo "DECLINE_SECURITY_UPDATES=true" >> $ORACLE_BASE/etc/install.rsp
soe@gaia:~/ [ic12102] echo "SECURITY_UPDATES_VIA_MYORACLESUPPORT=false" >> $ORACLE_BASE/etc/install.rsp

The installer does also require a OraInventory Location file:

soe@gaia:~/ [ic12102] echo "inventory_loc=$ORACLE_BASE/oraInventory" > $ORACLE_BASE/etc/oraInst.loc
soe@gaia:~/ [ic12102] echo "inst_group=oinstall" >> $ORACLE_BASE/etc/oraInst.loc

The JAR and the response file will then be used to install OUD in silent mode

soe@gaia:/tmp/download/ [ic12102] java -jar $DOWNLOAD/$FMW_OUD_JAR -silent \
 -responseFile $ORACLE_BASE/etc/install.rsp \
 -invPtrLoc $ORACLE_BASE/etc/oraInst.loc \
 -ignoreSysPrereqs -force \
 -novalidation ORACLE_HOME=$ORACLE_BASE/product/fmw12.2.1.3.0 \
 INSTALL_TYPE="Standalone Oracle Unified Directory Server (Managed independently of WebLogic server)"

Launcher log file is /private/var/folders/80/xtg0v0r16sl6z5sjmxhkvr540000gn/T/OraInstall2017-10-30_08-47-41PM/launcher2017-10-30_08-47-41PM.log.
Extracting the installer . . . . . Done
Checking if CPU speed is above 300 MHz. Actual 2969.6 MHz Passed
Checking swap space: must be greater than 512 MB. Actual 257166 MB Passed
Checking if this platform requires a 64-bit JVM. Actual 64 Passed
Checking temp space: must be greater than 300 MB. Actual 257166 MB Passed
Preparing to launch the Oracle Universal Installer from /private/var/folders/80/xtg0v0r16sl6z5sjmxhkvr540000gn/T/OraInstall2017-10-30_08-47-41PM
Log: /private/var/folders/80/xtg0v0r16sl6z5sjmxhkvr540000gn/T/OraInstall2017-10-30_08-47-41PM/install2017-10-30_08-47-41PM.log
Setting ORACLE_HOME to /u00/app/oracle/product/fmw12.2.1.3.0
Setting INSTALL_TYPE to Standalone Oracle Unified Directory Server (Managed independently of WebLogic server)
Copyright (c) 2010, 2017, Oracle and/or its affiliates. All rights reserved.
Reading response file..
Skipping Software Updates
Validations are disabled for this session.
Verifying data
Copying Files
Percent Complete : 10
Percent Complete : 20
Percent Complete : 30
Percent Complete : 40
Percent Complete : 50
Percent Complete : 60
Percent Complete : 70
Percent Complete : 80
Percent Complete : 90
Percent Complete : 100

The installation of Oracle Unified Directory 12.2.1.3.0 completed successfully.
Logs successfully copied to /u00/app/oracle/oraInventory/logs.

Copy the OUD binaries to your Raspberry Pi.

soe@gaia:~/ [ic12102] scp -r /u00/app/oracle/product/fmw12.2.1.3.0 oracle@oud2go.local:/u00/app/oracle/product

Clean up the temporary installation on the MacBook Pro:

soe@gaia:~/ [ic12102] rm -rf $ORACLE_BASE/etc/install.rsp
soe@gaia:~/ [ic12102] rm -rf $ORACLE_BASE/etc/oraInst.loc
soe@gaia:~/ [ic12102] rm -rf $ORACLE_BASE/oraInventory
soe@gaia:~/ [ic12102] rm -rf $ORACLE_BASE/product/fmw12.2.1.3.0
soe@gaia:~/ [ic12102] rm -rf /tmp/download

Thats it. You not have your OUD software on your pi. Now lets create an OUD instance.

Setup OUD Directory Server

Depending on your need, you may create an OUD directory server or an OUD proxy server. The setup scripts can either be execute interactive via GUI or command line or as on command. On my Raspberry Pi I do setup a directory server with just one command.

Create a password file for the OUD instance oud_pi_pwd.txt

oracle@oud2go:/u00/app/oracle/ [oud_pi] echo "manager" >/u00/app/oracle/local/etc/oud_pi_pwd.txt

Create the OUD directory server for Base DN dc=postgasse,dc=org with a bunch of dummy entries using oud-setup.

oracle@oud2go:/u00/app/oracle/ [oud_pi] $ORACLE_HOME/oud/oud-setup \
--cli \
--instancePath /u00/app/oracle/instances/oud_pi/OUD \
--adminConnectorPort 4444 \
--rootUserDN cn=Directory\ Manager \
--rootUserPasswordFile /u00/app/oracle/local/etc/oud_pi_pwd.txt \
--ldapPort 1389 \
--baseDN dc=postgasse,dc=org \
--sampleData 20 \
--serverTuning jvm-default \
--offlineToolsTuning jvm-default \
--no-prompt \
--noPropertiesFile

Oracle Unified Directory 12.2.1.3.0
Please wait while the setup program initializes...

Creating instance directory /u00/app/oracle/instances/oud_pi/OUD ..... Done.
See /u00/app/oracle/instances/oud_pi/OUD/logs/oud-setup for a detailed log of
this operation.

Configuring Directory Server ......... Done.
Importing Automatically-Generated Data (20 Entries) ....................................... Done.
Starting Directory Server ........................................ Done.

To see basic server configuration status and configuration you can launch
/u00/app/oracle/instances/oud_pi/OUD/bin/status

That’s it, we now have an empty directory server with 20 sample records 🙂 Since the Raspberry Pi only has limited resources, I’ve just configure the LDAP port. For a simple test and engineering system LDAPS is not really required. Specially because we do not setup any EUS integration.

Conclusion

It works… but the directory server is far away from a high performance setup. Nevertheless it’s a nice and handy setup for simple engineering work and simple demos. Striving for a bit more performance? You may also setup OUD in a docker container. I’ll provide more information on this topic in a couple of day’s. If you can’t wait, take a look at my Docker OUD Repository on GitHub now (docker-oud or docker-oudsm).

Files and References

Below you find a few references related to Raspberry Pi, USB OTG or Oracle Unified Directory:

Software related to this project:

  • Raspbian Stretch Lite latest
  • Oracle JDK 8 Update 144 for ARM 32Bit VFP HardFP MOS Patch 26512975
  • Oracle Unified Directory 12.2.1.3.0 on Oracle Technology Network
  • Oracle Software Delivery Cloud OSDC
  • Oracle Unified Directory FMW 12.2.1.3.0 MOS Patch 26270957
  • OUD base environment installation script. It’s a bash script including a TAR.  oudbase_install.sh
  • OUD base environment as TAR archive without installation script.  oudbase_install.tgz

My Oracle Support Notes:

  • Oracle Unified Directory 12c PS3 Released [2300623.1]
  • All Java SE Downloads on MOS [1439822.1]
  • Information Center: Using Oracle Unified Directory (OUD) [1419823.2]

Start OUD Servers on Boot using systemd

Starting Oracle Unified Directory on system boot is essential for production environment. Unfortunately OUD just provides a script to create the init.d script. But newer system in general use systemd initialise and startup. Nevertheless, creating a custom unit file for OUD is simple and straightforward. First, let’s create a regular init.d script with the create-rc-script from oud. The created custom script can be used as template for the systemd unit file.

create-rc-script does allow a couple of parameter to specify the script name, OS user for OUD and the JAVA_HOME. The following example of create-rc-script does show how to create a regular start script for OUD instance oud_ad_proxy.

export OUD_HOME=/u00/app/oracle/instances/oud_ad_proxy
export JAVA_HOME=/u00/app/oracle/product/jdk1.7.0_141

cd $OUD_HOME/OUD/bin
create-rc-script -f oud_ad_proxy.sh -u oracle -j $JAVA_HOME

This does create the following bornshell script for init.d.

#!/bin/sh
#
# Copyright (c) 2012, Oracle and/or its affiliates. All rights reserved.
#
#
# chkconfig: 345 90 30
# description: Oracle Unified Directory startup script
#


# Set the path to the Oracle Unified Directory instance to manage
INSTALL_ROOT="/u00/app/oracle/instances/oud_ad_proxy/OUD"
export INSTALL_ROOT

# Specify the path to the Java installation to use
OPENDS_JAVA_HOME="/u00/app/oracle/product/jdk1.7.0_141"
export OPENDS_JAVA_HOME

# Determine what action should be performed on the server
case "${1}" in
start)
  /bin/su - oracle -- "${INSTALL_ROOT}/bin/start-ds" --quiet
  exit ${?}
  ;;
stop)
  /bin/su - oracle -- "${INSTALL_ROOT}/bin/stop-ds" --quiet
  exit ${?}
  ;;
restart)
  /bin/su - oracle -- "${INSTALL_ROOT}/bin/stop-ds" --restart --quiet
  exit ${?}
  ;;
*)
  echo "Usage:  $0 { start | stop | restart }"
  exit 1
  ;;
esac

The same start / stop commands can now be used in the unit file. So let’s create a new custom unit file in /etc/systemd/system. The unit file is named according the old instance.

sudo vi /etc/systemd/system/oud_ad_proxy.service

Add the following content to the new unit file.

[Unit]
Description=OUD AD Proxy Instance oud_ad_proxy
Wants=network.target
After=network.target

[Service]
Type=forking
User=oracle
Group=osdba
Environment=OPENDS_JAVA_HOME="/u00/app/oracle/product/jdk1.7.0_141"
ExecStart=/u00/app/oracle/instances/oud_ad_proxy/OUD/bin/start-ds --quiet
ExecStop=/u00/app/oracle/instances/oud_ad_proxy/OUD/bin/stop-ds --quiet
ExecReload=/u00/app/oracle/instances/oud_ad_proxy/OUD/bin/stop-ds --restart --quiet
StandardOutput=syslog

[Install]
WantedBy=multi-user.target

As soon as we have the new unit file we have to enable the service.

sudo systemctl enable oud_ad_proxy.service

Start the OUD instance using systemctl.

sudo systemctl start oud_ad_proxy.service

Stop the OUD instance using systemctl.

sudo systemctl stop oud_ad_proxy.service

Display the status of the OUD service.

sudo systemctl status oud_ad_proxy.service

 oud_ad_proxy.service - OUD AD Proxy Instance oud_ad_proxy
   Loaded: loaded (/etc/systemd/system/oud_ad_proxy.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-05-16 22:41:09 CEST; 28s ago
  Process: 18300 ExecStop=/u00/app/oracle/instances/oud_ad_proxy/OUD/bin/stop-ds --quiet (code=exited, status=0/SUCCESS)
  Process: 18397 ExecStart=/u00/app/oracle/instances/oud_ad_proxy/OUD/bin/start-ds --quiet (code=exited, status=0/SUCCESS)
 Main PID: 18477 (java)
   CGroup: /system.slice/oud_ad_proxy.service
           └─18477 /u00/app/oracle/product/jdk1.7.0_141/jre/bin/java -server -Dorg.opends.server.scriptName=start-ds org.opends.server.core.DirectoryServer --configClass org.opends.server.extensions.ConfigFileHandler -...

May 16 22:41:01 euterpe systemd[1]: Starting OUD AD Proxy Instance oud_ad_proxy...
May 16 22:41:09 euterpe systemd[1]: Started OUD AD Proxy Instance oud_ad_proxy.

Some references and links to MOS Notes:

Update: AVDF installation fails on HP server with Smart Array

A couple of days ago I’ve wrote about some problems when installing Oracle Audit Vault and Database Firewall 12.1.2 on HP server with Smart Array Disk Controller. The problem is still not resolved, but in the meantime Oracle has open a Bug and added some Metalink Notes related to this issue.

  • AVDF 12.1.1 Installation Fails On HP server with Smart Array Disk Controller [1587742.1]
  • Unable To Install AVDF Server With HP Smart Array [1680134.1]
  • AVDF installation ISO [1680961.1]
  • Bug 18940816 AVDF SERVER FAILS TO INSTALL ON HP DL380 GEN8 WITH CCISS!C0D0 ERROR

The contents of MOS note 1680134.1 and 1680961.1 are certainly known to the regular readers of OraDBA. The workaround and procedure are the same as I’ve posted a couple of days ago. Oracle created MOS notes based on my blog post AVDF installation fails on HP server with Smart Array Disk Controller and AVDF installation ISO. In this case, my posts are somehow useful. 🙂 The Bug mentioned above is unfortunately not publicly available. I’ll provide more information as soon as it is available.

AVDF installation fails on HP server with Smart Array Disk Controller

I’ve successfully set up a couple of AVDF installation on different VM Server as well on HP Blade or Rack servers. On the VM server I never had any problems. For the installation of AVDF 12.1.1.x on HP servers BL465c Gen8 or DL380p Gen8, there were always warnings during partitioning of the disks. So far it was never an issue to just continue the installation. With AVDF 12.1.2 this has changed. On some HP servers with smart array disk controller the installation fails because of problems with the drivers respectively device names.

Earlier installation of AVDF like 12.1.1.3.0 simply complained about not enough space.

AVDF_12.1.1.3.0_setup02

OK, 0GB is a bit less for setting up an AVDF Server :-), nevertheless ignoring the error still worked. AVDF 12.1.1.2 as well AVDF 12.1.1.3 could be successfully setup using the cciss Driver for HP Smart Array. As of AVDF 12.1.2 the error is not that friendly any more.

AVDF_12.1.2.0.0_setup02

The title of the error “Error Parsing Kickstart Config” indicates that there is an issue at an early stage of the system setup. It is worth having a deeper look into the kickstart configuration file. The kickstart file can be found in the initrd.img image on the AVDF installation ISO. See AVDF installation ISO for how to extract the kickstart file.

In the kickstart file we can see at line 62, that a pre-script is executed to create the partition commands. This pre-script is a python script which does create a temporary file (/tmp/partition-include) with the partition commands based on the available disks. The partition command itself is then included at line 36.

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
########## Partition the disk ##############
%include /tmp/partition-include

# Create logical volume group - this is where all volumes will reside
volgroup vg_root pv.01
# Now create the volumes, a.k.a logical partitions. The data partions (/var/lib/oracle) is grown
# up to the specified size. The rest of the FS is left unallocated.
#
# You must make changes to ruby_lib/dbfw/dbfw_fstab.rb if you change the FS specification.
#
logvol swap --fstype swap --vgname=vg_root --size=4096 --name=lv_swap
logvol / --fstype ext3 --fsoptions="noatime" --vgname=vg_root --size=7000 --name=lv_root
logvol /images --fstype ext3 --fsoptions="noexec,nodev,nosuid,noatime" --vgname=vg_root --size=15000 --name=lv_images
logvol /usr/local/dbfw --fstype ext3 --fsoptions="noatime" --vgname=vg_root --size=1000 --name=lv_local_dbfw
logvol /usr/local/dbfw/tmp --fstype ext3 --fsoptions="noexec,nodev,nosuid,noatime" --vgname=vg_root --size=9000 --name=lv_local_dbfw_tmp
logvol /home --fstype ext3 --fsoptions="noexec,nodev,nosuid,noatime" --vgname=vg_root --size=1000 --name=lv_home
logvol /tmp --fstype ext3 --fsoptions="nodev,nosuid,noatime" --vgname=vg_root --size=2000 --name=lv_tmp
logvol /var/log --fstype ext3 --fsoptions="noexec,nodev,nosuid,noatime" --vgname=vg_root --size=6000 --name=lv_var_log
logvol /var/tmp --fstype ext3 --fsoptions="noexec,nodev,nosuid,noatime" --vgname=vg_root --size=6000 --name=lv_var_tmp
logvol /var/www --fstype ext3 --fsoptions="nodev,nosuid,noatime" --vgname=vg_root --size=1000 --name=lv_var_www
logvol /var/www/tmp --fstype ext3 --fsoptions="nodev,nosuid,noatime" --vgname=vg_root --size=1000 --name=lv_var_www_tmp
logvol /var/lib/oracle --fstype ext3 --fsoptions="noatime" --vgname=vg_root --size=20000 --name=lv_oracle
logvol /var/dbfw --fstype ext3 --fsoptions="noatime" --vgname=vg_root --size=10000 --name=lv_var_dbfw

# Tasks performed before installation
%pre

python /kickstart/partitions.py 2> /tmp/partitions_error
if [ $? -ne 0 ]; then
DISKERROR=$(/bin/cat /tmp/partitions_error)
fi

Having a look into the file /tmp/partition-include reveals the wrong partition command which leads to the error mentioned earlier. As you can see below the disks are specified with –ondisk=cciss!c0d0 rather than –ondisk=cciss/c0d0. The python script which builds the partition commands, has issues with the device names. Actually, for an HP smart array disk, the corresponding driver should be loaded so that the devices are visible as sd*. The root cause could be the missing driver or an error in the python script. I’ve opened a service request with oracle Support for further analysis.
AVDF_12.1.2.0.0_setup04

Workarounds

For the moment I just see the following two workarounds.

  • First install and configure AVDF 12.1.1.3 and perform an upgrade to AVDF 12.1.2.
  • Install AVDF 12.1.2 with an alternative kickstart file respectively partition commands

The first workaround is straightforward. It just takes a bit more time. For the second workaround you may create a new AVDF ISO image, but this is way to complex. It is much simpler to manually specify the boot options and provide an alternative kickstart file on an internal web server. The kickstart file is the same as for the regular AVDF 12.1.2 installation, it just has a fixed partition section. For that I have taken the partitioning commands from the file /tmp/partition-include and removed the –ondisk parameter. I’ll provide my kickstart file as an example for download. But do not use it directly the partitioning section must be adapted to your environment.
Action plan for the workaround:

  1. Create an alternative kickstart file with correct partition commands for your environment
  2. Put the kickstart file on a Webserver which is accessible by the AVDF Server
  3. Boot from AVDF 12.1.2 ISO image with custom boot parameter

My custom boot option did look like the following command. The IP address is the address of my web server.

vmlinuz noipv6 initrd=initrd.img ramdisk_size=8192 ks=http://192.128.1.40/avdf.cfg

Conclusion

This problem is quite annoying, especially if you have already done the installation on another physical or virtual servers several times. The workaround is basically simple. With a bit enhanced Linux knowledge and a web server, one has quickly created an installation with an alternative kickstart file. Nevertheless I highly recommend to open a service request with Oracle when you have similar issues with your hardware during the setup of a productive AVDF 12.1.2 environment.

References

Further information on this topic.

AVDF missing boot partition

While working on the problem with missing RAM on the AVDF test system (see ) I realized, that the linux boot partition is not available by default.

[root@melete2 log]# ls -al /boot
total 16
drwxr-xr-x  2 root root 4096 Jan 11  2013 .
drwxr-xr-x 24 root root 4096 Jul 11 20:19 ..

[root@melete2 log]# df -kh /boot
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_root-lv_root
                      6.6G  2.2G  4.1G  35% /

Initially I was a bit confused since it contains stuff like grub configuration, inited.img, kernel etc. All stuff that are needed for system boot. Ok, I have not thought about that for the bootloader, the file system does not have to be mounted. From the security point of view it’s even better to not have it mounted. If not mounted nobody can accidentally change something. 😉 Oracle has defined noauto for the boot partition. Therefore the device is not mounted automatically during system boot.

[root@melete2 log]# cat /etc/fstab|grep boot
LABEL=/boot                    /boot                    ext3   noatime,noauto,nodev,nosuid                  1 2

If you need to change the grub configuration just mount the boot partition manually.

[root@melete2 log]# mount /boot

[root@melete2 log]# vi /boot/grub/grub.conf

[root@melete2 ~]# umount /boot

AVDF Linux kernel could not recognize whole RAM

After initial setup of an Audit Vault and Database Firewall engineering system, I’ve started to add several audit vault agents and secure targets. In the beginning it went quite smoothly. But after a certain number of secured targets, there were continuously ORA-04031 errors. Most of the errors were related to large pool and PX Msg buffers issues. The analysis of the trace files has shown interesting stuff. 😉 But more on that in a later blog post. The real problem is the available memory.

Symptoms

The Audit Vault and Database Firewall engineering system is running on a HP ProLiant BL465c Gen 8. It comes with 32GB Memory. Should actually be sufficient for a system engineering. It turned out that the 32GB are not recognized by operating system. As you can see below the system has just 3GB memory in total.

[root@melete2 ~]# free
                     total    used   free shared buffers  cached
Mem:               3048108 2385888 662220      0   10720 1525036
-/+ buffers/cache:  850132 2197976
Swap:              4194296  453564 3740732

Reviewing dmesg shows that we lose 29 GB of memory.

Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.32-300.39.5.el5uek (mockbuild@ca-build56.us.oracle.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)) #1 SMP Wed Mar 13 11:26:53 PDT 2013
Command line: ro root=/dev/vg_root/lv_root console=tty9 udevtimeout=10
KERNEL supported cpus:
  Intel GenuineIntel
  AMD AuthenticAMD
  Centaur CentaurHauls
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
 BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000bddde000 (usable)
 BIOS-e820: 00000000bddde000 - 00000000bde0e000 (ACPI data)
 BIOS-e820: 00000000bde0e000 - 00000000d0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fee10000 (reserved)
 BIOS-e820: 00000000ff800000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 000000083efff000 (usable)
DMI 2.7 present.
last_pfn = 0x83efff max_arch_pfn = 0x400000000
MTRR default type: uncachable
MTRR fixed ranges enabled:
  00000-9FFFF write-back
  A0000-BFFFF uncachable
  C0000-FFFFF write-back
MTRR variable ranges enabled:
  0 base 000000000000 mask FFFF80000000 write-back
  1 base 000080000000 mask FFFFC0000000 write-back
  2 disabled
  3 disabled
  4 disabled
  5 disabled
  6 disabled
  7 disabled
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
e820 update range: 00000000c0000000 - 000000083efff000 (usable) ==> (reserved)
WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 29679MB of RAM.
------------[ cut here ]------------

Cause

According to an Oracle Metalink Note 1448147.1 this problem is related to a BIOS issue.

Solutions and Workaround

The solution described in Oracle Metalink Note 1448147.1 is to upgrade the BIOS or disable MTRR in kernel. Since BIOS upgrade is not an option for this environment I’ll try to workaround by disable MTRR.

Disable MTRR

Changing the grub.conf is basically quite easy if you find the boot files. When I first try it, I’d realized that there is no grub configuration available. It seems that Oracle decided to not mount /boot at startup. So it is mandatory to first mount the boot partition. Afterward you just can add disable_mtrr_trim as additional kernel option.

[root@melete2 ~]# mount /boot

[root@melete2 ~]# df -kh /boot
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             145M   26M  112M  19% /boot

[root@melete2 ~]# vi /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/vg_root/lv_root
#          initrd /initrd-version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Audit Vault Server 12.1.1.0.0
        root (hd0,0)
        kernel /vmlinuz-2.6.32-300.39.5.el5uek ro root=/dev/vg_root/lv_root console=tty9
udevtimeout=10 disable_mtrr_trim
        initrd /initrd-2.6.32-300.39.5.el5uek.img
title Audit Vault Server 12.1.1.0.0
        root (hd0,0)
        kernel /vmlinuz-2.6.32-300.38.1.el5uek ro root=/dev/vg_root/lv_root console=tty9
udevtimeout=10 disable_mtrr_trim
        initrd /initrd-2.6.32-300.38.1.el5uek.img

[root@melete2 ~]# reboot

Broadcast message from root (pts/0) (Thu Jul 11 20:17:56 2013):

The system is going down for reboot NOW!
[root@melete2 ~]# Connection to melete2 closed by remote host.
Connection to melete2 closed.

After reboot we now have 32GB memory available.

[root@melete2 ~]# free
                      total     used     free shared buffers  cached
Mem:               33024372  3930724 29093648      0   17868 2640744
-/+ buffers/cache:  1272112 31752260
Swap:              14680056        0 14680056

Unfortunately, the configuration of the AVDF appliance is not automatically updated to use the extra memory. We have to do some manual changes.

Update Kernel Parameters

The kernel setting have to be changed to allow a bigger SGA. See Metalink Note 1529433.1 for more detailed information on how calculate and set the kernel parameters. For the engineering system we will define a SGA with 20GB therefor we set the shmmax and shmall as follows:

[root@melete2 ~]# vi /etc/sysctl.conf

kernel.shmmax=23622320128
kernel.shmall=5368709120
...
[root@melete2 ~]# sysctl -p

Increase SWAP

With 32GB memory, it is also advisable to enlarge the swap space. I’ve discussed this already in the blog post Resize swap space on linux. Since the AVDF appliance does use logical volumes it’s even a bit easier.

[root@melete2 ~]# swapoff -v /dev/vg_root/lv_swap

[root@melete2 ~]# lvresize /dev/vg_root/lv_swap -L +8G

[root@melete2 ~]# mkswap /dev/vg_root/lv_swap

[root@melete2 ~]# swapon -v /dev/vg_root/lv_swap

Increase SGA

Finally we can increase the SGA.

SQL> ALTER system SET sga_max_size=20G scope=spfile;
System altered.

SQL> ALTER system SET sga_target=20G scope=spfile;
System altered.

SQL> startup force

Conclusion

Although AVDF is an appliance, it is mandatory to examine the system after installation. Eg. are there errors in the log files in /var/log, memory, storage etc. available. The solution described here makes it possible to use all the memory. Nevertheless, the appliance has been adjusted to an extent where is necessary to consider whether the support is still archive. If you run into a similar issue on your production AVDF setup I would recommend opening an Oracle SR. Looking forward to the next AVDF patchset. I hope this system stays patchable.

References

Some links related to this post.

  • Linux kernel could not recognize whole RAM [1448147.1]
  • Upon startup of Linux database get ORA-27102: out of memory Linux-X86_64 Error: 28: No space left on device[301830.1]
  • Requirements for Installing Oracle Database 12.1 on RHEL5 or OL5 64-bit (x86-64) [1529433.1]
  • Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 5 on AMD64/EM64T [880989.1]
  • Master Note of Linux OS Requirements for Database Server [851598.1]