Wednesday, May 25, 2016

CHEF - Sequel - RuntimeError: unknown privilege type SYSDBA

I explained in my previous post how to install Sequel Ruby gem to interact with Oracle databases in CHEF. However, there's little documentation on Sequel's website about connecting to Oracle databases.

I had two issues, (1) I couldn't figure out the connection string pattern for the database and (2) Once I did, I had issues connecting to the database using elevated privileges (e.g. connecting as SYSDBA).

Issue 1
The correct format for Oracle database is, you must quote the whole thing.
DB = Sequel.connect('oracle://user:password@hostname:port/service_name')
e.g.
DB = Sequel.connect('oracle://scott:tiger@oel1.mydomain.com:1523/orcl.mydomain.com')

Issue 2
If you want to connect with SYS user, the above syntax needs to be changed to

DB = Sequel.connect('oracle://sys:mypassword@oel1.mydomain.com:1523/orcl.mydomain.com', :privilege => :SYSDBA)


CHEF - Using an Oracle Database

There are a lot of cookbooks that let you run queries/operations against open source databases (MySQL/PostgreSQL etc). I wanted to use something to deal with Oracle database. I found an old blog on Oracle's website that shows few examples of this integration using oci8. But oci8 works specifically for Oracle and while it solved my problem, I couldn't use it for other databases. A friend recommended me Sequel which I found really useful. But using Oracle with Sequel still requires you to install and configure oci8. Here's the process:

Example below is for configuring on Mac OS. See footnotes for Linux.

Step 1
Install Ruby gem oci8. To install oci8, you must have at a minimum Oracle Instant client and SDK. Download them from Oracle's website.

Step 2
Copy the zip files to the homebrew library folder. If you are not using homebrew, you should!
ls /Library/Caches/Homebrew
instantclient-sdk-macos.x64-11.2.0.4.0.zip
instantclient-basic-macos.x64-11.2.0.4.0.zip

Step 3
Install the packages using homebrew.

brew install InstantClientTap/instantclient/instantclient-basic
==> Tapping instantclienttap/instantclient
Cloning into '/usr/local/Library/Taps/instantclienttap/homebrew-instantclient'...
Tapped 4 formulae (43 files, 37.9K)
==> Installing instantclient-basic from instantclienttap/instantclient
==> Downloading http://download.oracle.com/otn/mac/instantclient/11204/instantclient-basic-macos.x64-11.2.0.4.0.zip
Already downloaded: /Library/Caches/Homebrew/instantclient-basic-macos.x64-11.2.0.4.0.zip
==> /usr/bin/install_name_tool -id /usr/local/lib/libclntsh.dylib.11.1 /usr/local/Cellar/instantclient-basic/11.2.0.4.0/lib/libclntsh.dylib
.... [output clipped]
🍺  /usr/local/Cellar/instantclient-basic/11.2.0.4.0: 8 files, 181.0M, built in 3 seconds
/Library/Caches/Homebrew >

brew install InstantClientTap/instantclient/instantclient-sdk
==> Installing instantclient-sdk from instantclienttap/instantclient
==> Downloading http://download.oracle.com/otn/mac/instantclient/11204/instantclient-sdk-macos.x64-11.2.0.4.0.zip
Already downloaded: /Library/Caches/Homebrew/instantclient-sdk-macos.x64-11.2.0.4.0.zip
🍺  /usr/local/Cellar/instantclient-sdk/11.2.0.4.0: 40 files, 1.9M, built in 0 seconds
/Library/Caches/Homebrew >

Step 4
Install the oci8 Ruby gem, in this case I want to use it with chef so I am using chef's syntax to install the gem, it will work for just ruby too (without preceding the command with chef)

chef gem install ruby-oci8
Building native extensions.  This could take a while...
Successfully installed ruby-oci8-2.2.2
1 gem installed

Step 5
Install the Sequel Ruby gem.

chef gem install sequel
Fetching: sequel-4.34.0.gem (100%)
Successfully installed sequel-4.34.0
1 gem installed

That's it! You should now be able to use Sequel gem to query Oracle databases.

Important note for Linux:
If you want the CHEF recipes to work on your target systems (say Linux), these packages must be installed on them. The problem is that if you attempt to install these gems using chef_gem package in a recipe, the oci8 gem will fail to execute unless you set the LD_LIBRARY_PATH which at the time of writing I couldn't figure out how to. So as a workaround, I installed the Oracle instant client packages (rpm) manually, set the LD_LIBRARY_PATH to where libclntsh.so file was. In my case it was export LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib and then installed the gems using the CHEF recipe. Make sure you are in the same session or the LD_LIBRARY_PATH is set at the profile level.



Monday, May 23, 2016

CHEF - ArgumentError: could not find a temporary directory

When running chef-client as non root user, you may encounter the following error:

==============================
  Error Syncing Cookbooks:
  =============================
  Unexpected Error:
  -----------------
  ArgumentError: could not find a temporary directory
 
  Platform:
  ---------
  x86_64-linux


In the stacktrace, the error shows up as:
ArgumentError: could not find a temporary directory
/opt/chef/embedded/lib/ruby/2.1.0/tmpdir.rb:34:in `tmpdir'
/opt/chef/embedded/lib/ruby/2.1.0/tmpdir.rb:105:in `tmpdir'
/opt/chef/embedded/lib/ruby/2.1.0/tmpdir.rb:137:in `create'
/opt/chef/embedded/lib/ruby/2.1.0/tempfile.rb:136:in `initialize'
/opt/chef/embedded/lib/ruby/2.1.0/tempfile.rb:320:in `new'
/opt/chef/embedded/lib/ruby/2.1.0/tempfile.rb:320:in `open'


Solution:
This is not a CHEF issue rather a ruby one. Newer versions of ruby do not allow any directory to be used as a temporary directory that has (777) permissions set on it unless setuid of 1 is set on the directory too. As root user do the following:

chmod 1777 /tmp
ls -ld /tmp  drwxrwxrwt. /tmp

notice the "t" in the output.

CHEF - Running chef-client as non-root user

Chef works best when run as root user but there are scenarios when you would want to run a chef-client as non root user. To do that, do the following steps:

Assumption:
It is assumed that you already have bootstrapped the node and/or installed the chef-client.  Also, it assumes that you have access to the client.pem key needed to talk to the CHEF server. For this example, we'll assume the non root user is "oracle".

Step 1:
As root, copy the entire content of /etc/chef folder to "oracle" user's home directory and change permissions.
cp -rf  /etc/chef /home/oracle/.chef
chown -R oracle:oinstall /home/oracle/.chef

Step 2:
Open the /home/oracle/.che/client.rb file and add the following lines:
cache_path "/home/oracle/.chef"
client_key "/home/oracle/.chef/client.pem"
 

If you don't specify cache_path, you will get the error:
Unable to access cache at /var/chef. Switching cache to /home/oracle/.chef
whenever you run the chef-client as oracle user.

If you don't specify the client_key location, chef-client will look at /etc/chef/client.pem file, if the file is not readable (by default it won't), the client will error out with

Private Key Not Found:
----------------------
Your private key could not be loaded. If the key file exists, ensure that it is
readable by chef-client.

Relevant Config Settings:
-------------------------
validation_key "/etc/chef/validation.pem"


Step 3:
Now, run the chef-client with the following arguments:
chef-client -o recipe[] -c /home/oracle/.chef/client.rb



Monday, May 9, 2016

CHEF - Uninstalling CHEF Server

Once installed, it can be tricky to cleanup a CHEF server install. The guys at CHEF have updated their docs (https://docs.chef.io/uninstall.html) with instructions on how to deinstall CHEF server but I found that it does not cleans up the system entirely.

Note: Running these steps will completely wipe out all configurations you have, so proceed with caution.

I followed the following steps to remove CHEF server on my RHEL machine.

1. As mentioned in the CHEF docs, run
chef-server-ctl uninstall
This step will first stop all the processes and then run a cleanup.

2. If you do a grep on the running processes, you'll notice the EPM daemon would still be running. Kill it.
ps -ef|grep chef
/opt/opscode/embedded/service/opscode-chef-mover/erts-6.4/bin/epmd -daemon
kill -9 "process id of the epm daemon"

3. Now remove the CHEF packages. You can use whatever package manager you use, I use yum.
yum remove chef-*

Even if you did not install the CHEF package using yum, this will remove the package.

4. Cleanup all the folders.
rm -rf /opt/opscode*
rm -rf /opt/chef-manage
rm -rf /var/opt/opscode*
rm -rf /etc/chef

5. Restart the host.
init 6


Wednesday, February 3, 2016

Error: XATRANS Views are not installed on this Database

Running RCU 11.1.2.2 for installing OIM schemas in an Oracle database fails with:

Error:
RCU-6092:Component Selection validation failed. Please refer to log at /apps/demo/binaries/tmp/rcu.1921916505.tmp/rcu.log for details. 
RCU-6083:Failed - Check prerequisites requirement for selected component:OIM 
Please refer to RCU log at /apps/demo/binaries/tmp/rcu.1921916505.tmp/rcu.log for details.
Error:  XATRANS Views are not installed on this Database. This is required by the OIM Schema Action: Install view XAVIEWS as SYS user on this Database.
Refer to the Oracle Database Release Documentation for installation details.  

Starting 11gR2, Oracle Identity manager, XATRANS and XAVIEWS are required in the database. 

Fix:
As sys user, run the following two files to install the required views:

sqlplus / as sysdba @$ORACLE_HOME/javavm/install/initxa.sql

SQL*Plus: Release 11.2.0.4.0 Production on Wed Feb 3 11:11:31 2016
Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

PL/SQL procedure successfully completed.

JVMRMACTION
--------------------------------------------------------------------------------
FULL_REMOVAL
PL/SQL procedure successfully completed.
Package created.
Package body created.
Synonym created.
Grant succeeded.

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options


sqlplus / as sysdba @$ORACLE_HOME/rdbms/admin/xaview.sql

SQL*Plus: Release 11.2.0.4.0 Production on Wed Feb 3 11:11:51 2016
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

DROP VIEW v$xatrans$
*
ERROR at line 1:
ORA-00942: table or view does not exist
DROP VIEW v$pending_xatrans$
*
ERROR at line 1:
ORA-00942: table or view does not exist
View created.
View created.

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options






Tuesday, February 2, 2016

CHEF – Installing a Standalone CHEF Server


Hosted CHEF is a common practice and is a good way to start learning CHEF. But at times you don’t have access to the internet and hence that makes it difficult to keep working. I am getting my hands across CHEF these days and wanted to setup my own (standalone) CHEF server. I have the following setup:
A MAC machine that is my desktop – I’ll be installing the CHEF DK here and this will be my workstation.
A Linux VirtualBox server (mychef.mydomain.com) – This will be the CHEF server.
A Linux VirtualBox server (oel1.mydomain.com) – This will be the CHEF client.
This post will cover installing the standalone CHEF server. From here download the CHEF server RPM. In my case it was chef-server-core-12.3.1-1.el6.x86_64.rpm. From here download the CHEF manage rpm (chef-manage-2.1.1-1.el6.x86_64.rpm). And finally from here download the CHEF reporting rpm (opscode-reporting-1.5.6-1.el6.x86_64.rpm).  Place all the RPMs in a location that is accessible from the VirtualBox servers. I have setup a shared folder on my MAC and I share that across all my VirtualBox servers over NFS. In this example, all rpms are located at /oracle/stage.
Step 1: Install CHEF Core
From the command prompt on the CHEF server (mychef), execute the following:
[root@mychef ~]rpm -iv /oracle/stage/chef-server-core-12.3.1-1.el6.x86_64.rpm
Preparing packages for installation...
chef-server-core-12.3.1-1.el6
[root@mychef ~]#
Step 2: Install CHEF Manage
[root@mychef ~]# rpm -iv /oracle/stage/chef-manage-2.1.1-1.el6.x86_64.rpm
Preparing packages for installation...
chef-manage-2.1.1-1.el6
Thank you for installing the Chef Management Console add-on!
The next step in the process is to run:
chef-manage-ctl reconfigure
[root@mychef ~]#
Don’t run the reconfigure command just yet.
Step 3: Install CHEF Reporting
root@mychef ~]# rpm -iv /oracle/stage/opscode-reporting-1.5.6-1.el6.x86_64.rpm
Preparing packages for installation...
opscode-reporting-1.5.6-1.el6
Thank you for installing the Opscode reporting addon!
[root@mychef ~]#
Step 4: Configure CHEF Core
[root@mychef ~]# chef-server-ctl reconfigure
Starting Chef Client, version 12.5.1
resolving cookbooks for run list: ["private-chef::default"]
Synchronizing Cookbooks:
  - enterprise (0.5.1)
  - private-chef (0.1.0)
  - apt (2.7.0)
  - yum (3.6.0)
  - openssl (4.4.0)
  - runit (1.6.0)
.
.
. (lines truncated)
Running handlers:
Running handlers complete
Deprecated features used!
  Cannot specify both default and name_property together on property path of resource yum_globalconfig. Only one (name_property) will be obeyed. In Chef 13, this will become an error. at 1 location:
    - /opt/opscode/embedded/cookbooks/cache/cookbooks/yum/resources/globalconfig.rb:76:in `class_from_file'
Chef Client finished, 369/483 resources updated in 02 minutes 50 seconds
Chef Server Reconfigured!
[root@mychef ~]#
Step 5: Configure CHEF Manage
root@mychef ~]# chef-manage-ctl reconfigure
Starting Chef Client, version 12.4.1
resolving cookbooks for run list: ["omnibus-chef-manage::default"]
Synchronizing Cookbooks:
  - omnibus-chef-manage
  - chef-server-ingredient
  - enterprise
  - private_chef_addon
  - runit
  - unicorn
  - packagecloud
  - build-essential
  - yum
  - yum-epel
Compiling Cookbooks...
.
.
. (lines truncated)
Running handlers:
Running handlers complete
Chef Client finished, 63/89 resources updated in 46.284372132 seconds
chef-manage Reconfigured!
Step 6: Configure CHEF Reporting
[root@mychef ~]# opscode-reporting-ctl reconfigure
Starting Chef Client, version 12.6.0
resolving cookbooks for run list: ["opscode-reporting::default"]
Synchronizing Cookbooks:
  - enterprise (0.3.0)
  - opscode-reporting (0.2.0)
  - runit (1.3.0)
  - build-essential (2.2.3)
  - yum (3.6.1)
Compiling Cookbooks...
Recipe: opscode-reporting::default
.
.
Running handlers complete
Deprecated features used!
  Cannot specify both default and name_property together on property path of resource yum_globalconfig. Only one (name_property) will be obeyed. In Chef 13, this will become an error. at 1 location:
    - /opt/opscode-reporting/embedded/cookbooks/cache/cookbooks/yum/resources/globalconfig.rb:76:in `class_from_file'
Chef Client finished, 37/55 resources updated in 20 seconds
opscode-reporting Reconfigured!
Step 7: Check if services are up:
[root@mychef ~]# chef-server-ctl status
run: bookshelf: (pid 6592) 504s; run: log: (pid 5479) 550s
run: nginx: (pid 8334) 80s; run: log: (pid 6212) 511s
run: oc_bifrost: (pid 6353) 509s; run: log: (pid 4821) 620s
run: oc_id: (pid 6408) 508s; run: log: (pid 4893) 610s
run: opscode-erchef: (pid 6680) 502s; run: log: (pid 5648) 544s
run: opscode-expander: (pid 6504) 505s; run: log: (pid 5131) 595s
run: opscode-expander-reindexer: (pid 6550) 504s; run: log: (pid 5334) 556s
run: opscode-reporting: (pid 8347) 79s; run: log: (pid 8307) 82s
run: opscode-solr4: (pid 6460) 506s; run: log: (pid 5086) 598s
run: postgresql: (pid 6330) 509s; run: log: (pid 4307) 636s
run: rabbitmq: (pid 6237) 510s; run: log: (pid 2898) 658s\
run: redis_lb: (pid 5908) 526s; run: log: (pid 5901) 526s
[root@mychef ~]# chef-manage-ctl status
run: events: (pid 7353) 175s; run: log: (pid 7363) 173s
run: redis: (pid 7504) 164s; run: log: (pid 7316) 179s
run: web: (pid 7514) 163s; run: log: (pid 7438) 169s
run: worker: (pid 7455) 166s; run: log: (pid 7485) 164s
Step 8: Create an Administrator for CHEF
[root@mychef opscode]# chef-server-ctl user-create admin admin admin admin@foo.com welcome1 --filename /opt/opscode/admin.pem
Step 9: Create an Organization for CHEF
[root@mychef opscode]# chef-server-ctl org-create adminorg "admin org" --association_user admin
Step 10: Login to CHEF UI
From a browser, open https://mychef.mydomain.com (add the certificate exceptions to your browser)

-->
Enter the user credentials you created in Step 8 and Sign in.
 
--> Your CHEF server configurations are now complete.  

Monday, February 1, 2016

Solaris - Setting up GUI

Solaris was the first UNIX-like system I worked with but I hadn't touched it since Solaris 9. I had to setup Oracle Identity Manager on Solaris 11 for a project recently. To practice, I installed an x86 version on my local VirtualBox. To my surprise, I couldn't find a GUI installer for Solaris 11 (or rather I must have not looked enough). So after installing the text installer, I had to install X11 on it.

The following steps made it possible. Do note that you must have access to the internet from within your guest VM to be able to execute the below steps successfully and must be root (or equivalent).

root@sol1:~# beadm list 
BE      Flags Mountpoint Space Policy Created         
--      ----- ---------- ----- ------ -------         
solaris NR    /          3.33G static 2016-01-29 04:16

This command shows the boot environments available. What we are trying to do, is to install the GUI into a new boot environment while working on the text-based console (default boot environment). Once the GUI boot environment is setup, we'll switch to it and delete the text based one.

root@sol1:~# beadm create sol-gui
root@sol1:~# beadm mount sol-gui /gui
root@sol1:~# beadm list

BE      Flags Mountpoint Space  Policy Created         
--      ----- ---------- -----  ------ -------         
sol-gui -     /gui       187.5K static 2016-01-29 05:15
solaris NR    /          3.33G  static 2016-01-29 04:16

These commands will create a new boot environment and mount it against the file system. The "sol-gui" and "/gui" could be anything.

root@sol1:~# pkg -R /gui install group/system/solaris-desktop
Packages to install: 367
Services to change:  13

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                            367/367   48040/48040  624.7/624.7  340k/s

PHASE                                          ITEMS
Installing new actions                   80531/80531
Updating package state database                 Done
Updating package cache                           0/0
Updating image state                            Done
Creating fast lookup database                   Done
Updating package cache                           1/1

The pkg command lets you install a package (in this case, the X11 packages) in a given directory. This process will take a while depending on the speed of the internet connection you have.

root@sol1:~# bootadm update-archive -R /gui 
Use bootadm command to then update the existing boot archives, basically telling Solaris that there is another boot environment we want to register in the boot menu (updates GRUB).

root@sol1:~# beadm umount sol-gui
root@sol1:~# beadm activate sol-gui

These commands will unmount the new boot environment and activate it.
 
root@sol1:~# beadm list

BE      Flags Mountpoint Space Policy Created         
--      ----- ---------- ----- ------ -------         
sol-gui R     -          5.77G static 2016-01-29 05:15
solaris N     /          4.52M static 2016-01-29 04:16

The flags "R" means active on reboot and "N" means active now.

root@sol1:~# init 6

Reboot the system for settings to take place. After reboot, the new boot environment (sol-gui) will become the default.

root@sol1:~# beadm list

BE      Flags Mountpoint Space Policy Created         
--      ----- ---------- ----- ------ -------         
sol-gui NR    /          6.06G static 2016-01-29 05:15
solaris -     -          5.30M static 2016-01-29 04:16

root@sol1:~# beadm destroy solaris
Are you sure you want to destroy solaris?  This action cannot be undone(y/[n]): y

This command will delete the old text based boot environment leaving just the GUI based one.

root@sol1:~# beadm list

BE      Flags Mountpoint Space Policy Created         
--      ----- ---------- ----- ------ -------         
sol-gui NR    /          5.98G static 2016-01-29 05:15