Tuesday, July 1, 2014

Learning Openshift Origin

This post will be about my experience installing OpenShift Origin in my company's servers.

Installation

I find the comprehensive deployment guide in OpenShift site (http://openshift.github.io/documentation/oo_deployment_guide_comprehensive.html) is very useful, but not without flaws. My OS is RHEL, but I think my experiences would also apply to CentOS systems.
The first glitch found was that the yum update went broken just after step 1.1 (repository configuration). The problem was a complex dependency between packages, in short it could be fixed after I do "yum erase libart_lgpl-devel". That package is not needed for correct system operation.
I noted that the mcollective installation (Chapter 5) as prerequisite doesn't mention that in RHEL we need different package than Fedora, that is

yum install -y ruby193-mcollective-client


Openshift installation liberally uses SCL (software collections) packages, in which a different root  is used to install newer versions of certain software packages. For example, the ruby193 SCL is installed in (/opt/rh/ruby193/root/).  The old version of ruby (1.8.7) is still installed in /usr/bin/ruby, and when we need to use ruby-1.9.3 we need to add the /opt/rh/ruby193/root/usr/bin path to PATH and LD_LIBRARY_PATH. It could be done by the command :
scl enable ruby193 bash
But the openshift installation does this globally by creating /etc/profile.d/scl193.sh, making the scl path available to the global profile.

For most of the deployment guide, from top to bottom is the correct sequence to apply. I said mostly because the missing ruby193-mcollective-client made my system without /opt/rh/ruby193/root/etc/mcollective/client.cfg to configure, and so I must do the configuration in later steps when the file would be available.

Complications arise because the newer version of v8 engine, previously packaged as ruby193-v8, now packaged as different scl altogether (v8314-v8 and or v8314-runtime). A new scl folder now required to be integrated to openshift applications, such as admin-console and developer console. If no adjustments were made, developer console shows blank in the browser and missing javascript runtime error shows in admin-console. In my system this is done by changing  /var/www/openshift/console/script/console_ruby and /var/www/openshift/broker/script to include additional library path from the v8314 scl :

export LD_LIBRARY_PATH=/opt/rh/ruby193/root/usr/local/lib64:/opt/rh/ruby193/root/usr/lib64:/opt/rh/v8314/root/usr/lib64


Installation packages

In summary, here are packages that I installed in the broker host:
bind.x86_64                                32:9.8.2-0.23.rc1.el6_5.1     @updates
bind-libs.x86_64                           32:9.8.2-0.23.rc1.el6_5.1     @anaconda-RedHatEnterpriseLinux-201009221801.x86_64/6.3
bind-utils.x86_64                          32:9.8.2-0.23.rc1.el6_5.1     @anaconda-RedHatEnterpriseLinux-201009221801.x86_64/6.3

libmongodb.x86_64                          2.4.6-2.el6oso                @openshift-origin-deps

mongodb.x86_64                             2.4.6-2.el6oso                @openshift-origin-deps
openshift-origin-broker.noarch             1.15.1.2-1.el6                @openshift-origin
openshift-origin-broker-util.noarch        1.18.0.1-1.el6                @openshift-origin
openshift-origin-console.noarch            1.15.1.3-1.el6                @openshift-origin
openshift-origin-msg-common.noarch         1.17.0.1-1.el6                @openshift-origin
openshift-origin-util-scl.noarch           1.16.1.1-1.el6                @openshift-origin
ruby193-mcollective-client.noarch          2.2.3-2.el6oso                @openshift-origin-deps
ruby193-mcollective-common.noarch          2.2.3-2.el6oso                @openshift-origin-deps


Configuration files

The installation procedure also involves creating various configuration files :

Reference to openshift rpms from Red hat 's openshift repository :
/etc/yum.repos.d/openshift-origin-deps.repo
/etc/yum.repos.d/openshift-origin.repo
Environment variables changes to activate ruby193
/etc/profile.d/scl193.sh
Name server cloud domain configuration 
/var/named/dynamic/${domain}.db
/var/named/${domain}.key
Name server upstream configuration
/var/named/forwarders.conf
Name server and key to be used by broker
/etc/openshift/plugins.d/openshift-origin-dns-nsupdate.conf
Console authentication configuration
/var/www/openshift/console/httpd/conf.d/openshift-origin-auth-remote-user-basic.conf
Broker authentication configuration
/var/www/openshift/broker/httpd/conf.d/openshift-origin-auth-remote-user.conf

Some other files must be changed according to the guide :
Dns settings
/etc/resolv.conf
/etc/dhcp/dhclient-eth0.conf
/etc/sysconfig/network-scripts/ifcfg-
Hostname setting
/etc/sysconfig/network
Name server configuration to refer cloud domain config
/etc/named.conf
Mcollective configuration to use activemq service
/opt/rh/ruby193/root/etc/mcollective/client.cfg
Gear size, cloud domain, and mongodb service to be used by broker
/etc/openshift/broker.conf

Time keeping

To ensure best time keeping behavior using my company's VM-based infrastructure, I applied VMware KB 1006427 (see the KB here), in essence :
  • add tinker panic 0 in top of /etc/ntp.conf
  • comment Undisciplined Local Clock in /etc/ntp.conf, preventing NTP to prefer local clock rather than external network time source.

Node installation

Packages that I installed in the node host is : 
openshift-origin-msg-node-mcollective.noarch
                                           1.18.0.1-1.el6                @openshift-origin
rubygem-openshift-origin-node.noarch       1.18.0.1-1.el6                @openshift-origin
rubygem-passenger-native.x86_64            1:3.0.21-11.el6oso            @openshift-origin-deps
openshift-origin-port-proxy.noarch         1.8.1.2-1.el6                 @openshift-origin
openshift-origin-node-util.noarch          1.18.0.2-1.el6                @openshift-origin
rubygem-openshift-origin-container-selinux.noarch
                                           0.4.1.1-1.el6                 @openshift-origin
rubygem-openshift-origin-frontend-apache-mod-rewrite.noarch
                                           0.3.1.1-1.el6                 @openshift-origin
rubygem-openshift-origin-frontend-nodejs-websocket.noarch
                                           0.3.0.1-1.el6                 @openshift-origin
openshift-origin-cartridge-cron.noarch     1.17.0.1-1.el6                @openshift-origin
openshift-origin-cartridge-haproxy.noarch  1.18.0.1-1.el6                @openshift-origin
openshift-origin-cartridge-mongodb.noarch  1.17.1.1-1.el6                @openshift-origin
openshift-origin-cartridge-mysql.noarch    1.19.0.1-1.el6                @openshift-origin
openshift-origin-cartridge-php.noarch      1.18.0.3-1.el6                @openshift-origin

Network Troubles in Node

Several services will not start if your LAN device is not eth1. It is because most of the openshift shell scripts assume eth1 for network connectivity. Or it might occurred because I didn't specify EXTERNAL_ETH_DEV in /etc/openshift/node.conf. Anyway the cure is to set EXTERNAL_ETH_DEV  in /etc/openshift/node.conf

Summary

Installing openshift origin in multiple host is a priceless experience. I get to know some of the mechanisms that openshift relied on.

No comments: