Sunday, October 6, 2013

Installing Saprfc on CentOS 6

Our PHP web servers were powered by Linux, either Red Hat Enterprise Linux or CentOS Linux. Normally we use 5.x version of CentOS or RHEL, but after a few years running 5.x we decided that its now time to upgrade to 6.x.
The first problem we encounter is that the SAPCAR utility shows loading shared libraries error.

./SAPCAR_1-20002087.EXE RFC_45-10003377.SAR
./SAPCAR_1-20002087.EXE: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory


Checking the EXE file (actually is a Linux ELF binary but with misleading extension) with LDD, we find that :
[root@myserver opt]# ldd SAPCAR_1-20002087.EXE
        linux-gate.so.1 =>  (0x00e81000)
        libdl.so.2 => /lib/libdl.so.2 (0x00d05000)
        librt.so.1 => /lib/librt.so.1 (0x00317000)
        libstdc++.so.6 => not found
        libm.so.6 => /lib/libm.so.6 (0x003b0000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x0083d000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x005b7000)
        libc.so.6 => /lib/libc.so.6 (0x00694000)
        /lib/ld-linux.so.2 (0x00672000) 
Seems that the libstdc++ is missing. 
[root@myserver opt]# rpm -ql libstdc++
/usr/lib64/libstdc++.so.6
/usr/lib64/libstdc++.so.6.0.13
 But it is not missing... whats wrong here ? See the details. LDD's output is that the libraries in the '/lib' folder, and the libstdc++ exist in '/usr/lib64'. The /lib folder is for 32-bit libraries. So there the SAPCAR actually requires the 32-bit version. Lets install the 32-bit version :
[root@myserver ~]# yum install libstdc++.i386
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: buaya.klas.or.id
 * extras: buaya.klas.or.id
 * updates: buaya.klas.or.id
Setting up Install Process
No package libstdc++.i386 available.
 
OK, maybe this is not the 386 era anymore :
[root@myserver ~]# yum install libstdc++.i686
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: buaya.klas.or.id
 * extras: buaya.klas.or.id
 * updates: buaya.klas.or.id
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libstdc++.i686 0:4.4.7-3.el6 will be installed
--> Finished Dependency Resolution
Error: Protected multilib versions: libstdc++-4.4.7-3.el6.i686 != libstdc++-4.4.6-3.el6.x86_64
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

Yum refuses to install 32-bit library whose version number are different from the 64-bit library. So lets update both libraries.
[root@myserver ~]# yum install libstdc++.i686 libstdc++.x86_64
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: buaya.klas.or.id
 * extras: buaya.klas.or.id
 * updates: buaya.klas.or.id
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libstdc++.x86_64 0:4.4.6-3.el6 will be updated
--> Processing Dependency: libstdc++ = 4.4.6-3.el6 for package: gcc-c++-4.4.6-3.el6.x86_64
--> Processing Dependency: libstdc++(x86-64) = 4.4.6-3.el6 for package: libstdc++-devel-4.4.6-3.el6.x86_64
---> Package libstdc++.i686 0:4.4.7-3.el6 will be installed
---> Package libstdc++.x86_64 0:4.4.7-3.el6 will be an update
--> Running transaction check
---> Package gcc-c++.x86_64 0:4.4.6-3.el6 will be updated
---> Package gcc-c++.x86_64 0:4.4.7-3.el6 will be an update
--> Processing Dependency: gcc = 4.4.7-3.el6 for package: gcc-c++-4.4.7-3.el6.x86_64
---> Package libstdc++-devel.x86_64 0:4.4.6-3.el6 will be updated
...
Installed:
  libstdc++.i686 0:4.4.7-3.el6

Updated:
  libstdc++.x86_64 0:4.4.7-3.el6

Dependency Updated:
  cpp.x86_64 0:4.4.7-3.el6              gcc.x86_64 0:4.4.7-3.el6
  gcc-c++.x86_64 0:4.4.7-3.el6          gcc-gfortran.x86_64 0:4.4.7-3.el6
  gcc-java.x86_64 0:4.4.7-3.el6         libgcc.i686 0:4.4.7-3.el6
  libgcc.x86_64 0:4.4.7-3.el6           libgcj.x86_64 0:4.4.7-3.el6
  libgcj-devel.x86_64 0:4.4.7-3.el6     libgfortran.x86_64 0:4.4.7-3.el6
  libgomp.x86_64 0:4.4.7-3.el6          libstdc++-devel.x86_64 0:4.4.7-3.el6

Complete!
 
And retry the saprfc installation  :
[root@myserver opt]# ./SAPCAR_1-20002087.EXE -xvf RFC_45-10003377.SAR
SAPCAR: processing archive RFC_45-10003377.SAR (version 2.00)
x rfcsdk
x rfcsdk/bin
x rfcsdk/bin/genh
.
.
x rfcsdk/include/trfcserv.h
x rfcsdk/include/trfctest.h
SAPCAR: 46 file(s) extracted
[root@myserver opt]# mkdir /usr/sap
[root@myserver opt]# mv rfcsdk/ /usr/sap
 

Now download the saprfc extension from http://sourceforge.net/projects/saprfc/files/saprfc/, extract it :
[root@myserver opt]# tar -xvzf saprfc-1.4.1.tar.gz
saprfc-1.4.1/
saprfc-1.4.1/sapclasses/
saprfc-1.4.1/sapclasses/examples/
saprfc-1.4.1/sapclasses/examples/example_client.php
saprfc-1.4.1/sapclasses/examples/example_connect1.php
.

.
saprfc-1.4.1/test_sso.php
saprfc-1.4.1/trfcserv.php


Do phpize to prepare php extension compilation.
[root@myserver saprfc-1.4.1]# phpize
Configuring for:
PHP Api Version:         20090626
Zend Module Api No:      20090626
Zend Extension Api No:   220090626


And now do the classic configure and make install :

[root@myserver saprfc-1.4.1]# ./configure
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
checking for cc... cc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
.
.
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
configure: creating ./config.status
config.status: creating config.h
config.status: executing libtool commands

[root@myserver saprfc-1.4.1]# make install
/bin/sh /opt/saprfc-1.4.1/libtool --mode=compile cc  -I. -I/opt/saprfc-1.4.1 -DPHP_ATOM_INC -I/opt/saprfc-1.4.1/include -I/opt/saprfc-1.4.1/main -I/opt/saprfc-1.4.1 -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib -I/usr/sap/rfcsdk/include  -DHAVE_CONFIG_H  -g -O2   -c /opt/saprfc-1.4.1/saprfc.c -o saprfc.lo
libtool: compile:  cc -I. -I/opt/saprfc-1.4.1 -DPHP_ATOM_INC -I/opt/saprfc-1.4.1/include -I/opt/saprfc-1.4.1/main -I/opt/saprfc-1.4.1 -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib -I/usr/sap/rfcsdk/include -DHAVE_CONFIG_H -g -O2 -c /opt/saprfc-1.4.1/saprfc.c  -fPIC -DPIC -o .libs/saprfc.o
/opt/saprfc-1.4.1/saprfc.c: In function ‘zif_saprfc_open’:
/opt/saprfc-1.4.1/saprfc.c:481: warning: ‘zend_get_parameters_ex’ is deprecated (declared at /usr/include/php/Zend/zend_API.h:222)
/opt/saprfc-1.4.1/saprfc.c: In function ‘zif_saprfc_function_discover’:
/opt/saprfc-1.4.1/saprfc.c:551: warning: ‘zend_get_parameters_ex’ is deprecated (declared at /usr/include/php/Zend/zend_API.h:222)
/opt/saprfc-1.4.1/saprfc.c:555: warning: ‘zend_get_parameters_ex’ is deprecated (declared at /usr/include/php/Zend/zend_API.h:222)
.
.
Libraries have been installed in:
   /opt/saprfc-1.4.1/modules

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
Installing shared extensions:     /usr/lib64/php/modules/

Now lets try enable the extension and restart httpd :
[root@myserver saprfc-1.4.1]# cd /etc/php.d
[root@myserver php.d]# cat > saprfc.ini
; Enable saprfc extension module
extension=saprfc.so
[root@myserver php.d]# service httpd restart


Ok we still have another problem, namely SELinux :
PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/saprfc.so' - librfccm.so: failed to map segment from shared object: Permission denied in Unknown on line 0

We need to change several SELinux file attributes.
[root@myserver php.d]# cd /usr/lib64/php/modules
[root@myserver modules]# ls -Z
...
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       pdo.so
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       pdo_sqlite.so
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       phar.so
-rwxr-xr-x. root root unconfined_u:object_r:lib_t:s0   saprfc.so
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       sqlite3.so
[root@myserver modules]# restorecon -F saprfc.so
...
[root@myserver modules]# ls -Z
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       pdo_sqlite.so
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       phar.so
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       saprfc.so
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       sqlite3.so

And we need the linked library also.
[root@myserver modules]# ldd saprfc.so
        linux-vdso.so.1 =>  (0x00007fffd0dde000)
        librfccm.so => /usr/sap/rfcsdk/lib/librfccm.so (0x00007f8008f27000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f8008b6e000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f800896a000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f8008762000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8008545000)
        libstdc++.so.5 => /usr/lib64/libstdc++.so.5 (0x00007f800826a000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f8007fe6000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8007dcf000)
        /lib64/ld-linux-x86-64.so.2 (0x00000031e3a00000)
[root@myserver modules]# ls -Z  /usr/sap/rfcsdk/lib/
-rw-rw-r--. root root unconfined_u:object_r:usr_t:s0   librfc.a
-rwxrwxr-x. root root unconfined_u:object_r:usr_t:s0   librfccm.so

[root@myserver modules]# restorecon -v -F /usr/sap/rfcsdk/lib/
restorecon reset /usr/sap/rfcsdk/lib context unconfined_u:object_r:usr_t:s0->system_u:object_r:lib_t:s0
[root@myserver modules]# restorecon -v -F /usr/sap/rfcsdk/lib/*.so
restorecon reset /usr/sap/rfcsdk/lib/librfccm.so context unconfined_u:object_r:usr_t:s0->system_u:object_r:lib_t:s0

Restart HTTPD again.
[root@myserver modules]# service httpd restart

And all is well after that. Well, except that to be able to connect to other servers, we need to set SELinux policy to allow network connection from apache :
[root@myserver ~]# getsebool -a | grep httpd | grep network
httpd_can_network_connect --> off
httpd_can_network_connect_cobbler --> off
httpd_can_network_connect_db --> off
httpd_can_network_memcache --> off
httpd_can_network_relay --> off
[root@myserver ~]# setsebool -P httpd_can_network_connect on


The last command tooks a while in my server, please be patient.

Bad Elf Interpreter

Another variation of this problem is a bad ELF interpreter response :
rra-mobile ~ # ./SAPCAR_1-20002087.EXE
-bash: ./SAPCAR_1-20002087.EXE: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory
It happens that this cryptic error message were caused by missing glibc.i686 library, so the solution is simple :
rra-mobile ~ # yum install glibc.i686 glibc-common.i686

Silent AVC Denial

Another issue that might blocks the extension is SE linux denies access to the directory containing librfccm.so. The problem is, the denial is not logged in alert.log, making troubleshooting difficult if not impossible. The indication is :
do:
    service httpd restart
error message in /var/log/httpd/error_log :
[Sun Jan 26 07:30:45 2014] [notice] Digest: done
PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/saprfc.so' - librfccm.so: cannot open shared object file: No such file or directory in Unknown on line 0
do:
    setenforce 0
    service httpd restart
 no errors occured. 

I tried to fix this :
   setenforce 1
   restorecon -vF /usr/sap/rfcsdk
   service httpd restart

And no errors now..

Conclusion

Installing saprfc on CentOS 6 is not for the faint hearted. The system admin must be well versed in SELinux aspects as well.