Enhance your AIX packages management with yum and nim over http

As AIX is getting older and older our old favorite OS is still trying to struggle versus the mighty Linux and the fantastic Solaris (no sarcasm in that sentence I truly believe what I say). You may have notice that -with time- IBM is slowly but surely moving from proprietary code to something more open (ie. PowerVC/Openstack projects, integration with Chef, Linux on Power and tons of other examples). I’m a little bit deviating from the main topic of this blog post but speaking about open source I have many things to say. If someone from my company is reading this post please note that it is my point of view … but I’m still sure that we are going the WRONG way not being more open, and not publishing on github. Starting from now every AIX IT shop in world must consider using OpenSource software (git, chef, ansible, zsh and so on) instead of maintaining homemade tools, or worse paying for tools that are 100 % of the time worse than OpenSource tools. Even better, every IT admin and every team must consider sharing their sources with the rest of the world for one single good reason: “Alone we can do so little, together we can do so much”. Every company not considering this today is doomed. Take example on Bloomberg, Facebook (sharing to the world all their Chef’s cookbooks), twitter, they’re all using github to share their opensource projects. Even military, police and banks are doing the same. They’re still secure but they are open to world ready work to make and create things better and better. All of this to introduce you to new things coming on AIX. Instead of reinventing the wheel IBM had the great idea to use already well implanted tools. It was the case for Openstack/PowerVC and it is also for the tools I’ll talk about in this post. It is the case for yum (yellowdog updater modified). Instead of installing rpm packages by hand you now have the possibility to use yum and to definitely end the rpm dependency nightmare that we all had since AIX 5L was released. Next instead of using the proprietary nimsh protocol to install filesets (bff package) you can now tell the nim server and nimclient to this over http/https (secure is only for the authentication as far as I know) (an open protocol :-) ). By doing this you will enhance the way you are managing packages on AIX. Do this now on every AIX system you install, yum everywhere and stop using NFS … we’re now in an http world :-)

yum: the yellow dog updater modified

I’m not going to explain you what yum is. If you don’t know you’re not in the right place. Just note that my advice starting from now is to use yum to install every software of the AIX toolbox (ftp://ftp.software.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc/). IBM is providing an official repository than can be mirrored on your own site to avoid having to use a proxy or having an access to the internet from you servers (you must admit that this is almost impossible and every big company will try to avoid this). Let’s start by trying to install yum:

Installing yum

IBM is providing an archive with all the needed rpm mandatory to use and install yum on an AIX server, you can find this archive here: ftp://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/ezinstall/ppc/yum_bundle_v1.tar. Just download it and install every rpm in it and yum will be available on you system, simple as that:

A specific version of rpm binary command is mandatory to use yum. Before doing anything update the rpm.rte fileset. As AIX is rpm “aware” it already have an rpm database, but this one will not be manageable by yum. The installation of rpm in a version greater than 4.9.1.3 is needed. This installation will migrate the existing rpm database to a new one usable by yum. The fileset in the right version can be found here ftp://ftp.software.ibm.com/aix/freeSoftware/aixtoolbox/INSTALLP/ppc/

  • By default the rpm command is installed by an AIX fileset:
  • # which rpm
    /usr/bin/rpm
    # lslpp -w /usr/bin/rpm
      File                                        Fileset               Type
      ----------------------------------------------------------------------------
      /usr/bin/rpm                                rpm.rte               File
    # rpm --version
    RPM version 3.0.5
    
  • The rpm database is located in /usr/opt/freeware/packages :
  • # pwd
    /usr/opt/freeware/packages
    # ls -ltr
    total 5096
    -rw-r--r--    1 root     system         4096 Jul 01 2011  triggerindex.rpm
    -rw-r--r--    1 root     system         4096 Jul 01 2011  conflictsindex.rpm
    -rw-r--r--    1 root     system        20480 Jul 21 00:54 nameindex.rpm
    -rw-r--r--    1 root     system        20480 Jul 21 00:54 groupindex.rpm
    -rw-r--r--    1 root     system      2009224 Jul 21 00:54 packages.rpm
    -rw-r--r--    1 root     system       647168 Jul 21 00:54 fileindex.rpm
    -rw-r--r--    1 root     system        20480 Jul 21 00:54 requiredby.rpm
    -rw-r--r--    1 root     system        81920 Jul 21 00:54 providesindex.rpm
    
  • Install the rpm.rte fileset in the right version (4.9.1.3):
  • # file rpm.rte.4.9.1.3
    rpm.rte.4.9.1.3: backup/restore format file
    # installp -aXYgd . rpm.rte
    +-----------------------------------------------------------------------------+
                        Pre-installation Verification...
    +-----------------------------------------------------------------------------+
    Verifying selections...done
    Verifying requisites...done
    Results...
    
    SUCCESSES
    ---------
      Filesets listed in this section passed pre-installation verification
      and will be installed.
    
      Selected Filesets
      -----------------
      rpm.rte 4.9.1.3                             # RPM Package Manager
    [..]
    #####################################################
            Rebuilding RPM Data Base ...
            Please wait for rpm_install background job termination
            It will take a few minutes
    [..]
    Installation Summary
    --------------------
    Name                        Level           Part        Event       Result
    -------------------------------------------------------------------------------
    rpm.rte                     4.9.1.3         USR         APPLY       SUCCESS
    rpm.rte                     4.9.1.3         ROOT        APPLY       SUCCESS
    
  • After the installation check you have the correct version of rpm, you can also notice some changes in the rpm database files:
  • # rpm --version
    RPM version 4.9.1.3
    # ls -ltr /usr/opt/freeware/packages
    total 25976
    -rw-r--r--    1 root     system         4096 Jul 01 2011  triggerindex.rpm
    -rw-r--r--    1 root     system         4096 Jul 01 2011  conflictsindex.rpm
    -rw-r--r--    1 root     system        20480 Jul 21 00:54 nameindex.rpm
    -rw-r--r--    1 root     system        20480 Jul 21 00:54 groupindex.rpm
    -rw-r--r--    1 root     system      2009224 Jul 21 00:54 packages.rpm
    -rw-r--r--    1 root     system       647168 Jul 21 00:54 fileindex.rpm
    -rw-r--r--    1 root     system        20480 Jul 21 00:54 requiredby.rpm
    -rw-r--r--    1 root     system        81920 Jul 21 00:54 providesindex.rpm
    -rw-r--r--    1 root     system            0 Jul 21 01:08 .rpm.lock
    -rw-r--r--    1 root     system         8192 Jul 21 01:08 Triggername
    -rw-r--r--    1 root     system         8192 Jul 21 01:08 Conflictname
    -rw-r--r--    1 root     system        28672 Jul 21 01:09 Dirnames
    -rw-r--r--    1 root     system       221184 Jul 21 01:09 Basenames
    -rw-r--r--    1 root     system         8192 Jul 21 01:09 Sha1header
    -rw-r--r--    1 root     system         8192 Jul 21 01:09 Requirename
    -rw-r--r--    1 root     system         8192 Jul 21 01:09 Obsoletename
    -rw-r--r--    1 root     system         8192 Jul 21 01:09 Name
    -rw-r--r--    1 root     system         8192 Jul 21 01:09 Group
    -rw-r--r--    1 root     system       815104 Jul 21 01:09 Packages
    -rw-r--r--    1 root     system         8192 Jul 21 01:09 Sigmd5
    -rw-r--r--    1 root     system         8192 Jul 21 01:09 Installtid
    -rw-r--r--    1 root     system        86016 Jul 21 01:09 Providename
    -rw-r--r--    1 root     system       557056 Jul 21 01:09 __db.004
    -rw-r--r--    1 root     system     83894272 Jul 21 01:09 __db.003
    -rw-r--r--    1 root     system      7372800 Jul 21 01:09 __db.002
    -rw-r--r--    1 root     system        24576 Jul 21 01:09 __db.001
    

Then install yum. Please note that I already have some rpm installed on my current system that’s why I’m not installing db, or gdbm. If your system is free of any rpm install all the rpm found in the archive:

# tar xvf yum_bundle_v1.tar
x curl-7.44.0-1.aix6.1.ppc.rpm, 584323 bytes, 1142 media blocks.
x db-4.8.24-3.aix6.1.ppc.rpm, 2897799 bytes, 5660 media blocks.
x gdbm-1.8.3-5.aix5.2.ppc.rpm, 56991 bytes, 112 media blocks.
x gettext-0.10.40-8.aix5.2.ppc.rpm, 1074719 bytes, 2100 media blocks.
x glib2-2.14.6-2.aix5.2.ppc.rpm, 1686134 bytes, 3294 media blocks.
x pysqlite-1.1.7-1.aix6.1.ppc.rpm, 51602 bytes, 101 media blocks.
x python-2.7.10-1.aix6.1.ppc.rpm, 23333701 bytes, 45574 media blocks.
x python-devel-2.7.10-1.aix6.1.ppc.rpm, 15366474 bytes, 30013 media blocks.
x python-iniparse-0.4-1.aix6.1.noarch.rpm, 37912 bytes, 75 media blocks.
x python-pycurl-7.19.3-1.aix6.1.ppc.rpm, 162093 bytes, 317 media blocks.
x python-tools-2.7.10-1.aix6.1.ppc.rpm, 830446 bytes, 1622 media blocks.
x python-urlgrabber-3.10.1-1.aix6.1.noarch.rpm, 158584 bytes, 310 media blocks.
x readline-6.1-2.aix6.1.ppc.rpm, 489547 bytes, 957 media blocks.
x sqlite-3.7.15.2-2.aix6.1.ppc.rpm, 1334918 bytes, 2608 media blocks.
x yum-3.4.3-1.aix6.1.noarch.rpm, 1378777 bytes, 2693 media blocks.
x yum-metadata-parser-1.1.4-1.aix6.1.ppc.rpm, 62211 bytes, 122 media blocks.
# rpm -Uvh curl-7.44.0-1.aix6.1.ppc.rpm glib2-2.14.6-2.aix5.2.ppc.rpm pysqlite-1.1.7-1.aix6.1.ppc.rpm python-2.7.10-1.aix6.1.ppc.rpm python-devel-2.7.10-1.aix6.1.ppc.rpm python-iniparse-0.4-1.ai
x6.1.noarch.rpm python-pycurl-7.19.3-1.aix6.1.ppc.rpm python-tools-2.7.10-1.aix6.1.ppc.rpm python-urlgrabber-3.10.1-1.aix6.1.noarch.rpm yum-3.4.3-1.aix6.1.noarch.rpm yum-metadata-parser-1.1.4-
1.aix6.1.ppc.rpm
# Preparing...                ########################################### [100%]
   1:python                 ########################################### [  9%]
   2:pysqlite               ########################################### [ 18%]
   3:python-iniparse        ########################################### [ 27%]
   4:glib2                  ########################################### [ 36%]
   5:yum-metadata-parser    ########################################### [ 45%]
   6:curl                   ########################################### [ 55%]
   7:python-pycurl          ########################################### [ 64%]
   8:python-urlgrabber      ########################################### [ 73%]
   9:yum                    ########################################### [ 82%]
  10:python-devel           ########################################### [ 91%]
  11:python-tools           ########################################### [100%]

Yum is now ready to be configured and used :-)

# which yum
/usr/bin/yum
# yum --version
3.4.3
  Installed: yum-3.4.3-1.noarch at 2016-07-20 23:24
  Built    : None at 2016-06-22 14:13
  Committed: Sangamesh Mallayya  at 2014-05-29

Setting up yum and you private yum repository for AIX

A private repository

As nobody wants to use the official IBM repository available directly on internet the goal here is to create your own repository. Download all the content of the official repository and “serve” this directory (the one where you download all the rpms) on an private http server (yum is using http/https obviously :-) ).

  • Using wget download the content of the whole official repository. You can notice here that IBM is providing the metadata needed (repodata directory) (if you don’t have this repodata directory yum can’t work properly. This one can be created using the createrepo command available on akk good Linux distros :-) ):
  • # wget -r ftp://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc/
    # ls -ltr
    [..]
    drwxr-xr-x    2 root     system         4096 Jul 11 22:08 readline
    drwxr-xr-x    2 root     system          256 Jul 11 22:08 rep-gtk
    drwxr-xr-x    2 root     system         4096 Jul 11 22:08 repodata
    drwxr-xr-x    2 root     system         4096 Jul 11 22:08 rpm
    drwxr-xr-x    2 root     system         4096 Jul 11 22:08 rsync
    drwxr-xr-x    2 root     system          256 Jul 11 22:08 ruby
    drwxr-xr-x    2 root     system          256 Jul 11 22:09 rxvt
    drwxr-xr-x    2 root     system         4096 Jul 11 22:09 samba
    drwxr-xr-x    2 root     system          256 Jul 11 22:09 sawfish
    drwxr-xr-x    2 root     system          256 Jul 11 22:09 screen
    drwxr-xr-x    2 root     system          256 Jul 11 22:09 scrollkeeper
    
  • Configure you web server (here it’s just an alias because I’m using my http server for other things):
  • # more httpd.conf
    [..]
    Alias /aixtoolbox/  "/apps/aixtoolbox/"
    
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Require all granted
    
    
  • Restart your webserver and check you repository is accessible:
  • repo

  • That’s it the private repository is ready.

Configuring yum

On the client just modify the /opt/freeware/etc/yum/yum.conf or add a file in /opt/freeware/etc/yum/yum.repos.d to point to your private repository:

# cat /opt/freeware/etc/yum/yum.conf
[main]
cachedir=/var/cache/yum
keepcache=1
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1

[AIX_Toolbox]
name=AIX ToolBox Repository
baseurl=http://nimserver:8080/aixtoolbox/
enabled=1
gpgcheck=0

# PUT YOUR REPOS HERE OR IN separate files named file.repo
# in /etc/yum/repos.d

That’s it the client is ready.

Chef recipe to install and configre yum

My readers all knows that I’m using Chef as a configuration management tools. As you are going to do this on every single system you have I think giving you the Chef recipe installing and configuring yum can be useful (if you don’t care about it just skip it and go to the next session). If you are not using a configuration management tool maybe this simple example will help you to move on and stop doing this by hand or writing ksh scripts. I have to do that on tons of system so for me it’s just mandatory. Here is my recipe to do all the job, configuring and installing yum, and installing some RPM:

directory '/var/tmp/yum' do
  action :create
end

remote_file '/var/tmp/yum/rpm.rte.4.9.1.3'  do
  source "http://#{node['nimserver']}/powervc/rpm.rte.4.9.1.3"
  action :create
end

execute "Do the toc" do
  command 'inutoc /var/tmp/yum'
  not_if { File.exist?('/var/tmp/yum/.toc') }
end

bff_package 'rpm.rte' do
  source '/var/tmp/yum/rpm.rte.4.9.1.3'
  action :install
end

tar_extract "http://#{node['nimserver']/powervc/yum_bundle_v1.tar" do
  target_dir '/var/tmp/yum'
  compress_char ''
  user 'root'
  group 'system'
end

# installing some rpm needed for yum
for rpm in [ 'curl-7.44.0-1.aix6.1.ppc.rpm', 'python-pycurl-7.19.3-1.aix6.1.ppc.rpm', 'python-urlgrabber-3.10.1-1.aix6.1.noarch.rpm', 'glib2-2.14.6-2.aix5.2.ppc.rpm', 'yum-metadata-parser-1.1.4-1.aix6.1.ppc.rpm', 'python-iniparse-0.4-1.aix6.1.noarch.rpm', 'pysqlite-1.1.7-1.aix6.1.ppc.rpm'  ]
  execute "installing yum" do
    command "rpm -Uvh /var/tmp/yum/#{rpm}"
    not_if "rpm -qa | grep $(echo #{rpm} | sed 's/.aix6.1//' | sed 's/.aix5.2//' | sed 's/.rpm//')"
  end
end

# updating python
execute "updating python" do
  command "rpm -Uvh /var/tmp/yum/python-devel-2.7.10-1.aix6.1.ppc.rpm /var/tmp/yum/python-2.7.10-1.aix6.1.ppc.rpm"
  not_if "rpm -qa | grep python-2.7.10-1"
end

# installing yum
execute "installing yum" do
  command "rpm -Uvh /var/tmp/yum/yum-3.4.3-1.aix6.1.noarch.rpm"
  not_if "rpm -qa | grep yum-3.4.3.1.noarch"
end

# changing yum configuration
template '/opt/freeware/etc/yum/yum.conf' do
  source 'yum.conf.erb'
end

# installing some software with aix yum
for soft in [ 'bash', 'bzip2', 'curl', 'emacs', 'gzip', 'screen', 'vim-enhanced', 'wget', 'zlib', 'zsh', 'patch', 'file', 'lua', 'nspr', 'git' ] do
  execute "install #{soft}" do
    command "yum -y install #{soft}"
  end
end

# removing temporary file
execute 'removing /var/tmp/yum' do
  command 'rm -rf /var/tmp/yum'
  only_if { File.exists?('/var/tmp/yum')}
end

chef_yum1
chef_yum2
chef_yum3

After running the chef recipe yum is fully usable \o/ :

chef_yum4

Using yum on AIX: what you need to know

yum is usable just like it is on a Linux system. You may hit some issues when using yum on AIX. For instance you can have this kind of errors:

# yum check
AIX-rpm-7.2.0.1-2.ppc has missing requires of rpm
AIX-rpm-7.2.0.1-2.ppc has missing requires of popt
AIX-rpm-7.2.0.1-2.ppc has missing requires of file-libs
AIX-rpm-7.2.0.1-2.ppc has missing requires of nss

If you are not aware of what is the purpose of AIX-rpm please read this. This rpm is what I call a meta package. It does not install anything. This rpm is used because the rpm database does not know anything about things (binaries, libraries) installed by standard AIX filesets. By default rpm are not “aware” of what is installed by a fileset (bff) but most of rpms depends on things installed by filesets. When you install a fileset … let’s say it install a library like libc.a AIX run the updtvpkg program to rebuild this AIX-rpm and says “this rpm will resolve any rpm dependencies issue for libc.a. So first, never try to uninstall this rpm, second it’s not a real problem is this rpm has missing dependencies …. as it is providing nothing. If you really want to see what dependencies resolve AIX-rpm run the following command:

# rpm -q --provides AIX-rpm-7.2.0.1-2.ppc | grep libc.a
libc.a(aio.o)
# lslpp -w /usr/lib/libc.a
  File                                        Fileset               Type
  ----------------------------------------------------------------------------
  /usr/lib/libc.a                             bos.rte.libc          Symlink

If you want to get rid of these messages just install the missing rpm … using yum:

# yum -y install popt file-libs

A few examples

Here are a few example a software installation using yum:

  • Installing git:
  • # yum install git
    Setting up Install Process
    Resolving Dependencies
    --> Running transaction check
    ---> Package git.ppc 0:4.3.20-4 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================================================================================================================================
     Package                                    Arch                                       Version                                         Repository                                          Size
    ================================================================================================================================================================================================
    Installing:
     git                                        ppc                                        4.3.20-4                                        AIX_Toolbox                                        215 k
    
    Transaction Summary
    ================================================================================================================================================================================================
    Install       1 Package
    
    Total size: 215 k
    Installed size: 889 k
    Is this ok [y/N]: y
    Downloading Packages:
    Running Transaction Check
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
      Installing : git-4.3.20-4.ppc                                                                                                                                                             1/1
    
    Installed:
      git.ppc 0:4.3.20-4
    
    Complete!
    
  • Removing git :
  • # yum remove git
    Setting up Remove Process
    Resolving Dependencies
    --> Running transaction check
    ---> Package git.ppc 0:4.3.20-4 will be erased
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================================================================================================================================
     Package                                   Arch                                      Version                                           Repository                                          Size
    ================================================================================================================================================================================================
    Removing:
     git                                       ppc                                       4.3.20-4                                          @AIX_Toolbox                                       889 k
    
    Transaction Summary
    ================================================================================================================================================================================================
    Remove        1 Package
    
    Installed size: 889 k
    Is this ok [y/N]: y
    Downloading Packages:
    Running Transaction Check
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
      Erasing    : git-4.3.20-4.ppc                                                                                                                                                             1/1
    
    Removed:
      git.ppc 0:4.3.20-4
    
    Complete!
    
  • List available repo
  • yum repolist
    repo id                                                                                repo name                                                                                          status
    AIX_Toolbox                                                                            AIX ToolBox Repository                                                                             233
    repolist: 233
    

Getting rid of nimsh: USE HTTPS !

A new feature that is now available on latest version of AIX (7.2) allows you to use nim over http. It is a long awaited feature for different reasons (it’s just my opinion). I personally don’t like proprietary protocols such as nimsh and nimsh secure … security teams neither. Who has never experienced installation problems because of nimsh port not opened, because of ids, because of security teams ? Using http or https is the solution? No company is not allowing http or https ! This protocol is so used and secured, widely spread in a lot of products that everybody trust it. I personally prefer opening on single port than struggling opening all nimsh ports. You’ll understand that using http is far better than using nimsh. Before explaining this in details here are a few things you need to now. nimhttp is only available on latest version of AIX (7.2 SP0/1/2), same for the nimclient. If there is a problem using http the nimclient will automatically fallback in an NFS mode. Only certain nim operation are available over http:

Configuring the nim server

To use nim over http (nimhttp) you nim server must be at least deployed on an AIX 7.2 server (mine is updated to the latest service pack (SP2)). Start the service nimhttp on the nim server to allow nim to use http for its operations:

# oslevel -s
7200-00-02-1614
# startsrc -s nimhttp
0513-059 The nimhttp Subsystem has been started. Subsystem PID is 11665728.
# lssrc -a | grep nimhttp
 nimhttp                           11665728     active

The nimhttp service will listen on port 4901, this port is defined in the /etc/services :

# grep nimhttp /etc/services
nimhttp         4901/tcp
nimhttp         4901/udp
# netstat -an | grep 4901
tcp4       0      0  *.4901                 *.*                    LISTEN
# rmsock f1000e0004a483b8 tcpcb
The socket 0xf1000e0004a48008 is being held by proccess 14811568 (nimhttpd).
# ps -ef | grep 14811568
    root 14811568  4456760   0 04:03:22      -  0:02 /usr/sbin/nimhttpd -v

If you want to enable crypto/ssl to encrypt http authentication, just add the -a “-c” to your command line. This “-c” argument will tell nimhttp to start in secure mode and encrypt the authentication:

# startsrc -s nimhttp -a "-c"
0513-059 The nimhttp Subsystem has been started. Subsystem PID is 14811570.
# ps -ef | grep nimhttp
    root 14811570  4456760   0 22:57:51      -  0:00 /usr/sbin/nimhttpd -v -c

Starting the service for the first time will create an httpd.conf file in the root home directory :

# grep ^document_root ~/httpd.conf
document_root=/export/nim/
# grep ^service.log ~/httpd.conf
service.log=/var/adm/ras/nimhttp.log

If you choose to enable the secure authentication nimhttp will use the pem certificates file used by nim. If you are already using secure nimsh you don’t have to run the “nimconfig -c” command. If it is the first time this command will create the two pem files (root and server in /ssl_nim/certs) (check my blog post about secure nimsh for more information about that):

# nimconfig -c
# grep ^ssl. ~/httpd.conf
ssl.cert_authority=/ssl_nimsh/certs/root.pem
ssl.pemfile=/ssl_nimsh/certs/server.pem

The document_root of the http server will define the resource the nim http will “serve”. The default one is /export/nim (default nim place for all nim resources (spot, mksysb, lpp_source) and cannot be changed today (I think it is now ok on SP2, I’ll change the blog post as soon as the test will be done). Unfortunately for me one of my production nim was created by someone not very aware of AIX and … resources are not in /export/nim (I had to recreate my own nim because of that :-( )

On the client side ?

On the client side you just have nothing to do. If you’re using AIX 7.2 and nimhttp is enabled the client will automatically use http for communication (if it is enabled on the nim server). Just note that if you’re using nimhttp in secure mode, you must enable your nimclient in secure mode too:

# nimclient -c
Received 2788 Bytes in 0.0 Seconds
0513-044 The nimsh Subsystem was requested to stop.
0513-077 Subsystem has been changed.
0513-059 The nimsh Subsystem has been started. Subsystem PID is 13500758.
# stopsrc -s nimsh
# startsrc -s nimsh

Changing nimhttp port

You can easily change the port on which nimhttp is listening by modify the /etc/services file. Here is an example with the port 443 (I know this is not a good idea to use this one but it’s just for the example)

#nimhttp                4901/tcp
#nimhttp                4901/udp
nimhttp         443/tcp
nimhttp         443/udp
# stopsrc -s nimhttp
# startsrc -s nimhttp -a "-c"
# netstat -Aan | grep 443
f1000e00047fb3b8 tcp4       0      0  *.443                 *.*                   LISTEN
# rmsock f1000e00047fb3b8 tcpcb
The socket 0xf1000e00047fb008 is being held by proccess 14811574 (nimhttpd).

Same on the client side, just change the /etc/services file and use your nimclient as usual

# grep nimhttp /etc/services
#nimhttp                4901/tcp
#nimhttp                4901/udp
nimhttp         443/tcp
nimhttp         443/udp
# nimclient -l

To be sure I’m not using nfs anymore I’m removing any entries in my /etc/export file. I know that it will just work for some case (some type of resources) as nimesis is filling the file even if this one is empty:

# > /etc/exports
# exportfs -uav
exportfs: 1831-184 unexported /export/nim/bosinst_data/golden-vios-2233-08192014-bosinst_data
exportfs: 1831-184 unexported /export/nim/spot/golden-vios-22422-05072016-spot/usr
exportfs: 1831-184 unexported /export/nim/spot/golden-vios-22410-22012015-spot/usr
exportfs: 1831-184 unexported /export/nim/mksysb
exportfs: 1831-184 unexported /export/nim/hmc
exportfs: 1831-184 unexported /export/nim/lpp_source
[..]

Let’s do this

Let’s now try this with a simple example. I’m here installing powervp on a machine using a cust operation from the nimclient, on the client I’m doing like I have always do running the exact same command as before. Super simple:

# nimclient -o cust -a lpp_source=powervp1100-lpp_source -a filesets=powervp.rte

+-----------------------------------------------------------------------------+
                    Pre-installation Verification...
+-----------------------------------------------------------------------------+
Verifying selections...done
Verifying requisites...done
Results...

SUCCESSES
---------
  Filesets listed in this section passed pre-installation verification
  and will be installed.

  Selected Filesets
  -----------------
  powervp.rte 1.1.0.0                         # PowerVP for AIX

  << End of Success Section >>

+-----------------------------------------------------------------------------+
                   BUILDDATE Verification ...
+-----------------------------------------------------------------------------+
Verifying build dates...done
FILESET STATISTICS
------------------
    1  Selected to be installed, of which:
        1  Passed pre-installation verification
  ----
    1  Total to be installed

+-----------------------------------------------------------------------------+
                         Installing Software...
+-----------------------------------------------------------------------------+

installp: APPLYING software for:
        powervp.rte 1.1.0.0

0513-071 The syslet Subsystem has been added.
Finished processing all filesets.  (Total time:  4 secs).

+-----------------------------------------------------------------------------+
                                Summaries:
+-----------------------------------------------------------------------------+

Installation Summary
--------------------
Name                        Level           Part        Event       Result
-------------------------------------------------------------------------------
powervp.rte                 1.1.0.0         USR         APPLY       SUCCESS
powervp.rte                 1.1.0.0         ROOT        APPLY       SUCCESS

On the server side I’m checking the /var/adm/ras/nimhttp.log (log file for nimhttp) and I can check that files are transferred from the server to the client using the http protocol. So it works great.

# Thu Jul 21 23:44:19 2016        Request Type is GET
Thu Jul 21 23:44:19 2016        Mime not supported
Thu Jul 21 23:44:19 2016        Sending Response Header "200 OK"
Thu Jul 21 23:44:19 2016        Sending file over socket 6. Expected length is 600
Thu Jul 21 23:44:19 2016        Total length sent is 600
Thu Jul 21 23:44:19 2016        handle_httpGET: Entering cleanup statement
Thu Jul 21 23:44:20 2016        nim_http: queue socket create product (memory *)200739e8
Thu Jul 21 23:44:20 2016        nim_http: 200739e8 6 200947e8 20098138
Thu Jul 21 23:44:20 2016        nim_http: file descriptor is 6
Thu Jul 21 23:44:20 2016        nim_buffer: (resize) buffer size is 0
Thu Jul 21 23:44:20 2016        file descriptor is : 6
Thu Jul 21 23:44:20 2016        family is : 2 (AF_INET)
Thu Jul 21 23:44:20 2016        source address is : 10.14.33.253
Thu Jul 21 23:44:20 2016        socks: Removing socksObject 2ff1ec80
Thu Jul 21 23:44:20 2016        socks: 200739e8 132 <- 87 bytes (SSL)
Thu Jul 21 23:44:20 2016        nim_buffer: (append) len is 87, buffer length is 87
Thu Jul 21 23:44:20 2016        nim_http: data string passed to get_http_request: "GET /export/nim/lpp_source/powervp/powervp.1.1.0.0.bff HTTP/1.1

Let's do the same thing with a fileset coming from a bigger lpp_source (in fact an simage one for the latest release of AIX 7.2):

# nimclient -o cust -a lpp_source=7200-00-02-1614-lpp_source -a filesets=bos.loc.utf.en_KE
[..]

Looking on the nim server I notice that files are transfered from the server to the client, but NOT my fileset and it's dependencies .... but the whole lpp_source (seriously ? uh ? why ?)

# tail -f /var/adm/ras/nimhttp.log
Thu Jul 21 23:28:39 2016        Request Type is GET
Thu Jul 21 23:28:39 2016        Mime not supported
Thu Jul 21 23:28:39 2016        Sending Response Header "200 OK"
Thu Jul 21 23:28:39 2016        Sending file over socket 6. Expected length is 4482048
Thu Jul 21 23:28:39 2016        Total length sent is 4482048
Thu Jul 21 23:28:39 2016        handle_httpGET: Entering cleanup statement
Thu Jul 21 23:28:39 2016        nim_http: queue socket create product (memory *)200739e8
Thu Jul 21 23:28:39 2016        nim_http: 200739e8 6 200947e8 20098138
Thu Jul 21 23:28:39 2016        nim_http: file descriptor is 6
Thu Jul 21 23:28:39 2016        nim_buffer: (resize) buffer size is 0
Thu Jul 21 23:28:39 2016        file descriptor is : 6
Thu Jul 21 23:28:39 2016        family is : 2 (AF_INET)
Thu Jul 21 23:28:39 2016        source address is : 10.14.33.253
Thu Jul 21 23:28:39 2016        socks: Removing socksObject 2ff1ec80
Thu Jul 21 23:28:39 2016        socks: 200739e8 132 <- 106 bytes (SSL)
Thu Jul 21 23:28:39 2016        nim_buffer: (append) len is 106, buffer length is 106
Thu Jul 21 23:28:39 2016        nim_http: data string passed to get_http_request: "GET /export/nim/lpp_source/7200-00-02-1614/installp/ppc/X11.fnt.7.2.0.0.I HTTP/1.1

If you have a deeper look of what is nimclient doing when using nimhttp .... he is just transfering the whole lpp_source from the server to the client and then installing the needed fileset from a local filesystem. Filesets are storred into /tmp so be sure you have a /tmp bigger enough to store your biggest lpp_source. Maybe this will be changed in the future but it is like it is for the moment :-) . The nimclient is creating temporary directory named (prefix) "_nim_dir_" to store the lpp_source:

root@nim_server:/export/nim/lpp_source/7200-00-02-1614/installp/ppc# du -sm .
7179.57 .
root@nim_client:/tmp/_nim_dir_5964094/export/nim/lpp_source/7200-00-02-1614/installp/ppc# du -sm .
7179.74 .

More details ?

You can notice while running a cust operation from the nim client that nimhttp is also running in background (on the client itself). The truth is that the nimhttp binary running on client act as an http client. In the output below the http client is getting the file Java8_64.samples.jnlp.8.0.0.120.U and

# ps -ef |grep nim
    root  3342790 16253432   6 23:29:10  pts/0  0:00 /bin/ksh /usr/lpp/bos.sysmgt/nim/methods/c_installp -afilesets=bos.loc.utf.en_KE -alpp_source=s00va9932137:/export/nim/lpp_source/7200-00-02-1614
    root  6291880 13893926   0 23:29:10  pts/0  0:00 /bin/ksh /usr/lpp/bos.sysmgt/nim/methods/c_script -alocation=s00va9932137:/export/nim/scripts/s00va9954403.script
    root 12190194  3342790  11 23:30:06  pts/0  0:00 /usr/sbin/nimhttp -f /export/nim/lpp_source/7200-00-02-1614/installp/ppc/Java8_64.samples.jnlp.8.0.0.120.U -odest -s
    root 13500758  4325730   0 23:23:29      -  0:00 /usr/sbin/nimsh -s -c
    root 13893926 15991202   0 23:29:10  pts/0  0:00 /bin/ksh -c /var/adm/nim/15991202/nc.1469222947
    root 15991202 16974092   0 23:29:07  pts/0  0:00 nimclient -o cust -a lpp_source=7200-00-02-1614-lpp_source -a filesets=bos.loc.utf.en_KE
    root 16253432  6291880   0 23:29:10  pts/0  0:00 /bin/ksh /tmp/_nim_dir_6291880/script

You can use the nimhttp as a client to download file directly from the nim server. Here I'm just listing the content of /export/nim/lpp_source from the client

# nimhttp -f /export/nim/lpp_source -o dest=/tmp -v
nimhttp: (source)       /export/nim/lpp_source
nimhttp: (dest_dir)     /tmp
nimhttp: (verbose)      debug
nimhttp: (master_ip)    nimserver
nimhttp: (master_port)  4901

sending to master...
size= 59
pull_request= "GET /export/nim/lpp_source HTTP/1.1
Connection: close

"
Writing 1697 bytes of data to /tmp/export/nim/lpp_source/.content
Total size of datalen is 1697. Content_length size is 1697.
# cat /tmp/export/nim/lpp_source/.content
DIR: 71-04-02-1614 0:0 00240755 256
DIR: 7100-03-00-0000 0:0 00240755 256
DIR: 7100-03-01-1341 0:0 00240755 256
DIR: 7100-03-02-1412 0:0 00240755 256
DIR: 7100-03-03-1415 0:0 00240755 256
DIR: 7100-03-04-1441 0:0 00240755 256
DIR: 7100-03-05-1524 0:0 00240755 256
DIR: 7100-04-00-1543 0:0 00240755 256
DIR: 7100-04-01-1543 0:0 00240755 256
DIR: 7200-00-00-0000 0:0 00240755 256
DIR: 7200-00-01-1543 0:0 00240755 256
DIR: 7200-00-02-1614 0:0 00240755 256
FILE: MH01609.iso 0:0 00100644 1520027648
FILE: aixtools.python.2.7.11.4.I 0:0 00100644 50140160

Here I'm just downloading a python fileset !

# nimhttp -f /export/nim/lpp_source/aixtools.python.2.7.11.4.I -o dest=/tmp -v
[..]
Writing 65536 bytes of data to /tmp/export/nim/lpp_source/aixtools.python.2.7.11.4.I
Writing 69344 bytes of data to /tmp/export/nim/lpp_source/aixtools.python.2.7.11.4.I
Writing 7776 bytes of data to /tmp/export/nim/lpp_source/aixtools.python.2.7.11.4.I
Total size of datalen is 50140160. Content_length size is 50140160.
# ls -l /tmp/export/nim/lpp_source/aixtools.python.2.7.11.4.I
-rw-r--r--    1 root     system     50140160 Jul 23 01:21 /tmp/export/nim/lpp_source/aixtools.python.2.7.11.4.I

Allowed operation

All cust operations on nim objects type lpp_source, installp_bundle, fix_bundle, scripts, and file_res in push or pull are working great with nimhttp. Here are a few examples (from the official doc, thanks to Paul F for that ;-) ) :

  • Push:
  • # nim –o cust –a file_res=obj_name client_obj_name
    # nim –o cust –a script=obj_name client_obj_name
    # nim –o cust –a lpp_source=obj_name -a filesets=fileset names to install client_obj_name
    # nim –o cust –a lpp_source=obj_name -a installp_bundle=obj_name client_obj_name
    # nim –o cust –a lpp_source=obj_name ‐a fixes=update_all client_obj_name
    
  • Pull:
  • # nimclient -o cust -a lpp_source=obj_name -a filesets=fileset names to install
    # nimclient –o cust –a file_res=obj_name
    # nimclient –o cust –a script=obj_name nimclient –o cust –a lpp_source=obj_name -‐a filesets=fileset names to install
    # nimclient –o cust –a lpp_source=obj_name -a installp_bundle=obj_name
    # nimclient –o cust –a lpp_source=obj_name -a fixes=update
    

Proxying: use your own http server

You can use you own webserver to host nimhttp and the nimhttp binary will just act as a proxy between your client and you http server. I have tried to do it but didn't succeed with that I'll let you know if I'm finding the solution:

# grep ^proxt ~/httpd.conf
service.proxy_port=80
enable_proxy=yes

Conclusion: "about administration and post-installation"

Just a few words about best practices of post-installation and administration on AIX. On on the major purpose of this blog post is to prove to you than you need to get rid of an old way of working. The first thing to do is always to try using http or https instead of NFS. To give you an example of that I'm always using http to transfer my files whatever it is (configuration, product installation and so on ...). With an automation tool such as Chef it is so simple to integrate the download of a file from an http server that you must now avoid using NFS ;-) . Second good practice is to never install things "by hand" and using yum is one of the reflex you need to have instead of using the rpm command (Linux users will laugh reading that ... I'm laughing writing that, using yum is just something I'm doing for more than 10 years ... but for AIX admins it's still not the case and not so simple to understand :-) ). As always I hope it helps.

About blogging

I just wanted to say one word about blogging because I got a lot of questions about this (from friends, readers, managers, haters, lovers). I'm doing this for two reasons. The first one is that writing and explaining things force me to better understand what I'm doing and force me to always discover new features, new bugs, new everything. Second I'm doing this for you, for my readers because I remember how blogs were useful to me when I began AIX (Chris and Nigel are the best example of that). I don't care about being the best or the worst. I'm just me. I'm doing this because I love that that's all. Even if manager, recruiters or anybody else don't care about it I'll continue to do this whatever appends. I agree with them "It does not prove anything at all". I'm just like you a standard admin trying to do his job at his best. Sorry for the two months "break" about blogging but it was really crazy at work and in my life. Take care all. Haters gonna hate.