Hardware Management Console : Be autonomous with upgrade, update, and use the Integrated Management Module

I’m sure that like me you do not have a physical access to your Hardware Management Consoles, or even if you have this access, some of your HMC are so far away from your working site (even in a foreign country) that you can’t afford to physically move to this place to update it. Even worse if -like me- you are working in a big place (who says too big ?) this job is often performed by IBM Inspectors and you do not have to worry about your Hardware Management Consoles and just have to ask IBM guys for anything about HMC. For some reasons I had to update an old Hardware Management Console from v7r7.3.0 to v7r7.7.0 SP3. Everybody is confused about the differences between updating an HMC and upgrading an HMC. I know really good bloggers : Anthony English and Rob McNelly have already post about this particular subject but I have to write this post as a reminder and to clarify some points which are not tell in Anthony’s and Rob’s posts :

To finish this post I’ll talk about a feature nobody is using : the HMC is coming with an Integrated Management Module, this one allows you to have more control and to be autonomous with you HMC.

The difference between updating, upgrading and migrating

There is a lot of confusion when people are trying “update” their HMC. When do I have to update using the updhmc command, when do I have to upgrade using saveupgdata, getupdfiles and chhmc command, and finally when do I have to migrate using HMC Recovery CD/DVD ? All of this three operations are not well described by IBM. Here is what I’m doing in for each case, and this is the result of my own experience (don’t take this as an official document). Here is a little reminder it can be useful for a lot of people : an HMC version number looks like this : v7r7.7.0 SP3. v7 is the VERSION. 7.7.0 is the RELEASE and SP3 is the SERVICE PACK.

  • Updating : You have to update your HMC if your are applying a service pack, or a corrective fix update on the HMC, this operation can only be performed by the updhmc command. Use this method if fix central gives you an iso named “HMC_Update_*.iso” or a zip files named “MHxxxxx.zip”. These fixes can be applied to a minor version of the HMC.
  • Upgrading : You have to upgrade your HMC if your are moving from one minor version to another (from one release to another), for instance if your are moving from v7r7.7.0 to v7r7.8.0, this operation can be made by using HMC Recovery DVD (fix central will gives you two isos named “HMC_Recovery_*.iso”), or by using the images provided for a network upgrade (I’ll explain this in this post).
  • Migrating : You have to migrate your HMC by using HMC Recovery DVD when your are moving from one major version to another. For example when your are moving from an HMC v6 to and HMC v7 (for instance from any v6 version to v7r7.8.0). In this case you have no other choice than burning DVDs and moving in front of the HMC to perform the operation by yourself.

Upgrading

You can upgrade your HMC from its local storage by using the network images provided by IBM on a public FTP server, once connected to the public FTP server get the version you want to upgrade to and download all the files, bzImage and initrd.gz included :

# ftp://ftp.software.ibm.com/software/server/hmc/network/v7770/
# ls
FTP Listing of /software/server/hmc/network/v7770/ at ftp.software.ibm.com
[..]
Feb 26 2013 00:00      2708320 bzImage
Feb 26 2013 00:00    808497152 disk1.img
Feb 26 2013 00:00   1142493184 disk2.img
Feb 26 2013 00:00   1205121024 disk3.img
Feb 26 2013 00:00           78 hmcnetworkfiles.sum
Feb 26 2013 00:00     34160044 initrd.gz
# mget *.*

Put all the files on a server where you have an FTP server running (the HMC getupgfiles is using FTP to get the files) and download all the files with the getupgfiles command directly form the HMC (if your HMC has a direct access to the internet you can specify it the command):

hscroot@gaff:~> getupgfiles -h 192.168.0.99 -u root -d /export/HMC/network_ugrade/v7770
Enter the current password for user root:

While images are downloading the HMC is mounting a temporary filesystem called /hmcdump and put the images in it. Once the images are downloaded the filesystem /hmcdump is unmounted. You can check the download progression with a loop looking on the /hmcdump filesystem :

hscroot@gaff:~>  while true ; do date; ls -la /hmcdump; sleep 60; done
[..]
drwxr-xr-x  3 root root      4096 2013-12-24 16:26 .
drwxr-xr-x 30 root root      4096 2013-12-19 14:52 ..
-rw-r--r--  1 root hmc  824223312 2013-12-24 16:32 disk3.img
-rw-r--r--  1 root hmc         78 2013-12-24 16:26 hmcnetworkfiles.sum
drwx------  2 root root     16384 2007-12-19 03:24 lost+found
Tue Apr  1 08:10:30 CEST 2014
total 3121248
drwxr-xr-x  3 root root       4096 2013-12-24 16:52 .
drwxr-xr-x 30 root root       4096 2013-12-19 14:52 ..
-rw-r--r--  1 root hmc     2708320 2013-12-24 16:52 bzImage
-rw-r--r--  1 root hmc   808497152 2013-12-24 16:52 disk1.img
-rw-r--r--  1 root hmc  1142493184 2013-12-24 16:45 disk2.img
-rw-r--r--  1 root hmc  1205121024 2013-12-24 16:36 disk3.img
-rw-r--r--  1 root hmc          78 2013-12-24 16:26 hmcnetworkfiles.sum
-rw-r--r--  1 root hmc    34160044 2013-12-24 16:52 initrd.gz
drwx------  2 root root      16384 2007-12-19 03:24 lost+found

Please note that this filesystem is only mounted while the getupgfile command is running and can’t be mounted after the command execution … :

hscroot@gaff:~> mount /hmcdump
mount: only root can mount /dev/sda6 on /hmcdump

Before launching the upgrade save all the data needed for the upgrade to disk, close all HMC events and clear all the filesystems :

  • Save all HMC upgrade data to disk. This command is MANDATORY, it save all the partition profile data, and the user data and the whole HMC configuration, if you forget this command you have to reconfigure the HMC by hand, so be careful with this one :-) :
  • hscroot@gaff:~> saveupgdata -r disk
    
  • Close all HMC events :
  • hscroot@gaff:~> chsvcevent -o closeall
    
  • Remove all temporary HMC files from all filesystems :
  • hscroot@gaff:~> chhmcfs -o f -d 0
    

The images are now downloaded to the HMC, to upgrade the HMC you just have to tell the HMC to boot on its alternate disk and to use the files you’ve just download for the upgrade :

  • To set the alternate disk partition on the HMC as a startup device on the next HMC boot and enable the upgrade on the alternate disk use the chhmc command :
  • hscroot@gaff:~> chhmc -c altdiskboot -s enable --mode upgrade
    
  • Before rebooting check the altdiskboot attribute is set to enable :
  • hscroot@gaff:~> lshmc -r
    ssh=enable,sshprotocol=,remotewebui=enable,xntp=enable,"xntpserver=127.127.1.0,kronosnet1.fr.net.intra,kronosnet2.fr.net.intra,kronosnet3.fr.net.intra",syslogserver=,netboot=disable,altdiskboot=enable,ldap=enable,kerberos=disable,kerberos_default_realm=,kerberos_realm_kdc=,kerberos_clockskew=,kerberos_ticket_lifetime=,kerberos_keyfile_present=,"sol=disabled
    "
    
  • Reboot the HMC and wait :-) :
  • hscroot@gaff:~> hmcshutdown -t now -r
    

Depending on the HMC model and on the version of the HMC the upgrade can takes 10 minutes to 40 minutes, you’ll have to be patient and to cross your finger and pray everything is going well. But don’t worry I never had an issue with this method. Once the HMC is rebooted and upgraded, you can check that the altdiskboot attribute is now set to disable :

hscroot@gaff:~> lshmc -r
ssh=enable,sshprotocol=,remotewebui=enable,xntp=enable,"xntpserver=127.127.1.0,kronosnet1.fr.net.intra,kronosnet2.fr.net.intra,kronosnet3.fr.net.intra",syslogserver=,syslogtcpserver=,syslogtlsserver=,netboot=disable,altdiskboot=disable,ldap=enable,kerberos=disable,kerberos_default_realm=,kerberos_realm_kdc=,kerberos_clockskew=,kerberos_ticket_lifetime=,kpasswd_admin=,trace=,kerberos_keyfile_present=,legacyhmccomm=enable,sol=disabled

Updating

Once the HMC is upgraded you have to update it. Unfortunately updates files (often ISO files) are only available on fix central and not on the public FTP. Get the ISO updates file from fix central and put it on your FTP (once again) server, then use the updhmc command to update the HMC, repeat the operation for each updates, and then reboot the HMC (in the example below I’m using sftp) :

hscroot@gaff:~> updhmc -t s -i -h 192.168.0.99 -u root -f /export/HMC/v7r770/HMC_Update_V7R770_SP1.iso
Password:
iptables: Chain already exists.
ip6tables: Chain already exists.
[..]
The corrective service file was successfully applied. A mandatory reboot is required but was not specified on the command syntax.
hscroot@gaff:~> updhmc -t s -i -h 192.168.0.99 -u root -f /export/HMC/v7r770/HMC_Update_V7R770_SP2.iso
Password:

ip6tables: Chain already exists.
ACCEPT  tcp opt -- in eth3 out *  0.0.0.0/0  -> 0.0.0.0/0  tcp dpt:5989
ACCEPT  udp opt -- in eth3 out *  0.0.0.0/0  -> 0.0.0.0/0  udp dpt:657
[..]
The corrective service file was successfully applied. A mandatory reboot is required but was not specified on the command syntax.
hscroot@gaff:~> hmcshutdown -t now -r

After upgrading and updating the HMC check the version is ok with the lshmc command :

hscroot@gaff:~> lshmc -V
"version= Version: 7
 Release: 7.7.0
 Service Pack: 3
HMC Build level 20131113.1
","base_version=V7R7.7.0
"

Using and configuring the Integrated Management Module

I like to be autonomous and do things on my own. Who had never been stuck on a problem with an HMC and was forced to called an IBM inspector to reboot the HMC or even to insert a CD in the CDRom reader. A few people know this but the HMC is based on an IBM Xserie server (Who said Lenovo ?) and is shipped with an Integrated Management Module allowing you to boot, start, and stop the HMC without the need to have someone in the data-center. Unfortunately this method seems not to be supported by IBM so do it at your own risk.

Use the dedicated port for the Integrated Management Console (the Red port)

hmc_imm_port_good

From the HMC command line using the chhmc command configure the Integrated Management Module IP address :

hscroot@gaff:~> chhmc -c imm -s modify -a 10.10.20.4 -nm 255.255.255.0 -g 10.10.20.254

Restart the Integrated Management Module to commit the changes. The IMM will not be pingable before restart :

hscroot@gaff:~> chhmc -c imm -s restart

The Integrated Management Module is now pingable and you can check its configuration :

hscroot@gaff:~> lshmc -i
ipv4addr=10.10.20.4,networkmask=255.255.255.0,gateway=10.10.20.254,username=USERID,mode=Dedicated

By default the username is USERID and the password is PASSW0RD (with a zero), you can change it to fit your needs :

hscroot@gaff:~> chhmc -c imm -s modify -u immusername --passwd "abc123"

The Integrated Management Module is now configured and can be accessed from the web interface of from SSH :

hmc_imm_login

I will not detail all the actions you can do with the Integrated Management Module but here is a screen showing the Hardware Health of the HMC :

hmc_imm_hardware

One thing you can do for free (without IMM license) is to control the Power of the HMC, choosing to stop/start/restart or reboot. This feature can be very useful when the HMC is stucked :

hmc_imm_actions

If you choose to restart the HMC the Integrated Management Module will warn you before restarting :

hmc_imm_restart

You can access the HMC Integrated Management Module by using the SSH command line :

  • Use the power command to control the power of the HMC :
  • system> help power
    usage:
       power on    [-options]   - power on server
       power off   [-options]   - power off server
       power cycle [-options]   - power off, then on
       power state              - display power state
       power -rp [alwayson|alwaysoff|restore]   - host power restore policy
    options:
       -s                       - shut down OS first
       -every day               - daily or weekly on,off or cycle commands
    [Sun|Mon|Tue|Wed|Thu|Fri|Sat|Day|clear]
       -t   time                - time (hh:mm)
    additional options for on.
       -d  date                 - date (mm/dd/yyyy)
       -clear                   - clear on date
    
  • Here is an example to restart the HMC :
  • system> power cycle -s
    ok
    
  • Checking the power state :
  • system> power state
    power on
    State:Booting OS or in unsupported OS
    

The Integrated Management Module is a licensed product and unfortunately IBM does not support the Integrated Management Module on the HMC.It seems that the IMM license can’t be acquired for the HMC. I have checked on the trial licenses page and the HMC Hardware does not even exists when you have to choose the Hardware model for the trial license. This a shame because the licensed IMM allows to remote control the HMC, and to manage Virtual CDrom ….. useful for migration. So if an IBMer is reading this and have an explanation about this feel free to tell me what I’ve missed in the comments :

hmc_imm_remote_control

I hope this post will let you manage your HMC alone and to be autonomous :-)

PowerVC Express using local storage : Overview, tips and tricks, and lessons learned from experience

Everybody talks about PowerVC since the October 8th announcement, after seeing a few videos and reading a few articles about it, I didn’t find anything telling what’s the product really has in his guts. I had the chance to deploy and test a PowerVC express version (using local storage), faced a lot of problems and found some interesting things to share with you. Rather than boiling the ocean :-) and asking for new features (oh everybody wants new features !), here is a practical how-to, some tips and tricks and the lessons I’ve learned about it. After a few weeks of work I can say that PowerVC is really good and pretty simple to use and deploy. Here we go :

Preparing the PowerVC express host

Setting SELinux from enforcing to permissive

cool1

Please refer to my previous about installing a Linux On Power if you have any doubt about this. Before trying to install PowerVC express edition you first have to disable selinux or at least set the policy form enforcing to permissive. Please note that a mandatory reboot is needed for this modification :

# sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/selinux/config
# grep ^SELINUX /etc/selinux/config
SELINUX=permissive
SELINUXTYPE=targeted

Reboot the PowerVC express host :

[root@powervc-standard ~]# shutdown -fh now
Broadcast message from root@powervc-standard
(/dev/pts/0) at 15:43 ...
The system is going down for halt NOW!

Yum repository

cool2

Before running the installer you have to configure your yum repository because the installer needs to install rpm shipped with the Red Hat Enterprise installation cdrom. I choose to use the cdrom as repository but this one can be served through http, without any problems :

# mkdir /mnt/cdrom ; mount -o loop /dev/cdrom /mnt/cdrom
# cat /etc/yum.repos.d/rhel-cdrom.repo
[rhel-cdrom]
name=RHEL Cdrom
baseurl=file:///mnt/cdrom
gpgckeck=0
enabled=1
# yum update
# yum upgrade

If using x86 version : noop scheduler

If you are using the x86 version of PowerVC express you can experience some slowness while trying to install the product. In my case I had to change the I/O scheduler from cfq to noop. My advice is just to temporarily enable it. My installation of PowerVC express takes hours (no joke, almost 5 hours) before changing the I/O scheduler to noop. Enabling this option reduce this time to an half hour (in my case) :

# cat /sys/block/vda/queue/scheduler
noop anticipatory deadline [cfq]
# echo "noop" > /sys/block/vda/queue/scheduler
# cat /sys/block/vda/queue/scheduler
[noop] anticipatory deadline cfq

PATH modification

Add the /opt/ibm/powervc/bin to your path to be allowed to run PowerVC commands such as powervc-console-term, powervc-services, powervc-get-token, and so on …..

# more /root/.bash_profile
PATH=$PATH:$HOME/bin:/opt/ibm/powervc/bin

I’ll not detail the installation here but just run this installer and follow the questions asked by the installer :

# ./install
Select the offering type to install:
   1 - Express  (IVM support)
   2 - Standard (HMC support)
   9 - Exit
1
Extracting license content
International Program License Agreement
[..]

Preparing the Virtual I/O Server and the IVM

Before trying to do anything you have to configure the Virtual I/O Server and the IVM, check that all the points below are ok before registering the host :

  • You need at least one Shared Ethernet Adapter to use PowerVC express, you can on an IVM have up to four Shared Ethernet Adapter.
  • A virtual media repository created with at least 40Gb free.
  • # lsrep
    Size(mb) Free(mb) Parent Pool         Parent Size      Parent Free
       40795    29317 rootvg                   419328           342272
    
  • A PowerVM enterprise edition or PowerVM for IBM PowerLinux key.
  • The max virtual adapter correctly configured (in my case 256).
  • The maximum number of ssh sessions opened on the Virtual I/O Server has to be at least 20.
  • # grep MaxSessions /etc/ssh/sshd_config
    MaxSessions 20
    # stopsrc -s sshd
    # startsrc -s sshd
    
  • FTP transfers and FTP ports are opened between the PowerVC host and the IVM.

PowerVC usage

Host Registering

It’s very easy but registering the host is one of the most important step in this configuration. Just set your IVM hostname user and password. The tricky part is to check the box to use local storage, you then have to choose the directory where images will be stored. Be careful when choosing this directory, it can’t be changed on the fly, and you have to remove and re-register the host if you want to do this. My advice is not to choose the default /home/padmin directory, and to create a dedicated logical volume for this.

deckard

If the host registration fails check all the Virtual I/O Server prerequisites, then retry. If it fails again check the /var/log/nova/api.log and /var/log/nova/compute_xxx.log

host1

Manage existing Virtual Machines

Unlike VMcontrol PowerVC allows to manage existing machines, so if your IVM is correctly configured, you’ll not have trouble to import existing machines, and manage them with PowerVC. This is one of the strength of PowerVC, it assure a backward compatibility for your existing Virtual I/O Clients. And it’s simple to use (look at the images below) :

manage_existing
manage_existing1

Network Definition

Create a network using one of your Shared Ethernet Adapter to be able to deploy machines :

network_1

First installation with ISO image

Importing ISO images

For the first installation (if you do not have any systems already installed on your system) you first need to import an iso to PowerVC, be very careful to read the next steps because I had a lot problems of space with this. Importing images is managed by glance, so if you have any problem checking in /var/log/glance file can be useful (putting in verbose mode too in /etc/glance/glance.conf). Just use the powervc-iso-import command to do so :

# powervc-iso-import --name aix-7100-02-02 --os-distro aix --location /root/AIX_7.1_Base_Operating_System_TL_7100-02-02_DVD_1_of_2_32013.iso 
Password: 
+----------------------------+--------------------------------------+
| Property                   | Value                                |
+----------------------------+--------------------------------------+
| Property 'architecture'    | ppc64                                |
| Property 'hypervisor_type' | powervm                              |
| Property 'os_distro'       | aix                                  |
| checksum                   | df548a0cc24dbec196d0d3ead92feaca     |
| container_format           | bare                                 |
| created_at                 | 2014-02-04T19:45:29.125109           |
| deleted                    | False                                |
| deleted_at                 | None                                 |
| disk_format                | iso                                  |
| id                         | ee0a6544-c065-4ab7-aec8-7d6ee4248672 |
| is_public                  | True                                 |
| min_disk                   | 0                                    |
| min_ram                    | 0                                    |
| name                       | aix-7100-02-02                       |
| owner                      | 437b161186414e2bb0d4778cbd6fa14c     |
| protected                  | False                                |
| size                       | 3835723776                           |
| status                     | active                               |
| updated_at                 | 2014-02-04T19:49:29.031481           |
+----------------------------+--------------------------------------+

importing_iso

The result of the command above do not tell you anything about what is really done by the command.

Images are stored forever in /var/lib/glance/images and copied here by powerc-iso-import, this is the place where you need to have free space, don’t forget to remove you source image from the PowerVC host or you’ll need to have more space (in fact double space :-) ). Check the /var/lib/glance/images while running powervc-iso-import shows you that images are copied :

# ls -lh /var/lib/glance/images
total 3.3G
-rw-r-----. 1 glance glance 3.3G Feb  4 22:08 3b95401b-85b4-4682-a7a5-332ea9e48348
# ls -lh /var/lib/glance/images
total 3.4G
-rw-r-----. 1 glance glance 3.4G Feb  4 22:09 3b95401b-85b4-4682-a7a5-332ea9e48348

image_import2

Deploying a Virtual Machine with ISO image :

Be careful when deploying images to have enough space in /home/padmin directory of the Virtual I/O Server : images are first copied to this directory before being available on the Virtual I/O Server media repository in /var/vio/VMLibrary (they are -apparently- removed later). On the PowerVC host itself, be careful to have enough space in /var/lib/nova/images and /var/lib/glance/images. On the PowerVC host images are stored by glance PowerVC so DON’T DELETE IMAGES in /var/lib/glance/images ! My understanding of this is that images are copied on fly from glance (/var/lib/images/glances) where images are stored (by powervc-import-iso), to nova (/var/lib/nova/images) where images are copied and then sent to the Virtual I/O Server, and then added to the Virtual I/O Server repository. PowerVC is using ftp to copy files to the Virtual I/O Server, so be sure to have ports open between PowerVC host and the Virtual I/O Server.

  • Here is an exemple of iso file present in /home/padmin on the Virtual I/O Server when deploying a server with an image, below we can see that image was copied in /var/lib/nova/images before beeing copied on the Virtual I/O Server :
  • padmin@deckard# ls *.iso
    config                                                  rhel-server-6.4-beta-ppc64-dvd.iso                      smit.script
    89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e.192_168_0_100.iso  rhel-server-ppc-6.4-boot.iso                            smit.transaction
    ioscli.log                                              smit.log                                                tivoli
    [root@powervc-express ~]# ls -l /var/lib/nova/images/
    total 4579236
    -rw-r--r--. 1 nova nova 4689133568 Feb  4 22:31 89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e.192_168_0_100.iso
    
  • Once images are copied from PowerVC they are imported to the Virtual I/O Server repository :
  • padmin@deckard# ps -ef | grep mkvopt
      padmin  6422716  8519802   0 05:41:42      -  0:00 ioscli mkvopt -name 89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e -file /home/padmin/89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e.192_168_0_100.iso -ro
      padmin  8519802 10485798   0 05:41:42      -  0:00 rksh -c ioscli mkvopt -name 89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e -file /home/padmin/89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e.192_168_0_100.iso -ro;echo $?
      padmin 10158232  9699504   2 05:42:30  pts/0  0:00 grep mkvopt
    
  • The Virtual Optical Device is then used to load the CDROM to the partition :
  • padmin@deckard# lsmap -all
    SVSA            Physloc                                      Client Partition ID
    --------------- -------------------------------------------- ------------------
    vhost0          U8203.E4A.06E7E53-V1-C11                     0x00000002
    
    VTD                   vtopt0
    Status                Available
    LUN                   0x8200000000000000
    Backing device        /var/vio/VMLibrary/89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e
    Physloc               
    Mirrored              N/A
    
    VTD                   vtscsi0
    Status                Available
    LUN                   0x8100000000000000
    Backing device        lv00
    Physloc               
    Mirrored              N/A
    
    padmin@deckard# lsrep
    Size(mb) Free(mb) Parent Pool         Parent Size      Parent Free
       40795    29317 rootvg                   419328           342272
    
    Name                                                  File Size Optical         Access 
    89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e                       4472 vtopt0          ro     
    fa9b3cf0-a649-4bf0-b309-5f2bab6379ea                       3659 None            ro     
    rhel-server-ppc-6.4-boot.iso                                227 None            ro     
    rhel-server-ppc-6.4.iso                                    3120 None            ro     
    
    

When deploying an iso image after all the steps below are finished the Virtual Machine newly created will be in shutoff state :

shutoff_before_start

Run the console term before starting the Virtual Machine, then start the Virtual Machine (by PowerVC) :

# powervc-console-term tyler61
Password: 
Starting terminal.
  • When deploying multiple hosts with the same image it is possible that some virtual machines will have the same name; in this case the powervc-console-term will warn you :
  • # powervc-console-term --f mary
    Password: 
    Multiple servers were found with the same name. Specify the server ID.
    089ecbc5-5bed-4d06-8659-bf7c57529c95 mary
    231ad074-7557-42b5-82b9-82ae2483fccd mary
    powervc-console-term --f 089ecbc5-5bed-4d06-8659-bf7c57529c95
    
    padmin@deckard# ps -ef | grep -i mkvt
      padmin  2556048  8323232   0 06:39:15  pts/1  0:00 rksh -c ioscli rmvt -id 2 && ioscli mkvt -id 2 && exit
    

    starting_first

    Then follow the instruction on the screen to finish these first installation (like if you were installing an AIX form the cdrom):

    IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
    IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
    -
    Elapsed time since release of system processors: 151 mins 48 secs
    /
    Elapsed time since release of system processors: 151 mins 57 secs
    -------------------------------------------------------------------------------
                                    Welcome to AIX.
                       boot image timestamp: 18:20:10 02/04/2013
                     The current time and date: 05:00:31 02/05/2014
            processor count: 1;  memory size: 2048MB;  kernel size: 29153194
    boot device: /vdevice/v-scsi@30000002/disk@8200000000000000:\ppc\chrp\bootfile.exe
                           kernel debugger setting: enabled
    -------------------------------------------------------------------------------
    
    AIX Version 6.1
    
    

    Preparing the capture of the first installed Virtual Machine

    The Activation Engine

    Before capturing the Virtual machine run the Activation Engine. The script will allow the machine to be captured and the captured image to be automatically reconfigured on the fly after first boot time. Be careful when running the Activation Engine, the Virtual Machine will be shut-off just by running this script.

    # scp 192.168.0.98:/opt/ibm/powervc/activation-engine/vmc.vsae.tar .
    # tar xvf vmc.vsae.tar
    x activation-engine-2.2-106.aix5.3.noarch.rpm, 1240014 bytes, 2422 media blocks.
    [..]
    x aix-install.sh, 2681 bytes, 6 media blocks.
    # rm /opt/ibm/ae/AP/*
    # cp /opt/ibm/ae/AS/vmc-network-restore/resetenv /opt/ibm/ae/AP/ovf-env.xml
    # JAVA_HOME=/usr/java5/jre
    # ./aix-install.sh
    Install VSAE and VMC extensions
    package activation-engine-jython-2.2-106 is already installed
    package activation-engine-2.2-106 is already installed
    package vmc-vsae-ext-2.4.4-1 is already installed
    # /opt/ibm/ae/AE.sh --reset
    JAVA_HOME=/usr/java5/jre
    [..]
    [2014-02-04 23:49:51,980] INFO: OS: AIX Version: 6
    [..]
    [2014-02-04 23:51:20,095] INFO: Cleaning AR and AP directories
    [2014-02-04 23:51:20,125] INFO: Shutting down the system
    
    SHUTDOWN PROGRAM
    Tue Feb  4 23:51:21 CST 2014
    
    
    Broadcast message from root@tyler (tty) at 23:51:21 ... 
    
    shutdown: PLEASE LOG OFF NOW !!!
    System maintenance is in progress.
    All processes will be killed now. 
    
    Broadcast message from root@tyler (tty) at 23:51:21 ... 
    
    shutdown: THE SYSTEM IS BEING SHUT DOWN NOW
    
    [..]
    
    Wait for '....Halt completed....' before stopping. 
    Error reporting has stopped.
    

    Capturing the host

    Just select the virtual machine you want to capture, and click capture ;-) :

    powervc_capture_ted
    snapshot1

    Here are the step realized by PowerVC when running a capture (so be careful to have enough space on PowerVC and Virtual I/O Server before running it :

    • By looking on the Virtual I/O Server itself, the main capture process is a simple dd command capturing the logical volume of the physical volume used as rootvg backing device, once the dd is finished this one is gzipped (in /home/padmin) :
    • padmin@deckard# ps -ef | grep dd      
          root  5832754  7078058   9 06:54:09      -  0:01 dd if=/dev/lv00 bs=1024k
          root  7078058  9043976  82 06:54:09      -  0:14 dd if=/dev/lv00 bs=1024k
        padmin  8388674  9699504   2 06:59:20  pts/0  0:00 grep dd
      padmin@deckard# ls -l /home/padmin/5154d176-6c3b-4eda-aa20-998deb207ca8.gz
      -rw-r--r--    1 root     staff    6102452605 Feb 05 07:13 5154d176-6c3b-4eda-aa20-998deb207ca8.gz
      
    • Once again the captured image is transferred to nova with ftp (ftpd process is spawned on the Virtual I/O Server) :
    • padmin@deckard# ps -ef | grep ftp
        padmin  7012516  9699504   1 07:20:47  pts/0  0:00 grep ftp
        padmin  7078072  4587660  47 07:14:18      -  0:11 ftpd
      [root@powervc-express ~]#  ls -l /var/lib/nova/images
      total 4666504
      -rw-r--r--. 1 nova nova 4778496000 Feb  5 00:22 5154d176-6c3b-4eda-aa20-998deb207ca8.gz
      
    • Then again the image is unzipped to glance :
    • # ls -l /var/lib/glance/images
      total 6453096
      -rw-r-----. 1 glance glance 1918828544 Feb  5 00:30 5154d176-6c3b-4eda-aa20-998deb207ca8
      -rw-r-----. 1 glance glance 4689133568 Feb  4 22:26 89386e2e-c2f0-4f9d-b0f4-267f4a99dc1e
      
    • During all these steps you can check take the Virtual Machine is in snapshot mode :
    • snapshot2

    • After the capture completion, you can have a look on the details :
    • image_3

    Deploying

    Deploying an host is very easy, just follow the instruction :

    • Here is an example of a deploy screen (I like visual thing when reading documents :-) ) :
    • deploy1

    • Choose on which host you want to deploy the machine. You can on this step select the number of instance to deploy (you had to have a dhcp network configured for multiple instance), and select the size of the machine (by default a few one are pre-defined but you can define you own templates) :
    • deploy_1
      choose

    • PowerVC is smart enough to tell you a prediction of your machine usage. (It will show you in yellow the whole usage of your Power Server after the machine deployment (practical, and visual, love it !) :
    • deploy_3

    • Then just wait for the deployment to finish, steps are the same as an iso deployment, but the activation engine will be started at first boot to reconfigure the virtual machine :
    • deploy2

    Here is a image to sum-up the capture, the ISO deployment, and a deployment of a Virtual Machine, I think it’ll be easier for you to understand with an image :

    powervc-deploy-capture

    Tips and tricks

    Using PowerVC express on Power 6 machines

    There are a few things not written and tell in the manual. By looking in the source code you can find an hidden option to add in the /etc/nova/nova.conf file. There is one very interesting option for PowerVC express that allows you to try it on a Power6 server. If you want to do this, just add ivm_power6_enabled = true to /etc/nova/nova.conf. Restart PowerVC service before you can add any Power 6 server. The piece of code can be found in /usr/lib/python2.6/site-packages/powervc_discovery/registration/compute/ivm_powervm_registrar.py file :

    LOG.info("ivm_power6_enabled set to TRUE in nova.conf, "
             "so POWER6 will be allowed for testing")
    

    If you want to do so, just add it in the /etc/nova/nova.conf file in the [DEFAULT] section

    # grep power6 /etc/nova/nova.conf
    ivm_power6_enabled = true
    

    Just for the story, I was sure this was possible because the first presentation I’ve found on the internet about PowerVC was on PowerVC Express on a 8203-EA4 machine which is a Power 6 machine, the screenshots provided in these presentation were enough to tell me it was possible (don’t blame anybody for this). Next grep was my best friend to find were this option was hidden. Be aware that this option is only available for test purpose, so don’t open a PMR about this or it’ll be directly closed by IBM. Once again if IBMers are reading this one, tell me if it is ok to publish this option. If not I can remove it from the post.

    Enabling verbose and debug output

    PowerVC is not verbose at all when something is going wrong it’s sometimes difficult to check what is going on. First off all, the product is based on OpenStack so you’ll have access to all OpenStack log files. These files are located to /var/log/nova, /var/log/glance and so on. By default debug and verbose output are disabled for each OpenStack part. This is not supported by PowerVC but you can enable this verbose and debug output. For instance I had problem with nova when registering an host, putting verbose and debug mode in /etc/nova/nova.conf helped me a lot and let me check the ssh command run on the Virtual I/O Server (look for on the example below):

    # grep -iE "verbose|debug" /etc/nova/nova.conf
    verbose=true
    debug=true
    # vi /var/log/nova/compute-192_168_0_100.log
    2014-02-16 22:07:48.523 13090 ERROR powervc_nova.virt.ibmpowervm.ivm.exception [req-a4bee79a-5eb8-43fd-8ca6-ed75ebee880f 04c4ca89f32046ed91e0493c9e554d1d 437b161186414e2bb0d4778cbd6fa14c] Unexpected exception while running IVM command.
    Command: mksyscfg -r lpar -i "max_virtual_slots=64,max_procs=4,lpar_env=aixlinux,desired_procs=1,min_procs=1,proc_mode=shared,virtual_eth_adapters=\"36/0/1//0/0\",desired_proc_units=0.100000000000000,sharing_mode=uncap,min_mem=512,desired_mem=512,virtual_eth_mac_base_value=fa3f3d3cae,max_proc_units=4,lpar_proc_compat_mode=default,name=priss-6712136f-000000cd,max_mem=4096,min_proc_units=0.1"
    Exit code: 1
    Stdout: []
    Stderr: ['[VIOSE01040181-0025] Value for attribute desired_proc_units is not valid.', '']
    

    Using the PowerVC Rest API

    Systems engineers and systems administrator like me are rarely using REST APIs. If you want to automate some PowerVC actions such as deploying virtual machines without the need to go one the web interface you have to use the REST API provided with PowerVC. First of all here are the places where you’ll find some useful documentation for the PowerVC REST API

    • On the PowerVC inforcenter, you’ll find good tips and tricks for using the REST API :
    • The PowerVC programming guide :

    PowerVC is providing a script that is using the REST API. This one will generate an API token used for each calls of the API. This script is written in Python so I decided to take this script as reference to develop my own scripts based on this one :

    • You first have to use powervc-get-token to generate a token used to call the API. In general GET requests are used to query PowerVC (list virtual machines, list networks), and POST request to create things (create a network, create a virtual machine).
    • Get an API token to begin, by using powervc-get-token :
    • # powervc-get-token 
      Password: 
      323806024c70455d84a7a1db900a4f89
      
    • To create a virtual machine you’ll need to know three things, the tenant, the network on which the VM will be deployed and the images used to deploy the server.
    • Here is the script I used to get the tenant (url : /powervc/openstack/identity/v2.0/tenants) :
    • import httplib
      import json
      import os
      import sys
      
      def main():
          token = raw_input("Please enter PowerVC token : ")
          print "PowerVC token used = "+token
      
          conn = httplib.HTTPSConnection('localhost')
          headers = {"X-Auth-Token":token, "Content-type":"application/json"}
          body = ""
      
          conn.request("GET", "/powervc/openstack/identity/v2.0/tenants", body, headers)
          response = conn.getresponse()
          raw_response = response.read()
          conn.close()
          json_data = json.loads(raw_response)
          print json.dumps(json_data, indent=4, sort_keys=True)
      
      if __name__ == "__main__":
          main()
      
    • By running the script I get the tenant id 437b161186414e2bb0d4778cbd6fa14c :
    • # ./powervc-get-tenants
      Please enter PowerVC token : a3a9904fa5a24a24aa6833358f54c7ce
      PowerVC token used = a3a9904fa5a24a24aa6833358f54c7ce
      {
          "tenants": [
              {
                  "description": "IBM Default Tenant", 
                  "enabled": true, 
                  "id": "437b161186414e2bb0d4778cbd6fa14c", 
                  "name": "ibm-default"
              }
          ], 
          "tenants_links": []
      }
      
    • Here is the script I used to get the network id (url :/powervc/openstack/network/v2.0/networks) :
    • import httplib
      import json
      import os
      import sys
      
      def main():
          token = raw_input("Please enter PowerVC token : ")
          print "PowerVC token used = "+token
          tenant_id = raw_input("Please enter PowerVC Tenant ID : ")
          print "Tenant ID = "+tenant_id
      
          conn = httplib.HTTPSConnection('localhost')
          headers = {"X-Auth-Token":token, "Content-type":"application/json"}
          body = ""
      
          conn.request("GET", "/powervc/openstack/network/v2.0/networks", body, headers)
          response = conn.getresponse()
          raw_response = response.read()
          conn.close()
          json_data = json.loads(raw_response)
          print json.dumps(json_data, indent=4, sort_keys=True)
      
      if __name__ == "__main__":
          main()
      
    • By running the script I get the network id 83e233a7-34ef-4bf2-ae95-958046da770f :
    • # ./powervc-list-networks
      Please enter PowerVC token : a3a9904fa5a24a24aa6833358f54c7ce
      PowerVC token used = a3a9904fa5a24a24aa6833358f54c7ce
      Please enter PowerVC Tenant ID : 437b161186414e2bb0d4778cbd6fa14c
      Tenant ID = 437b161186414e2bb0d4778cbd6fa14c
      {
          "networks": [
              {
                  "admin_state_up": true, 
                  "id": "83e233a7-34ef-4bf2-ae95-958046da770f", 
                  "name": "local_net", 
                  "provider:network_type": "vlan", 
                  "provider:physical_network": "default", 
                  "provider:segmentation_id": 1, 
                  "shared": false, 
                  "status": "ACTIVE", 
                  "subnets": [
                      "6b76f7e6-02fa-427f-9032-e8d28aaa6ef4"
                  ], 
                  "tenant_id": "437b161186414e2bb0d4778cbd6fa14c"
              }
          ]
      }
      
    • Here is the script I used to get the image id (url : /powervc/openstack/compute/v2/”+tenant_id+”/images)
    • import httplib
      import json
      import os
      import sys
      
      def main():
          token = raw_input("Please enter PowerVC token : ")
          print "PowerVC token used = "+token
          tenant_id = raw_input("Please enter PowerVC Tenant ID : ")
          print "Tenant ID ="+tenant_id
      
          conn = httplib.HTTPSConnection('localhost')
          headers = {"X-Auth-Token":token, "Content-type":"application/json"}
          body = ""
      
          conn.request("GET", "/powervc/openstack/compute/v2/"+tenant_id+"/images", body, headers)
          response = conn.getresponse()
          raw_response = response.read()
          conn.close()
          json_data = json.loads(raw_response)
          print json.dumps(json_data, indent=4, sort_keys=True)
      
      if __name__ == "__main__":
          main()
      
    • By running the script I get the image id 0537da41-8542-41a0-b1b0-84ed75c6ed27 :
    • # ./powervc-list-images
      Please enter PowerVC token : a3a9904fa5a24a24aa6833358f54c7ce
      PowerVC token used = a3a9904fa5a24a24aa6833358f54c7ce
      Please enter PowerVC Tenant ID : 437b161186414e2bb0d4778cbd6fa14c
      Tenant ID = 437b161186414e2bb0d4778cbd6fa14c
      {
          "images": [
              {
                  "id": "0537da41-8542-41a0-b1b0-84ed75c6ed27", 
                  "links": [
                      {
                          "href": "http://localhost:8774/v2/437b161186414e2bb0d4778cbd6fa14c/images/0537da41-8542-41a0-b1b0-84ed75c6ed27", 
                          "rel": "self"
                      }, 
                      {
                          "href": "http://localhost:8774/437b161186414e2bb0d4778cbd6fa14c/images/0537da41-8542-41a0-b1b0-84ed75c6ed27", 
                          "rel": "bookmark"
                      }, 
                      {
                          "href": "http://192.168.0.12:9292/437b161186414e2bb0d4778cbd6fa14c/images/0537da41-8542-41a0-b1b0-84ed75c6ed27", 
                          "rel": "alternate", 
                          "type": "application/vnd.openstack.image"
                      }
                  ], 
                  "name": "ted_capture_201402161858"
              }
          ]
      }
      
    • With all this information, the token (a3a9904fa5a24a24aa6833358f54c7ce), the tenant id (437b161186414e2bb0d4778cbd6fa14c), the network id (83e233a7-34ef-4bf2-ae95-958046da770f), the image id (0537da41-8542-41a0-b1b0-84ed75c6ed27), I create a script to create a virtual machine (url : /powervc/openstack/compute/v2/”+tenant_id+”/servers) :
    • import httplib
      import json
      import os
      import sys
      
      def main():
          token = raw_input("Please enter PowerVC token : ")
          print "PowerVC token used = "+token
          tenant_id = raw_input("Please enter PowerVC Tenant ID : ")
          print "Tenant ID ="+tenant_id
          headers = {"Content-Type": "application/json"}
      
          conn = httplib.HTTPSConnection('localhost')
          headers = {"X-Auth-Token":token, "Content-type":"application/json"}
      
          body = {
            "server": {
              "flavor": {
                "OS-FLV-EXT-DATA:ephemeral": 10,
                "disk": 10,
                "extra_specs": {
                  "powervm:proc_units": 1
                },
                "ram": 512,
                "vcpus": 1
              },
              "imageRef": "0537da41-8542-41a0-b1b0-84ed75c6ed27",
              "max_count": 1,
              "name": "api",
              "networkRef": "83e233a7-34ef-4bf2-ae95-958046da770f",
              "networks": [
                {
                "fixed_ip": "192.168.0.21",
                "uuid": "83e233a7-34ef-4bf2-ae95-958046da770f"
                }
              ]
            }
          }
      
          conn.request("POST", "/powervc/openstack/compute/v2/"+tenant_id+"/servers",
                       json.dumps(body), headers)
          response = conn.getresponse()
          raw_response = response.read()
          conn.close()
          json_data = json.loads(raw_response)
          print json.dumps(json_data, indent=4, sort_keys=True)
      
      if __name__ == "__main__":
          main()
      
    • Running the script will finally create the virtual machine, you can check that the virtual machine is in deploying state in the PowerVC web interface : :
    • api

      # ./powervc-create-vm 
      Please enter PowerVC token : a3a9904fa5a24a24aa6833358f54c7ce
      PowerVC token used = a3a9904fa5a24a24aa6833358f54c7ce
      Please enter PowerVC Tenant ID : 437b161186414e2bb0d4778cbd6fa14c
      Tenant ID =437b161186414e2bb0d4778cbd6fa14c
      {
          "server": {
              "OS-DCF:diskConfig": "MANUAL", 
              "adminPass": "LE2bqbA2y87X", 
              "id": "0c7521d1-7e09-4c07-bc19-40e9ac3b756f", 
              "links": [
                  {
                      "href": "http://localhost:8774/v2/437b161186414e2bb0d4778cbd6fa14c/servers/0c7521d1-7e09-4c07-bc19-40e9ac3b756f", 
                      "rel": "self"
                  }, 
                  {
                      "href": "http://localhost:8774/437b161186414e2bb0d4778cbd6fa14c/servers/0c7521d1-7e09-4c07-bc19-40e9ac3b756f", 
                      "rel": "bookmark"
                  }
              ], 
              "security_groups": [
                  {
                      "name": "default"
                  }
              ]
          }
      }
      

    Backup and restore

    PowerVC does not have any HA solution, so my advice is to run it on a ip alias and to have a second dormant PowerVC instance ready to be setup at the time you need it. To do so my advice is to regularly run a powervc-backup (why not in crontab). If you need to restore PowerVC on the dormant instance, the only thing to do is to restore the backup (put it in /var/opt/imb/powervc/backups before running powervc-restore). The backup/restore is just an export/import of each db2 database (cinder,glance,nova,…), so it can take space and time (in my case my backup takes 8Gb and restoring the backup on the dormant instance takes me 1 hour).

    • Backuping :
    • # powervc-backup 
      Continuing with this operation will stop all PowerVC services.  Do you want to continue?  (y/N):y
      PowerVC services stopped.
      Database CINDER backup completed.
      Database QTM_IBM backup completed.
      Database NOSQL backup completed.
      Database NOVA backup completed.
      Database GLANCE backup completed.
      Database KEYSTONE backup completed.
      Database and file backup completed. Backup data is in archive /var/opt/ibm/powervc/backups/20142199544840294/powervc_backup.tar.gz.
      PowerVC services started.
      PowerVC backup completed successfully.
      
    • Restoring :
    • # powervc-restore --noPrompt
      

    Places to check

    Finding information about PowerVC is not so simple, the product is still young and there are not many feedbacks and informations about it. Here are a few places to check if you have any problems. Keep in mind that the community is very active and is growing day by day :

    If I have one last word to say this one will be future. In my opinion PowerVC is the future for deployment on Power Systems. I had the chance to use VMcontrol and PowerVC : both are powerful, but the second is so simple to use that I can easily say that it will be used by IBM customers (has anyone used VMcontrol in production outside of PureSystems ?). Where VMcontrol has fail PowerVC can succeed …. but looking in the code you’ll find some part of VMcontrol (in the Activation Engine). So the ghost of VMcontrol is not so far and wil surely be kicked by PowerVC. Once again, I hope it helps, comments are welcome, I really need it to be sure my posts are useful and simple to understand.

    end

    Use your NIM server to install Red Hat Enterprise Linux 6 for IBM Power

    I’m actually working on a project including the deployement of PowerVC. Not one PowerVC but many. Like everybody know PowerVC is not running on AIX but is only -for the moment and forever- available on Red Hat Enterprise Linux 6.4. If you do not have the infrastructure or servers (check IBM Installation Toolkit for PowerLinux) to install Red Hat Enterprise Linux, you can choose to use your NIM server to do so. We will not use the NIM commands for that purpose because NIM is not designed to install Linux servers, we’ll just re-use tools and services used by NIM such as bootp and tftp to deploy our new Red Hat Enterprise Linux for Power. Here is how I did it with the errors and mistakes made while trying to do it.

    Copying the Red Hat Enterprise Linux 6 DVD on the NIM Server

    You’ll need to copy the whole Red Hat Enterprise Linux 6 DVD on the NIM server because you’ll somes files (initrd.img, vmlinuz) for booting the lpar with tftp and bootp, and all the RPM files for the installation itself. I personnally choose to copy the content of the DVD in /export/nim/lpp_source/nim/rhel/6.4 :

    # mkdir -p /export/nim/lpp_source/rhel/6.4
    # loopmount -i /export/iso/rhel-server-6.4-ppc64-dvd.iso -o "-V cdrfs -o ro" -m /mnt
    # cp -ra /mnt /export/nim/lpp_source/rhel/6.4
    # umount /mnt
    

    tftpd configuration

    By using the NIM server to install a Linux you’ll need to download files with tftp service. By default NIM is configured to allow dowloading files from tftp in the /tftpboot directory only. You’ll need to add (an)other location(s) for your own use, in my case I want to download yaboot.conf file initrd.img and vmlinuz files (I first wanted to add the ppc64.img file, but I had to remove it and replace it by the yaboot file -check the explication below). To allow downloading with tftp from different locations modify the /etc/tftpaccess.ctl configuration file. In my case I had to add /etc and /tftpboot/etc filesystems (I’ll will explain this later).

    cat /etc/tftpaccess.ctl
    # NIM access for network boot
    allow:/tftpboot
    allow:/tftpboot/etc
    allow:/etc
    allow:/export/lpp_source/rhel/6.4
    

    Identify the file you’ll need to make your installation :

    • yaboot image and yaboot configuration file are needed to network load yaboot (you do not have to move yaboot image (it will not be modified)), just copy the yaboot.conf from the DVD to /etc (yaboot will search in /etc) :
    • # cp /export/nim/lpp_source/rhel/6.4/ppc/ppc64/yaboot.conf /etc/yaboot.conf
      
    • vmlinuz and initrd.img will be downloaded with tftp after yaboot will be loaded so move these files in /tftpboot/etc (I’ll explain this later on the yaboot configuration part) :
    • # mkdir -p /tftpboot/etc
      # cp /export/nim/lpp_source/rhel/6.4/ppc/ppc64/initrd.img /export/nim/lpp_source/rhel/6.4/ppc/ppc64/vmlinuz /tftpboot/etc 
      

    bootptab modification

    You cannot modifiy the /etc/bootptab file for you own use by using NIM commands so you had to do it by hand. The bootptab file will tell your client which files to download to begin the installation. Before Red Hat Enterprise Linux 6 you can use the ppc64.img to begin the installation directly by booting from this file. On Red Hat Enterprise Linux 6 you have to do it by using yaboot, so edit the /etc/bootptab file and add this parameters :

    • The name of the server, in my case : powervc.lab.chmod666.org
    • The boot file used to boot the server, in our case the yaboot file : /export/nim/lpp_source/rhel6/6.4/ppc/chrp/yaboot
    • The IP address of the server : 10.10.20.21
    • The type of network : ethernet
    • The IP address of the NIM server : 10.10.20.20
    • The gateway address :10.10.20.254
    • The netmask : 255.255.255.0

    Fields are separated by ‘:’, do not forget to add the last ‘:’.

    # tail -1 /etc/bootptab
    powervc.lab.chmod666.org:bf=/export/nim/lpp_source/rhel6/6.4/ppc/chrp/yaboot:ip=10.10.20.21:ht=ethernet:sa=10.10.20.20:gw=10.10.20.254:sm=255.255.255.0:
    

    It’s a matter of size : do not try to boot ppc64.img

    If you already use this method to install a Red Hat Enterprise Linux 5 you probably just use the ppc64.img and do not have to be annoyed by all the yaboot configuration. You can’t do that with Red Hat Enterprise Linux 6 because the size of the ppc64.img file is greater than 32MB. tftp downloads are limited to 32MB, it is like this by design and you can verify it in the RFC. I’m not the only one who had this problem and a official bug was opened on the Red Hat Bugzilla : https://bugzilla.redhat.com/show_bug.cgi?id=613929 : Cannot install RHEL6 via TFTP netboot using ppc64.img. Both Red Hat Enterprise Linux 6 and tftp are defected by designed. If you try do so you’ll get an error while downloading the ppc64.img file “No operating system install”

    Configuring yaboot

    All of this comes from my own experience while trying to configure yaboot, yaboot boot process is not very well documented and I had to look in the source code to check how it works :

    1. yaboot image is loaded with tftp.
    2. yaboot will search it’s configuration file located in the /etc/, for example : /etc/00aabbccddee
    3. If yaboot does not find this file it will try fo find it’s configuration file by searching the IP address of your server in hexadecimal. If it does not find it, it’ll remove the last byte from the address, in our example /etc/0a0a1415 then /etc/0a0a14, and so on, until the first byte.
    4. If yaboot does not fine any mac address or IP address based file it will finally search for an /etc/yaboot.conf file.

    I personnally chose to configure yaboot to let me install Red Hat Enterprise Linux with or without a kickstart file, so I have two blocks one called linux and the other one called linux-ks. The Linux one will let me install Linux by using the Red Hat installer and I have to answer the question for the installation. The seconde one : linux-ks let me install Linux with a kickstart based installation. In both case I have to load with tftp initrd.img and vmlinuz files, and in the kickstart case I have to download the ks.cfg file (In my case I have configured a web server to provide it but you can use AIX nfs if you want :-) ). Here is my /etc/yaboot.conf file

    • Configuration file header for /etc/yaboot.conf :
    • init-message = "\nWelcome to the 64-bit Red Hat Enterprise Linux 6.4 installer!\nHit  for boot options.\n\n"
      timeout=6000
      default=linux
      
    • linux block :
    • image=etc/vmlinuz
              label=linux
              initrd=etc/initrd.img
              read-only
      
    • linux-ks block :
    • image=etc/vmlinuz
              label=linux-ks
              initrd=etc/initrd.img
              append="ks=http://10.10.20.20:8080/rhel6/ks.cfg ksdevice=eth0 ip=10.253.112.146 netmask=255.255.255.254 gateway=10.253.112.129 noipv6"
              read-only
      

    As you can see on the output above, yaboot needs initrd.img file and vmlinuz file, and it’ll search it in /tftpboot/etc do not add /tftpboot/ as the tftpboot client root is /tftpboot and you can use like me relative paths. I had to configure a web server to allow net based installation without NFS, but you can use NFS provided by the NIM server.

    # tail -7 /etc/opt/freeware/apache/httpd.conf 
    Alias /rhel6/ "/export/nim/lpp_source/rhel/6.4/"
    
        AllowOverride None
        Options ExecCGI Includes FollowSymLinks Indexes
        Order allow,deny
        Allow from all
    
    

    My kickstart file is also provided by this web server :

    # ls -l /export/nim/lpp_source/rhel/6.4/ks.cfg
    -rw-r--r--    1 root     system         1394 Jan  2 11:06 /export/lpp_source/core/rhel/6.4/ks.cfg
    

    Installation sequence

    Here is the boot sequence after everything is configured and working as desired. Configure your IPL device as you always do, and boot it through the SMS menu :

    • First, yaboot image is downloaded with tftp :
    • TFTP BOOT ---------------------------------------------------
      Server IP.....................10.10.20.20
      Client IP.....................10.10.20.21
      Gateway IP....................10.10.20.254
      Subnet Mask...................255.255.255.0
      ( 1  ) Filename................./export/nim/lpp_source/rhel/6.4/ppc/chrp/yaboot
      TFTP Retries..................5
      Block Size....................512
      FINAL PACKET COUNT = 516
      
    • Then yaboot is searching for its configuration file, first trying by searching a configuration file named with the mac address of the machine :
    • System has 256 Mbytes in RMA
      Try to netboot
      
      claim of 0x2000000 at 0xe000000 returned 0xe000000
      
      TFTP BOOT ---------------------------------------------------
      Server IP.....................10.10.20.20
      Client IP.....................10.10.20.21
      Gateway IP....................10.10.20.254
      Subnet Mask...................255.255.255.0
      ( 1  ) Filename.................\etc\01-8a-70-76-c4-13-02
      TFTP Retries..................5
      Block Size....................512
      1
      2
      3
              !BA017021 !
      
      Error, can't read config file
      
    • If yaboot does not find this file it’ll try with the IP address of the server in hexadecimal :
    • Error, can't read config file
      
      
      claim of 0x2000000 at 0xe000000 returned 0xe000000
      
      TFTP BOOT ---------------------------------------------------
      Server IP.....................10.10.20.20
      Client IP.....................10.10.20.21
      Gateway IP....................10.10.20.254
      Subnet Mask...................255.255.255.0
      ( 1  ) Filename.................\etc\0a0a2021
      TFTP Retries..................5
      Block Size....................512
      1
      2
      3
             !BA017021 !
      
      Error, can't read config file
      
      
      claim of 0x2000000 at 0xe000000 returned 0xe000000
      
      TFTP BOOT ---------------------------------------------------
      Server IP.....................10.10.20.20
      Client IP.....................10.10.20.21
      Gateway IP....................10.10.20.254
      Subnet Mask...................255.255.255.0
      ( 1  ) Filename.................\etc\0a0a20
      TFTP Retries..................5
      Block Size....................512
      1
      2
      3
      
    • Finally yaboot will search for a file named yaboot.conf :
    • FTP BOOT ---------------------------------------------------
      Server IP.....................10.10.20.201
      Client IP.....................10.10.20.21
      Gateway IP....................10.10.20.254
      Subnet Mask...................255.255.255.0
      ( 1  ) Filename.................\etc\yaboot.conf
      TFTP Retries..................5
      Block Size....................512
      FINAL PACKET COUNT = 1
      FINAL FILE SIZE = 442  BYTES
      Config file read, 442 bytes
      
      Welcome to the 64-bit Red Hat Enterprise Linux 6.4 installer!
      Hit  for boot options.
      
    • Choose one option to boot on (in our case linux-ks) :
    • Welcome to yaboot version 1.3.14 (Red Hat 1.3.14-41.el6)
      Enter "help" to get some basic usage information
      boot:
      * linux                      linux-ks
      
      boot: linux-ks
      Please wait, loading kernel...
      
    • Kernel vmlinuz is loaded with tftp:
    • claim of 0x2000000 at 0xe000000 returned 0xe000000
      
      TFTP BOOT ---------------------------------------------------
      Server IP.....................10.10.20.20
      Client IP.....................10.10.20.21
      Gateway IP....................10.10.20.254
      Subnet Mask...................255.255.255.0
      ( 1  ) Filename.................etc\vmlinuz
      TFTP Retries..................5
      Block Size....................512
      FINAL PACKET COUNT = 32730
      FINAL FILE SIZE = 16757576  BYTES
         Elf64 kernel loaded...
      Loading ramdisk...
      
    • Ramdisk initrd.img is loaded with tftp :
    • claim of 0x2000000 at 0xe000000 returned 0xe000000
      
      TFTP BOOT ---------------------------------------------------
      Server IP.....................10.10.20.21
      Client IP.....................10.10.10.20
      Gateway IP....................10.10.20.254
      Subnet Mask...................255.255.255.0
      ( 1  ) Filename.................etc\initrd.img
      TFTP Retries..................5
      Block Size....................512
      FINAL PACKET COUNT = 54371
      
    • Finally the installation begin because with choose the linux-ks :
    • Welcome to Red Hat Enterprise Linux for ppc64
          +----------------------------¦ Retrieving +----------------------------+
          ¦                                                                      ¦
          ¦ Retrieving /install.img...                                           ¦
          ¦                                                                      ¦
          ¦                                  17%                                 ¦
          ¦                                                                      ¦
          +----------------------------------------------------------------------+
        / between elements  |  selects |  next screen
      

    IBM Installation toolkit for PowerLinux

    Finally if your are planning to install a lot of Linux Servers on Power Hardware IBM provide a solution to easily install and deploy Linux OS on Power Hardware. This tool is called IBM Installation Toolkit for PowerLinux and can be find here : http://www-304.ibm.com/webapp/set2/sas/f/lopdiags/installtools/home.html. You’ll need to dedicated (Linux ?) server for this product if you want to do so. I’ve never tested it but if I had to deploy more and more Linux servers I’ll give it a try, after all NIM server are used to install AIX and not Linux :-)

    Building my AIX/Power Home Lab : HMC/iSCSI/Power Saving and other tricks

    I’m living in fear ! A few months ago I decided to quit my job for some reasons I can’t tell you in this blog. As many of you already know, you know exactly what you quit when you are leaving a place, but do not exactly know what you will get in the new one. Unfortunately this is the way France is doing today about jobs in general (just ask you why we do not have any Champion for Power Systems in France ?). Anyway this blog isn’t about politics but about technical stuffs. So I finally admitted that my professional path will not be a straight forward line, and with this comes the feeling that I needed to work and experiment some stuffs on my own without being at work. Here are some benefits I have to own my Power Server at home :

    • I’m the only developer of nmon2graphite. I really need a test platform to add new features and maintain nmon2graphite.
    • I can experiment things on my own without the fear to broke some production hosts, and to finally be fired. Example : I deliberately create a Shared Ethernet Adapter with wrong parameters to create a network loop (with some wireshark trace) to really understand why so many people are doing this mistake and how it really works. (I’m planning to post about this.)
    • I do not have to wait to experiment things, I’m the master/keeper in the house, no more firewall, dns problems, I can do what I want !
    • I can broke things and do it again and again. Need an IVM ? No problem less than one hour to build an IVM. Need an HMC … same thing.
    • It’s cool when some geek friends are coming home and I can say “Hey buddy, wanna play on my Power server ?”

    I know this is a lost cause by I give it a try here. If some IBMers are reading this blog. My P520 Express is running a PowerVM standard edition, if someone can provides me a COD code for an enterprise edition it’ll be very cool (Am I dreaming ? Is the world really managed by money ?) . Remember that this blog is here to help the Power community, there is no business behind this blog. It’s free. I’m blogging in my free time, it’s a passion.

    mypower1

    It’s a long way to the promise land.

    Finding a Power Server at a cheap price is not easy. My first tough was to ask people who already buy one for their own use how they did it. I decided to ask Brian Smith who owns a few Power Systems : How did you get your servers ? He kindly answers to my question and told me to check on eBay or in governmental auctions. So I started to check eBay in the beginning of May 2013. In Europe prices were too expansives for my budget (no server cheaper than 5000€) ; so I decided to check in US if prices were cheaper. For price convenience my advice is to check for a Power 520 express or a Power710 express if you can afford to spend more money. Before buying anything always :

    • Ask for the Serial Number to check activated options, go on the IBM POD for this http://www-912.ibm.com/pod/pod.
    • Always ask for the PowerVM Edition. In my case my need is PowerVM Standard or Enterprise. Go on this page to check how to identify your PowerVM edition : http://www-01.ibm.com/support/docview.wss?uid=isg3T1010860
    • Always ask for adapters and disks shipped with the server. A lot of servers are sold without any adapter.
    • Always ask for power supply with the server. A lot of servers are sold without any power.
    • If you are planning to install dual Virtual I/O Server check if a split backplane is installed with the system (in my case I have no split backplane, and I’ll tell you how to setup dual Virtual I/O Server in this case.)
    • Check the price ! Don’t get ripped off !

    After three months asking questions and checking eBay everyday. I finally found something interesting. The NeuComp Company (an IBM Business Partner) is selling refurbished Power Server on eBay. I wanted to thanks Katie K. (if you are reading this Katie tell me if I can give you real name :-) ) for her kindness and for answering all my questions. If you are buying from a foreign country don’t make the same mistake as me and be aware of the custom fees (in my case around 15% of the total prices), seriously DHL guys are ruthless …

    poweropened

    What about the Hardware Management Console ?

    If you are planning to have a PowerVM Standard/Enterprise edition server it’s easier to manage it through an Hardware Management Console : two solutions :

    • Find an old hardware management console, on eBay it’s not to expansive and affordable.
    • Like Scott Vetter point in this post : The Power Systems Hardware Management Console as a virtual appliance ? it is possible to virtualize an Hardware Management Console, or an SDMC. Like him I cannot tell you here how to do it but it’s doable, and it’s not my case :-) (Don’t ask me for this one). Anyway if you want an SDMC you are totally in legality if you download and install the SDMC ova file and deploy it on a KVM or a VMware infrastructure.

    Protect your ear. Check electricity bill : enable Power Save mode.

    Do not forget to enable power saver mode if you are owning a Power6 or Power7 server. First : fans speeds will be decreased and the server will produce less noise. Second : you’ll notice a huge difference at the end of the month on your electricity bill if you are running the server 24 hours a day ! Notice that when the power saver mode on a managed system is enabled, the processor voltage and clock frequency are lowered to reduce the power consumption of the processors in the managed system.

    • Enabling Power Saver mode (and check with pmcycles command on a partition) :
    • # chpwrmgmt -m power520e-8203-EA4 -r sys -o enable
      root@priss.lab.chmod666.org [/root] # pmcycles -Mm
      CPU 0 runs at 3618 MHz
      CPU 1 runs at 3617 MHz
      CPU 2 runs at 3619 MHz
      CPU 3 runs at 3619 MHz
      
    • Disabling Power Save mode :
    • # chpwrmgmt -m power520e-8203-EA4 -r sys -o disable
      root@priss.lab.chmod666.org [/root] # pmcycles -Mm
      CPU 0 runs at 4201 MHz
      CPU 1 runs at 4201 MHz
      CPU 2 runs at 4202 MHz
      CPU 3 runs at 4202 MHz
      

    As you can see on the output above the processor speed was decreased from 4.2GHz to 3.6GHz. I do not have any recorder but I can swear that fans are decreasing their speed at the moment the command was typed on the Hardware Management Console. Power Saving mode is not a joke it really works.

    Updating/Installation

    Firmware update

    The server was not shipped with the latest firmware level. If you are working in a big enterprise like me this kind of operations are performed by IBM Inspectors, so the last time I updated a firmware was a couple of months ago and I did it through the IBM Systems Director, this is not the kind of operation I’m used to do, so here is a quick reminder on how to do it :

    • First of all copy all the files (xml and sdd files included) you have downloaded with the new firmware on the Hardware Management Console :
    • hscroot@gaff:~> ls /home/hscroot/fw
      01EL350_149_038.dd.xml	01EL350_149_038.pd.sdd	    01EL350_149_038.rpm
      01EL350_149_038.html	01EL350_149_038.readme.txt  01EL350_149_038.xml
      
    • You can check with the lslic command if there are “flashable” firmware in a specific directory :
    • hscroot@gaff:~> lslic -m p520express-8203-E4A-SN0666000 -r mountpoint -d /home/hscroot/fw/ -F retrievable_release concurrent_retrievable_level disruptive_retrievable_level
      None 149 149
      
    • Flash the firmware with the updlic command (note that a copy of the old microcode is backuped in the /var filesystem):
    • hscroot@gaff:~> updlic -o a -t sys -l latest -m p520express-8203-E4A-SN0666000  -r mountpoint -d /home/hscroot/fw -v
      Current profile data backup files have been copied: 
      8203-E4A*0666000: /var/hsc/profiles/0666000/backupFile_FirmwareUpdate01EL350, /var/hsc/profiles/0666000/directory/backupFile_FirmwareUpdate01EL350.dir
      
      8203-E4A*0666000: Retrieving updates.
      8203-E4A*0666000 Managed System Primary: Retrieving updates.
      8203-E4A*0666000: Installing updates.
      8203-E4A*0666000 Managed System Primary: Preparing for install.
      8203-E4A*0666000: Installing updates.
      8203-E4A*0666000 Managed System Primary: Installing updates.
      8203-E4A*0666000 Managed System Primary: Writing update files.
      8203-E4A*0666000 Managed System Primary: Writing file 80a00020.  0 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 80a00701.  5959118 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 80a00701.  16043962 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 80a00711.  7858212 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 81cf0689.  0 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 81e00100.  2684818 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 81e00101.  2684818 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 81e00200.  65378 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 81e00200.  5304258 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 81e00400.  0 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file 81e00704.  0 bytes written.
      8203-E4A*0666000 Managed System Primary: Writing file a0e00c21.  0 bytes written.
      8203-E4A*0666000 Managed System Primary: Finished writing update files.
      8203-E4A*0666000: Activating updates.
      8203-E4A*0666000 Managed System Primary: Activating updates - Restarting Flexible Service Processor.
      8203-E4A*0666000 Managed System Primary: Flexible Service Processor Restart completed successfully.
      8203-E4A*0666000 Managed System Primary: Activation completed.
      8203-E4A*0666000: Completed All Updates.
      

    Virtual I/O Server install through the Hardware Management Console.

    To install my Virtual I/O Server I decided to use the new integrated Hardware Management Console feature. This feature allows you to install the Virtual I/O Server through the Hardware Management Console the first time you boot it. Unfortunately you need to have all NIM ports opened because it’s just a graphical interface to the old well known installios command based on the old nimol product and running on the HMC. What a shame I was dreaming of a full integrated installation without the need of bootp/NFS and so on. So I’m very disappointed about this new feature :-( .

    • You first have to download the Virtual I/O Server media to the Hardware Management Console, check you have enough space left on the HMC before importing images :
    • hmcimportvios

    • Wait until images are correctly uploaded to the Hardware Management Console :
    • hmcuploadimagewait

    • If images were correctly uploaded on the HMC, you’ll see it on the Virtual I/O Server image repository :
    • virtualioserverimagerepository

    • When you’ll boot your Virtual I/O Server the HMC will ask you if you want to install a Virtual I/O Server on it :
    • bootviosinstall

    • The Virtual I/O Server will boot and you’ll be asked for the IP address to use and the image you want to install, in this case set the HMC ip as NIM server (the HMC is a nimol server):
    • discoveringadapters
      hmcviosip

    • After this step the installation will begin and the output will be visible in the HMC window :
    • installhmc1

    • When the installation is finished you can check log output HMC window :
    • installhmc2
      installhmc3

    Need a cheap SAN : Use iSCSI

    This is the trickiest part of this post. My Power Server was not shipped with a split backplane my only solution was to create an iSCSI target and to boot my Virtual I/O Server on iSCSI, the iSCSI target partition was created on an AIX lpar and I decided to use file backed luns for backup convenience (I just have to copy the file to save my Virtual I/O Server rootvg :-) )

    Creating target

    Filesets for iSCSI target can be found on the AIX Expansion Pack DVD. You have to install these fileset on the iSCSI target before trying to configure it.

    # installp -aXYgd . devices.tmiscsw.rte 
    [..]
    # lslpp -Lc | grep devices.tmiscsw.rte 
    devices.tmiscsw:devices.tmiscsw.rte:7.1.2.0: : :C: :iSCSI Target Software Device Driver : : : : : : :0:0:/:1241
    
    • To create the target follow these steps :
    • # mkdev -c driver -s node -t tmsw 
      tmsw0 Available
      # mkdev -c tmiscsi -s tmtarget -t target -p tmiscsi0 -a owner='tmsw0' -a iscsi_name="inq.priss.lab.chmod666.org"  
      target0 Available
      
    • To create lun follow these steps : In my case I choose to create file backed iSCSI lun. I use dd command to create the lun itself, and then add it to the iSCSI configuration :
    • # dd if=/dev/zero of=/luns/lun1  bs=1m count=10k
      10240+0 records in.
      10240+0 records out.
      # ls -l /luns/lun1
      -rw-r--r--    1 root     system   10737418240 Dec 14 10:22 /luns/lun1
      # mkdev -c tmiscsi -s tmdev -t lu -p 'target1' -l 'lun1' -a back_dev_type='file' -a back_dev='lun1' -a back_file='/luns/lun1'
      lun1 Available
      # lsattr -El lun1
      back_dev        lun1       Backing Device Name                           True
      back_dev_option            N/A                                           True
      back_dev_type   file       Backing Device Type                           True
      back_file       /luns/lun1 Backing File Full Path Name                   True
      dev_size                   N/A                                           True
      lun_id          0x0        Logical Unit Number ID                        False
      queue_depth     3          Maximum Number of Commands to Queue to Device True
      

    Initiator configuration

    • Give a name to your initiator :
    • # chdev  -l 'iscsi0' -a initiator_name='inq.deckard'
      
    • Modify the /etc/iscsi/targets file to add the name and port of the iSCSI target, then cfgmgr :
    • # echo "192.168.0.21 3260 inq.priss.lab.chmod666.org" > /etc/iscsi/targets
      # tail -1 /etc/iscsi/targets
      192.168.0.21 3260 inq.priss.lab.chmod666.org
      # cfgmgr -vl iscsi0
      # lspv
      hdisk0          00ce7e539a9ee2fb                    rootvg          active      
      hdisk1          00ce7e53d67b71ca                    None 
      # lsdev -Cc disk | grep hdisk1 
      hdisk1 Available  Other iSCSI Disk Drive
      

    Boot on iSCSI

      If like me you decided to boot on iSCSI, follow the same steps as above to configure the target. Then boot you partition on SMS menu, begin the installation. The configuration of the iSCSI initiator is performed on the BOS installation menu, follow the steps below to configure it, and then boot on iSCSI :

    • When the menu comes up, choose Configure Network Disks (iSCSI) :
    •                            Welcome to Base Operating System
                            Installation and Maintenance
      
      Type the number of your choice and press Enter. Choice is indicated by >>>.
      
      >>> 1 Start Install Now with Default Settings
          2 Change/Show Installation Settings and Install
          3 Start Maintenance Mode for System Recovery
          4 Configure Network Disks (iSCSI)
          5 Select Storage Adapters
      
          88  Help ?
          99  Previous Menu
      
      >>> Choice [1]: 4  
      
    • Then choose Configure iSCSI :
    •                                 Configure iSCSI
      
      Move cursor to desired item and press Enter.
      
        Configure iSCSI
        Network Utilities
      
    • Set your iSCSI target name, ip, port and lun number, if all is ok you’ll see the iSCSI block devices presented to your server as hdisk devices :
    •                       iSCSI Configuration -- SW Initiator
      
      Type or select values in entry fields.
      Press Enter AFTER making all desired changes.
      
                                                              [Entry Fields]
      * Ethernet Interface                                  en0
      * IP Address of Ethernet Interface                    192.168.0.6
      * Network Mask of Ethernet Interface                  255.255.255.0
      * Gateway to iSCSI Target                             192.168.0.254
      * iSCSI Target Name                                  [inq.priss]
      * IP Address of iSCSI Target                         [192.168.0.21]
      * Port Number of iSCSI Target                        [3260]
      * iSCSI Initiator Name                               [inq.deckard]
      
                                       COMMAND STATUS
      
      Command: OK            stdout: yes           stderr: no
      
      Before command completion, additional instructions may appear below.
      
      en0 changed
      inet0 changed
      iscsi0 changed
      hdisk0 Available  N/A
      hdisk1 Available  N/A
      hdisk2 Available  N/A
      hdisk3 Available  N/A
      hdisk4 Available  N/A
      
    • After this step you can choose your disks. Do it the same way as a normal installation.
    • If you are checking the bootlist on a server booting on iSCSI, check with the -v option to have a detailed output :
    • # bootlist -o -m normal
      hdisk0 
      # bootlist -o -m normal -v
      'ibm,max-boot-devices' = 0x5
      NVRAM variable: (boot-device=/lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=192.168.0.6,subnet-mask=255.255.255.0,itname=inq.deckard,iport=3260,ilun=0,iname=inq.priss,siaddr=192.168.0.21,2)
      Path name: (/lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=192.168.0.6,subnet-mask=255.255.255.0,itname=inq.deckard,iport=3260,ilun=0,iname=inq.priss,siaddr=192.168.0.21,2)
      match_specific_info: ut=disk/iscsi/osdisk
      hdisk0 
      
    • I had to change my iSCSI target ip address for a dumb reason, but after changing this on my lpars booting on iSCSI were not able to find their boot disk anymore. The only solution I found was to change the boot device NVRAM boot-device variable Open firmware prompt :
    • 0 > printenv 
      [..]
      boot-device              /lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=8.0.0.2,subnet-mask=255.255.255.0,itname=iqn.localhost.hostid.00000000,iport=3260,ilun=0,iname=inq.priss,siaddr=8.0.0.1,2 
      [..]
      0 > setenv boot-device /lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=192.168.0.6,subnet-mask=255.255.255.0,itname=inq.deckard,,iport=3260,ilun=0,iname=inq.priss,siaddr=192.168.0.21,2  ok
      0 > printenv 
      [..]
      boot-device              /lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=192.168.0.6,subnet-mask=255.255.255.0,itname=inq.deckard,,iport=3260,ilun=0,iname=inq.priss,siaddr=192.168.0.21,2 
      [..]
      0 > boot -v
      iSCSI BOOT ---------------------------------------------------
      Server IP.....................192.168.0.21
      Client IP.....................192.168.0.6
      Subnet Mask...................255.255.255.0
      iSCSI Initiator...............inq.deckard
      iSCSI Target..................inq.priss
      Target Port...................3260 
      Target LUN....................0 
      
      \
      Elapsed time since release of system processors: 45 mins 24 secs
      

    If you are planning to buy your own Power Server don’t hesitate to contact me if you have any question. It really gives me the opportunity to test things I was not able to do at work, PowerHA migration for 6 to 7, PowerSC, some HMC features, some NIM features (suma), the possibility are infinite. If you want to contribute to the rising of chmod666.org all over the AIX world, I’m now looking for :

    • A Cisco manageable switch with ios and 16 ports.
    • A Power7 based server (any model).
    • An old DS* array.
    • A split backplane.
    • Any old hardware.

    Don’t hesitate to contact me if you are trashing old machines/switch I’m interested if you want to make me gifts :-) , I can buy it if you do not want to give it :-) . Once again I hope this post will help you or will make you dream. I’m living in fear it’s hard to assume :-( .