I’m living in fear ! A few months ago I decided to quit my job for some reasons I can’t tell you in this blog. As many of you already know, you know exactly what you quit when you are leaving a place, but do not exactly know what you will get in the new one. Unfortunately this is the way France is doing today about jobs in general (just ask you why we do not have any Champion for Power Systems in France ?). Anyway this blog isn’t about politics but about technical stuffs. So I finally admitted that my professional path will not be a straight forward line, and with this comes the feeling that I needed to work and experiment some stuffs on my own without being at work. Here are some benefits I have to own my Power Server at home :
- I’m the only developer of nmon2graphite. I really need a test platform to add new features and maintain nmon2graphite.
- I can experiment things on my own without the fear to broke some production hosts, and to finally be fired. Example : I deliberately create a Shared Ethernet Adapter with wrong parameters to create a network loop (with some wireshark trace) to really understand why so many people are doing this mistake and how it really works. (I’m planning to post about this.)
- I do not have to wait to experiment things, I’m the master/keeper in the house, no more firewall, dns problems, I can do what I want !
- I can broke things and do it again and again. Need an IVM ? No problem less than one hour to build an IVM. Need an HMC … same thing.
- It’s cool when some geek friends are coming home and I can say “Hey buddy, wanna play on my Power server ?”
I know this is a lost cause by I give it a try here. If some IBMers are reading this blog. My P520 Express is running a PowerVM standard edition, if someone can provides me a COD code for an enterprise edition it’ll be very cool (Am I dreaming ? Is the world really managed by money ?) . Remember that this blog is here to help the Power community, there is no business behind this blog. It’s free. I’m blogging in my free time, it’s a passion.
It’s a long way to the promise land.
Finding a Power Server at a cheap price is not easy. My first tough was to ask people who already buy one for their own use how they did it. I decided to ask Brian Smith who owns a few Power Systems : How did you get your servers ? He kindly answers to my question and told me to check on eBay or in governmental auctions. So I started to check eBay in the beginning of May 2013. In Europe prices were too expansives for my budget (no server cheaper than 5000€) ; so I decided to check in US if prices were cheaper. For price convenience my advice is to check for a Power 520 express or a Power710 express if you can afford to spend more money. Before buying anything always :
- Ask for the Serial Number to check activated options, go on the IBM POD for this http://www-912.ibm.com/pod/pod.
- Always ask for the PowerVM Edition. In my case my need is PowerVM Standard or Enterprise. Go on this page to check how to identify your PowerVM edition : http://www-01.ibm.com/support/docview.wss?uid=isg3T1010860
- Always ask for adapters and disks shipped with the server. A lot of servers are sold without any adapter.
- Always ask for power supply with the server. A lot of servers are sold without any power.
- If you are planning to install dual Virtual I/O Server check if a split backplane is installed with the system (in my case I have no split backplane, and I’ll tell you how to setup dual Virtual I/O Server in this case.)
- Check the price ! Don’t get ripped off !
After three months asking questions and checking eBay everyday. I finally found something interesting. The NeuComp Company (an IBM Business Partner) is selling refurbished Power Server on eBay. I wanted to thanks Katie K. (if you are reading this Katie tell me if I can give you real name :-)) for her kindness and for answering all my questions. If you are buying from a foreign country don’t make the same mistake as me and be aware of the custom fees (in my case around 15% of the total prices), seriously DHL guys are ruthless …
What about the Hardware Management Console ?
If you are planning to have a PowerVM Standard/Enterprise edition server it’s easier to manage it through an Hardware Management Console : two solutions :
- Find an old hardware management console, on eBay it’s not to expansive and affordable.
- Like Scott Vetter point in this post : The Power Systems Hardware Management Console as a virtual appliance ? it is possible to virtualize an Hardware Management Console, or an SDMC. Like him I cannot tell you here how to do it but it’s doable, and it’s not my case (Don’t ask me for this one). Anyway if you want an SDMC you are totally in legality if you download and install the SDMC ova file and deploy it on a KVM or a VMware infrastructure.
Protect your ear. Check electricity bill : enable Power Save mode.
Do not forget to enable power saver mode if you are owning a Power6 or Power7 server. First : fans speeds will be decreased and the server will produce less noise. Second : you’ll notice a huge difference at the end of the month on your electricity bill if you are running the server 24 hours a day ! Notice that when the power saver mode on a managed system is enabled, the processor voltage and clock frequency are lowered to reduce the power consumption of the processors in the managed system.
- Enabling Power Saver mode (and check with pmcycles command on a partition) :
# chpwrmgmt -m power520e-8203-EA4 -r sys -o enable firstname.lastname@example.org [/root] # pmcycles -Mm CPU 0 runs at 3618 MHz CPU 1 runs at 3617 MHz CPU 2 runs at 3619 MHz CPU 3 runs at 3619 MHz
# chpwrmgmt -m power520e-8203-EA4 -r sys -o disable email@example.com [/root] # pmcycles -Mm CPU 0 runs at 4201 MHz CPU 1 runs at 4201 MHz CPU 2 runs at 4202 MHz CPU 3 runs at 4202 MHz
As you can see on the output above the processor speed was decreased from 4.2GHz to 3.6GHz. I do not have any recorder but I can swear that fans are decreasing their speed at the moment the command was typed on the Hardware Management Console. Power Saving mode is not a joke it really works.
The server was not shipped with the latest firmware level. If you are working in a big enterprise like me this kind of operations are performed by IBM Inspectors, so the last time I updated a firmware was a couple of months ago and I did it through the IBM Systems Director, this is not the kind of operation I’m used to do, so here is a quick reminder on how to do it :
- First of all copy all the files (xml and sdd files included) you have downloaded with the new firmware on the Hardware Management Console :
hscroot@gaff:~> ls /home/hscroot/fw 01EL350_149_038.dd.xml 01EL350_149_038.pd.sdd 01EL350_149_038.rpm 01EL350_149_038.html 01EL350_149_038.readme.txt 01EL350_149_038.xml
hscroot@gaff:~> lslic -m p520express-8203-E4A-SN0666000 -r mountpoint -d /home/hscroot/fw/ -F retrievable_release concurrent_retrievable_level disruptive_retrievable_level None 149 149
hscroot@gaff:~> updlic -o a -t sys -l latest -m p520express-8203-E4A-SN0666000 -r mountpoint -d /home/hscroot/fw -v Current profile data backup files have been copied: 8203-E4A*0666000: /var/hsc/profiles/0666000/backupFile_FirmwareUpdate01EL350, /var/hsc/profiles/0666000/directory/backupFile_FirmwareUpdate01EL350.dir 8203-E4A*0666000: Retrieving updates. 8203-E4A*0666000 Managed System Primary: Retrieving updates. 8203-E4A*0666000: Installing updates. 8203-E4A*0666000 Managed System Primary: Preparing for install. 8203-E4A*0666000: Installing updates. 8203-E4A*0666000 Managed System Primary: Installing updates. 8203-E4A*0666000 Managed System Primary: Writing update files. 8203-E4A*0666000 Managed System Primary: Writing file 80a00020. 0 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 80a00701. 5959118 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 80a00701. 16043962 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 80a00711. 7858212 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 81cf0689. 0 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 81e00100. 2684818 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 81e00101. 2684818 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 81e00200. 65378 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 81e00200. 5304258 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 81e00400. 0 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file 81e00704. 0 bytes written. 8203-E4A*0666000 Managed System Primary: Writing file a0e00c21. 0 bytes written. 8203-E4A*0666000 Managed System Primary: Finished writing update files. 8203-E4A*0666000: Activating updates. 8203-E4A*0666000 Managed System Primary: Activating updates - Restarting Flexible Service Processor. 8203-E4A*0666000 Managed System Primary: Flexible Service Processor Restart completed successfully. 8203-E4A*0666000 Managed System Primary: Activation completed. 8203-E4A*0666000: Completed All Updates.
Virtual I/O Server install through the Hardware Management Console.
To install my Virtual I/O Server I decided to use the new integrated Hardware Management Console feature. This feature allows you to install the Virtual I/O Server through the Hardware Management Console the first time you boot it. Unfortunately you need to have all NIM ports opened because it’s just a graphical interface to the old well known installios command based on the old nimol product and running on the HMC. What a shame I was dreaming of a full integrated installation without the need of bootp/NFS and so on. So I’m very disappointed about this new feature :-(.
- You first have to download the Virtual I/O Server media to the Hardware Management Console, check you have enough space left on the HMC before importing images :
- Wait until images are correctly uploaded to the Hardware Management Console :
- If images were correctly uploaded on the HMC, you’ll see it on the Virtual I/O Server image repository :
- When you’ll boot your Virtual I/O Server the HMC will ask you if you want to install a Virtual I/O Server on it :
- The Virtual I/O Server will boot and you’ll be asked for the IP address to use and the image you want to install, in this case set the HMC ip as NIM server (the HMC is a nimol server):
- After this step the installation will begin and the output will be visible in the HMC window :
- When the installation is finished you can check log output HMC window :
Need a cheap SAN : Use iSCSI
This is the trickiest part of this post. My Power Server was not shipped with a split backplane my only solution was to create an iSCSI target and to boot my Virtual I/O Server on iSCSI, the iSCSI target partition was created on an AIX lpar and I decided to use file backed luns for backup convenience (I just have to copy the file to save my Virtual I/O Server rootvg :-))
Filesets for iSCSI target can be found on the AIX Expansion Pack DVD. You have to install these fileset on the iSCSI target before trying to configure it.
# installp -aXYgd . devices.tmiscsw.rte [..] # lslpp -Lc | grep devices.tmiscsw.rte devices.tmiscsw:devices.tmiscsw.rte:22.214.171.124: : :C: :iSCSI Target Software Device Driver : : : : : : :0:0:/:1241
- To create the target follow these steps :
# mkdev -c driver -s node -t tmsw tmsw0 Available # mkdev -c tmiscsi -s tmtarget -t target -p tmiscsi0 -a owner='tmsw0' -a iscsi_name="inq.priss.lab.chmod666.org" target0 Available
# dd if=/dev/zero of=/luns/lun1 bs=1m count=10k 10240+0 records in. 10240+0 records out. # ls -l /luns/lun1 -rw-r--r-- 1 root system 10737418240 Dec 14 10:22 /luns/lun1 # mkdev -c tmiscsi -s tmdev -t lu -p 'target1' -l 'lun1' -a back_dev_type='file' -a back_dev='lun1' -a back_file='/luns/lun1' lun1 Available # lsattr -El lun1 back_dev lun1 Backing Device Name True back_dev_option N/A True back_dev_type file Backing Device Type True back_file /luns/lun1 Backing File Full Path Name True dev_size N/A True lun_id 0x0 Logical Unit Number ID False queue_depth 3 Maximum Number of Commands to Queue to Device True
- Give a name to your initiator :
# chdev -l 'iscsi0' -a initiator_name='inq.deckard'
# echo "192.168.0.21 3260 inq.priss.lab.chmod666.org" > /etc/iscsi/targets # tail -1 /etc/iscsi/targets 192.168.0.21 3260 inq.priss.lab.chmod666.org # cfgmgr -vl iscsi0 # lspv hdisk0 00ce7e539a9ee2fb rootvg active hdisk1 00ce7e53d67b71ca None # lsdev -Cc disk | grep hdisk1 hdisk1 Available Other iSCSI Disk Drive
Boot on iSCSI
- When the menu comes up, choose Configure Network Disks (iSCSI) :
If like me you decided to boot on iSCSI, follow the same steps as above to configure the target. Then boot you partition on SMS menu, begin the installation. The configuration of the iSCSI initiator is performed on the BOS installation menu, follow the steps below to configure it, and then boot on iSCSI :
Welcome to Base Operating System Installation and Maintenance Type the number of your choice and press Enter. Choice is indicated by >>>. >>> 1 Start Install Now with Default Settings 2 Change/Show Installation Settings and Install 3 Start Maintenance Mode for System Recovery 4 Configure Network Disks (iSCSI) 5 Select Storage Adapters 88 Help ? 99 Previous Menu >>> Choice : 4
Configure iSCSI Move cursor to desired item and press Enter. Configure iSCSI Network Utilities
iSCSI Configuration -- SW Initiator Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] * Ethernet Interface en0 * IP Address of Ethernet Interface 192.168.0.6 * Network Mask of Ethernet Interface 255.255.255.0 * Gateway to iSCSI Target 192.168.0.254 * iSCSI Target Name [inq.priss] * IP Address of iSCSI Target [192.168.0.21] * Port Number of iSCSI Target  * iSCSI Initiator Name [inq.deckard]
COMMAND STATUS Command: OK stdout: yes stderr: no Before command completion, additional instructions may appear below. en0 changed inet0 changed iscsi0 changed hdisk0 Available N/A hdisk1 Available N/A hdisk2 Available N/A hdisk3 Available N/A hdisk4 Available N/A
# bootlist -o -m normal hdisk0 # bootlist -o -m normal -v 'ibm,max-boot-devices' = 0x5 NVRAM variable: (boot-device=/lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=192.168.0.6,subnet-mask=255.255.255.0,itname=inq.deckard,iport=3260,ilun=0,iname=inq.priss,siaddr=192.168.0.21,2) Path name: (/lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=192.168.0.6,subnet-mask=255.255.255.0,itname=inq.deckard,iport=3260,ilun=0,iname=inq.priss,siaddr=192.168.0.21,2) match_specific_info: ut=disk/iscsi/osdisk hdisk0
0 > printenv [..] boot-device /lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=126.96.36.199,subnet-mask=255.255.255.0,itname=iqn.localhost.hostid.00000000,iport=3260,ilun=0,iname=inq.priss,siaddr=188.8.131.52,2 [..] 0 > setenv boot-device /lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=192.168.0.6,subnet-mask=255.255.255.0,itname=inq.deckard,,iport=3260,ilun=0,iname=inq.priss,siaddr=192.168.0.21,2 ok 0 > printenv [..] boot-device /lhea@23c00200/ethernet@23e00200:iscsi,ciaddr=192.168.0.6,subnet-mask=255.255.255.0,itname=inq.deckard,,iport=3260,ilun=0,iname=inq.priss,siaddr=192.168.0.21,2 [..] 0 > boot -v iSCSI BOOT --------------------------------------------------- Server IP.....................192.168.0.21 Client IP.....................192.168.0.6 Subnet Mask...................255.255.255.0 iSCSI Initiator...............inq.deckard iSCSI Target..................inq.priss Target Port...................3260 Target LUN....................0 \ Elapsed time since release of system processors: 45 mins 24 secs
If you are planning to buy your own Power Server don’t hesitate to contact me if you have any question. It really gives me the opportunity to test things I was not able to do at work, PowerHA migration for 6 to 7, PowerSC, some HMC features, some NIM features (suma), the possibility are infinite. If you want to contribute to the rising of chmod666.org all over the AIX world, I’m now looking for :
- A Cisco manageable switch with ios and 16 ports.
- A Power7 based server (any model).
- An old DS* array.
- A split backplane.
- Any old hardware.
Don’t hesitate to contact me if you are trashing old machines/switch I’m interested if you want to make me gifts :-), I can buy it if you do not want to give it :-). Once again I hope this post will help you or will make you dream. I’m living in fear it’s hard to assume :-(.