HMC AIX Setup

This page will document the installation procedure for a system controlled by an IBM (v)HMC.

The installation steps are in summary,

• Install VIOS
• Create virtual network bridges
• Create virtual disks and assign
• Install AIX
• Setup AIX

Hardware prerequisites assumed: - At least 1 storage device on SATA/SAS - Network port connected to HMC - At least 2 external network ports connected, 1 to public & 1 to private network

VIOS install

The VIOS install uses a VIOS image stored on the HMC to bootstrap the server. Once the HMC is installed, it will be necessary to SSH to it, and use

chfs -a size=+10G /

to grow the default filesystem, create a directory on the HMC, and scp the vios-* image to there. The current (3.1) VIOS image is stored in ~erik-k's downloads as

PwrVMVIOSBIV3.1.3.10Fls92021.iso

References:

https://www.ibm.com/docs/en/power9?topic=hmc-installing-vios

Top -> System -> 'create vio server'

Clicking through menus, assign 2 processors, the NICs and at least one SAS adapter with disk (vios will not install on nvme).

Select 'management console images' to install and 'vios-3.1.3.10-flash'. If an image isn't present, see top of this section to upload it.

It is necessary to assign a working ethernet port to the vios. All 'm c image install' does is just temporarily throw up a NIM server, install and then tear it down. Make certain that the assigned network port is on the correct switch & that the switch has that port on the right vlan.

I have decided to use 172.17.20.x for the VIOS ethernets. Enter 172.17.202.79 (vina IP) for gateway... this does not work but it needs something.

Click install, and go have lunch. I've clocked this process in at around 45 minutes. It may be that temporarily assigning more processors would speed it up?

The finishing of the install procedure is unreliable. It may or may not report "done". As long as it does _not_ report failure, once it is done, you can try to click 'accept license.'

Next step:

Server -> vio servers -> [click server] -> vios actions -> console -> open terminal

After a bit, a godawfully barely-functional console will ask to run.

Enter a password

Accept license.

Run oem_setup_env to get prompted again, and accept license.

Okay, new blank slate VIOS is installed.

Partitions

At this point we also want to create OS partitions because we will need them available when we setup virtual disks

System -> top 'create partition' -> name it

VIOS configuration

Virtual networking configuration: https://www.ibm.com/docs/en/power9?topic=avnw-adding-virtual-network-by-creating-virtual-network-bridge

System -> powervm -> virtual networks -> add virtual network

name: brpriv Bridged: yes, tagged no, pvid 172, use default switch, next

Enable jumbo frames [critical for private network!] and LSO, next

Assign backing device from list. Note: This must be the correct device. The virbr setup process rewrites the MTU for the backing device when jumbo frames are enabled, and you *can't* reset it from inside the vios easily it seems. If this is chosen wrong, the only apparent option is delete the whole virbr and start over.

It will be created and use the default 802.3 virtual switch.

Now go back and greate the public bridge:

system -> powervm -> virtual networks -> add virtual network

name: brpub Bridged: yes, tagged no, pvid 128, check 'advanced' and use new virtual switch, next

ok, create

Virtual install library setup

First we need to import OS install images to the VIOS (much like we used the hmc to bootstrap the vios, the vios needs the image to bootstrap the partition).

SSH to the vios, username 'padmin'

oem_setup-env
mkdir -p /Maingroup/images
cd /Maingroup/images
chfs -a size=+15G / # to enlarge storage sufficiently

using scp, ISO images are available on erik-k's downloads:

aix_7200-05-03-2136_flash_092021.iso
rhel-8.2-ppc64le-dvd.iso
ubuntu-20.04.1-live-server-ppc64el.iso

SCP these to to /Maingroup/images

Now go to system -> virtual storage -> vio server -> manage

optical devices -> create virtual library -> 25GB

optical devices -> action -> add media -> from existing file

/Maingrouop/images/ubuntu-20.04.1-liver-server-ppc64el.iso e.g.

Quite insanely you have to type the entire filename manually, there is no browser box. *blink blink*.

Partition configuration

systems -> powervm -> virtual storage

select vio server -> action -> manage

click storage pools -> Create a storage pool & assign the NVMEs to it

click virtual disks -> create

name: 'aixroot' or something pool: nvmepool size: 500G e.g. assign to partition: aix-part

create & assign adapter on partition -> yes please

Now click system -> partitions -> [partition] -> virtual networks -> attach virtual network

[X] show and attaach new adapters [X] check brpub and brpriv to connect to both networks

Click system -> virtual storage -> [vios] -> manage

virtual optical devices -> [select image] -> modify assignment to partition we're installing -> ok

AIX install

After assigning virtual networks, virtual disk drive & aix-7.2 install media,

system -> partitions -> aix partition -> start

system -> partitions -> aix partition -> system actions -> console -> open

AIX INSTALL CRITICAL: Select software options, and install both openssh client and server, or you'll be kicking yourself in the balls to distract yourself from the pain while you try to find another way to install it after.

Look at the devices and confirm you have a scsi disk and a cd drive

AIX configuration

Upon firstboot, smitty comes up.

Set timezone to use/los angeles Use passwd to set password

Run

df -sm

to see what an infinitesimal size of disk has been assigned to a bunch of partitions. Use

chfs -a size=+xG /filesystem

to grow them. Remember we have 500G of virtual disk!

Ethernet configuration

lsdev | grep -i eth -> should print en0 and en1

cd /etc vi dhcpcd.ini

Page down to the bottom, append,

interface en0 {
  option 12 "cirrus-aix"
}
interface en1 {
  option 12 "cirrus-aix"
}

but of course use the right hostname.

Use system -> virtual networking diagram to find which interface is connected to which network.

Use ifconfig -a to get MAC addresses

Edit named on cato and dhcpd on mnemosyne to make sure DNS and IP assignment work correctly.

Once this is done,

startsrc -s dhcpcd

to connect and get IP addresses.

Now we have to configure jumbo frames on the host,

ifconfig -a

will reveal which adapter is mated to the private network, N

See: https://developer.ibm.com/articles/au-aix-largesend-jumboframes/

If N is 0, this can be pasted - as one line - to restart the sucker and set jumbo frames on,

chdev -l en0 -a state=down; chdev -l en0 -a state=detatch; chdev -l ent0 -a jumbo_frames=yes; chdev -l en0 -a mtu=9000; chdev -l en0 -a state=up; mkdev -l inet0

Check

lsattr -El (device) | grep -e mtu -e jumbo

Check afterwards from another oaciss machine (because aix ping command is stupid) too:

orthus# ping -s 9000 cumulus-aix.stor

NFS

chnfsdom .stor
startsrc -s nfsrgyd
nfso -p -o nfs_use_reserved_ports=1

The file system mount creation commands:

mknfsmnt -f /home/users -d /vol/users -h 172.17.202.252 -M 'sys' -B -A -t rw -w bg -K 4 -k tcp
mknfsmnt -f /packages -d /mnt/packtree/aix72 -h 172.17.202.252 -M 'sys' -B -A -t rw -w bg -K 4 -k tcp

At this point

startsrc -s nfs

should work

This will _massively_ simplify moving data back and forth!