Procedure:OnDemandSetup

This page lists various security permission that are involved with OOD,

https://osc.github.io/ood-documentation/latest/installation/modify-system-security.html

It is necessary to permit all to access ports 80, 443 and 5554 (the OOD authenticator)/tcp.

https://leo.leung.xyz/wiki/Open_OnDemand contains a lot of potential underdocumented bugfixes / setup BS.

Step -1: NoVNC connection failure debug

If the desktop launches (i.e. squeue shows a sustained job running on the host for more than 10 seconds) but NoVNC fails to connect, something like this is probably going to be showing up on OOD at /var/log/apache2/ood.nic.uoregon.edu_access_ssl.log:

73.157.215.248 - - [21/Mar/2025:16:24:06 +0000] "GET /rnode/voltar.stor/39352/websockify HTTP/1.1" 404 1212 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0"

This is because the host in question was not added to the rnode_uri's regular expression filter in ood_config.yml as described in host setup below, so OOD is refusing to act as a proxy for this node.

Step 3 - Authentication setup

• set ssl cert file and keyfile at ood_config.yml:36
• under dex:, set ssl: true
• utilize most of their given template for LDAP connector, with some minor changes required for our ldap CN/DN/DCs
• Must set OIDCSSLValidateServer Off under httpd conf.d/ood-portal.conf after generating or it will not work (self-signed ldap1 cert)

At this point OOD is online/available and login works

Step 4 - cluster configuration

https://osc.github.io/ood-documentation/latest/installation/add-cluster-config.html

followed by slurm instructions at

https://osc.github.io/ood-documentation/latest/installation/resource-manager/slurm.html

per config documents, we need to setup munge/munged and the cluster munge key as well. It appears this system is able to access the cluster scheduler info outputs.

setting up slurm config on ood:

• must have munge-devel installed or it will silently fail to build munge support. One might perhaps speculate that this calls for something more than a tiny, buried not-even-an-error message given how critical munge is to 99% of slurm clusters...
• slurm install path must link to ../etc for slurm bins to find config info

Slurm is built at /opt/slurm/version__ as on other systems.

Step 5 - cluster node software setup

Two nonstandard packages are needed on the remote desktop hosts, tigervnc and websockify.

Repo for turbovnc:

[root@voltar yum.repos.d]# cat TurboVNC.repo 
[TurboVNC]
name=TurboVNC official RPMs
baseurl=https://sourceforge.net/projects/turbovnc/files
gpgcheck=1
gpgkey=https://sourceforge.net/projects/turbovnc/files/VGL-GPG-KEY
       https://sourceforge.net/projects/turbovnc/files/VGL-GPG-KEY-1024
enabled=1

Commands:

yum install python3-websockify turbovnc
# The below is due to how ood cluster launcher script on OOD at root@ood3:/etc/ood/config/clusters.d is configured
# which is in turn because of how websockify ended up installed on the axis nodes so we're default stuck with it now
cd /usr/local/bin
ln -s /usr/bin/websockify ./websockify

These will install the needed packages to make the thing go.

Step 5 - desktop configuration on the cluster

https://osc.github.io/ood-documentation/master/enable-desktops/add-cluster.html

Desktop will now attempt to start however we have an error in the *vnc launch command* of all things,

-> It turns out that OOD does not like tigervnc -> Install TurboVNC

By default it will fail at

VNC_OUT=$(vncserver -log "vnc.log" -rfbauth "vnc.passwd" -nohttpd -noxstartup -geometry 800x600 -idletimeout 0  2>&1)

in the job script with

 Could not start Xvnc.

Unrecognized option: -nohttpd

Ugh, it appears this has been removed from turbovnc as of late. Grep Sledgehammer to the rescue,

[root@ood /]# grep nohttpd * -rn
opt/ood/ondemand/root/usr/share/gems/2.7/ondemand/2.0.23/gems/ood_core-0.17.6/lib/ood_core/batch_connect/templates/vnc.rb:90:                VNC_OUT=$(vncserver -log "#{vnc_log}" -rfbauth "#{vnc_passwd}" -nohttpd -noxstartup #{vnc_args} 2>&1)
opt/ood/ondemand/root/usr/share/gems/2.7/ondemand/2.0.23/gems/ood_core-0.18.1/lib/ood_core/batch_connect/templates/vnc.rb:90:                VNC_OUT=$(vncserver -log "#{vnc_log}" -rfbauth "#{vnc_passwd}" -nohttpd -noxstartup #{vnc_args} 2>&1)

The invocation now works.

Next step at https://osc.github.io/ood-documentation/latest/app-development/interactive/setup/modify-cluster-configuration.html does need to be modified.

This does actually need to be modified. Current content of /etc/ood/config/clusters.d/axis.yml,

v2:
  metadata:
    title: "Axis cluster"
  login:
    host: "orthus.nic.uoregon.edu"
  job:
    adapter: "slurm"
    cluster: "axis"
    bin: "/opt/slurm/current/bin"
    conf: "/opt/slurm/etc/slurm.conf"
  batch_connect:
    basic:
      script_wrapper: |
        module purge
        %s
      set_host: "host=$(hostname -A | awk '{print $1}')"
    vnc:
      script_wrapper: |
        module purge
        export PATH="/opt/TurboVNC/bin:$PATH"
        export WEBSOCKIFY_CMD="/usr/local/bin/websockify"
        %s
      set_host: "host=$(hostname -A | awk '{print $1}')"

This sets the path to turbovnc and websockify to make sure they launch and sets the hostname (out of those that can be chosen).

The correct (yum) installation of python3-websockify puts it in /usr/bin/websockify. Because existing config specs /usr/local/bin, just symlink

gnome desktop

Under /etc/ood/config/apps/bc_desktop/ must create a new axis_gnome.yml that sets the default desktop to gnome not mate,

title: "Axis | Gnome Desktop"
cluster: "axis"
form:
  - desktop
attributes:
  desktop: "gnome"