Procedure:OnDemandSetup
This page lists various security permission that are involved with OOD,
https://osc.github.io/ood-documentation/latest/installation/modify-system-security.html
It is necessary to permit all to access ports 80, 443 and 5554 (the OOD authenticator)/tcp.
https://leo.leung.xyz/wiki/Open_OnDemand contains a lot of potential underdocumented bugfixes / setup BS.
Step -1: Apparent working user access / bug bypass procedure
Current status: It APPEARS that the OOD desktop launch process, in and of itself, works. The browser VNC viewer fails due to problems on ood itself; Any attempt to use the ood vnc viewer provokes the following in the logs,
==> httpd/access_ssl.log <== 128.223.202.205 - erik-k [08/Jul/2022:16:45:15 -0700] "GET /pun/sys/dashboard/batch_connect/sessions.js?_=1657318165288 HTTP/1.1" 200 5620 "https://ood.nic.uoregon.edu/pun/sys/dashboard/batch_connect/sessions" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0" 128.223.202.205 - - [08/Jul/2022:16:45:16 -0700] "GET /rnode/axis1/44232/websockify HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
So clearly something is wrong with websockify setup on the server.
However: The normal login and desktop (not desktop-gnome) process succeeds and does launch a VNC viewer on axis1 (note - the cluster node setup process has only been carried out on axis1 until a procedure for a fully working environment is decrypted Thus this will only work if axis1 is idle, for the moment).
At this point, the user may SSH to orthus using
ssh orthus.nic.uoregon.edu -L3456:axis1.stor:590X
where X is determined from the connection.yml that is reachable via OOD by clicking your session ID once it starts. Then connect a VNC viewer to localhost:3456 with the password that is also in the connection.yml file.
Step 3 - Authentication setup
- set ssl cert file and keyfile at ood_config.yml:36
- under dex:, set ssl: trye
- utilize most of their given template for LDAP connector, with some minor changes required for our ldap CN/DN/DCs
- During testing, set OIDCSSLValidateServer Off under httpd conf.d/ood-portal.conf or it will not work
At this point OOD is online/available and login works but apps are not configured.
Step 4 - cluster configuration
https://osc.github.io/ood-documentation/latest/installation/add-cluster-config.html
followed by slurm instructions at
https://osc.github.io/ood-documentation/latest/installation/resource-manager/slurm.html
per config documents, we need to setup munge/munged and the cluster munge key as well. It appears this system is able to access the cluster scheduler info outputs.
Step 5 - desktop configuration
https://osc.github.io/ood-documentation/master/enable-desktops/add-cluster.html
Cluster has been added to desktop environment settings for OOD, now complains about finding slurm. So Slurm must be installed on OOD!
Compilation & installation of slurm-21.08.4 on OOD at the same directory as on Orthus.
-> slurm config will need to add OOD to config?
setting up slurm config on ood:
- must have munge-devel installed or it will silently fail to build munge support. One might perhaps speculate that this calls for something more than a tiny, buried not-even-an-error message given how critical munge is to 99% of slurm clusters...
- slurm install path must link to ../etc for slurm bins to find config info
Slurm now installed on OOD. Slurm configuration updated to have OOD in the slurm cluster.
Started mysql on orthus to get slurmdbd working because without it slurm does not work on ood...
Working at https://osc.github.io/ood-documentation/master/enable-desktops/add-cluster.html ,
Desktop will now attempt to start however we have an error in the *vnc launch command* of all things,
-> It turns out that OOD does not like tigervnc -> Install the turbovnc library and install turbovnc,
- Contents of /etc/yum.repos.d/TurboVNC.repo:
[TurboVNC]
name=TurboVNC official RPMs
baseurl=https://sourceforge.net/projects/turbovnc/files
gpgcheck=1
gpgkey=https://sourceforge.net/projects/turbovnc/files/VGL-GPG-KEY
https://sourceforge.net/projects/turbovnc/files/VGL-GPG-KEY-1024
enabled=1
Still fails at
VNC_OUT=$(vncserver -log "vnc.log" -rfbauth "vnc.passwd" -nohttpd -noxstartup -geometry 800x600 -idletimeout 0 2>&1)
in the job script with
Could not start Xvnc. Unrecognized option: -nohttpd
Ugh, it appears this has been removed from turbovnc as of late. Grep Sledgehammer to the rescue,
[root@ood /]# grep nohttpd * -rn
opt/ood/ondemand/root/usr/share/gems/2.7/ondemand/2.0.23/gems/ood_core-0.17.6/lib/ood_core/batch_connect/templates/vnc.rb:90: VNC_OUT=$(vncserver -log "#{vnc_log}" -rfbauth "#{vnc_passwd}" -nohttpd -noxstartup #{vnc_args} 2>&1)
opt/ood/ondemand/root/usr/share/gems/2.7/ondemand/2.0.23/gems/ood_core-0.18.1/lib/ood_core/batch_connect/templates/vnc.rb:90: VNC_OUT=$(vncserver -log "#{vnc_log}" -rfbauth "#{vnc_passwd}" -nohttpd -noxstartup #{vnc_args} 2>&1)
Removing these and will re-try once axis1 is unused (I have only test installed the turbovnc system on axis1).
Okay, tested with -nohttpd removed and some progress,
now apparently require websockify to be installed on axis node. For test purposes, installing this manually on axis1.
https://pypi.org/project/websockify/
A better solution is needed for the rest of the nodes...
websockify
This does actually need to be modified. Current content of /etc/ood/config/clusters.d/axis.yml,
v2:
metadata:
title: "Axis cluster"
login:
host: "orthus.nic.uoregon.edu"
job:
adapter: "slurm"
cluster: "axis"
bin: "/opt/slurm/current/bin"
conf: "/opt/slurm/etc/slurm.conf"
batch_connect:
vnc:
script_wrapper: |
module purge
export PATH="/opt/TurboVNC/bin:$PATH"
export WEBSOCKIFY_CMD="/usr/local/bin/websockify"
%s
This sets the path to turbovnc and websockify.
gnome desktop
Under /etc/ood/config/apps/bc_desktop/ must create a new axis_gnome.yml that sets the default desktop to gnome not mate,
title: "Axis | Gnome Desktop" cluster: "axis" form: - desktop attributes: desktop: "gnome"
This gets us far enough that the desktop does actually claim to start and run which is further than we've ever gotten before.