Rocm on power9 - Revision history

Nic-wiki: /* 8.1 - install HCC */

2021-01-28T01:09:16Z

8.1 - install HCC

← Older revision		Revision as of 01:09, 28 January 2021
Line 236:		Line 236:
	HCC configurator is unsmart, tell it things:		HCC configurator is unsmart, tell it things:

	<pre>cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hcc/git -DHSA_HEADER=/packages/rocm/git/include/hsa/hsa.h -DHSA_LIBRARY=/packages/rocm/git/lib/libhsa-runtime64.so -DROCM_INSTALL_PATH=/packages/rocm/git -DROCM_ROOT=/packages/rocm/git</~~code~~>		<pre>cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hcc/git -DHSA_HEADER=/packages/rocm/git/include/hsa/hsa.h -DHSA_LIBRARY=/packages/rocm/git/lib/libhsa-runtime64.so -DROCM_INSTALL_PATH=/packages/rocm/git -DROCM_ROOT=/packages/rocm/git</pre>

			The idiotcmake is hardcoded to look for /usr/bin/cc and will ignore the result of 'which gcc' and $CC (same c++). Thus on some platforms (glares at RHEL-7) it will be necessary to add
			<pre>-DCMAKE_C_COMPILER=$(which gcc) -DCMAKE_CXX_COMPILER=$(which g++)</pre>
			to the above.

	There are a few more edits that are evidently required:		There are a few more edits that are evidently required:

	<~~code~~>build/include/kalmar_runtime.h:		<pre>build/include/kalmar_runtime.h:
	#define _Float16 __fp16 </~~code~~>		#define _Float16 __fp16 </pre>

	from: https://reviews.llvm.org/D57188		from: https://reviews.llvm.org/D57188

	<~~code~~> /packages/rocm/git/hsa/include/hsa/hsa.h: insert		<pre> /packages/rocm/git/hsa/include/hsa/hsa.h: insert
	#define LITTLEENDIAN_CPU		#define LITTLEENDIAN_CPU
	</~~code~~>		</pre>

	Without these compilation will fail riiiight at the veeeeery end.		Without these compilation will fail riiiight at the veeeeery end.

	<~~code~~>make -j8		<pre>make -j8
	make install</pre>		make install</pre>

	Note that building this includes building LLVM, which has some mind-blowingly massive link stages (as in using 10GB of ram each massive), so if the machine does not have a gigantic amount of memory, constraining -j may be recommended.		Note that building this includes building LLVM, which has some mind-blowingly massive link stages (as in using 10GB of ram each massive), so if the machine does not have a gigantic amount of memory, constraining -j may be recommended.


	=== 8.2 - Install HIP ===		=== 8.2 - Install HIP ===

Nic-wiki at 01:57, 12 November 2020

2020-11-12T01:57:54Z

@@ Line 8: / Line 8: @@
 The desired parameters of the installation are:
 * Installation root: /packages/rocm/git (part of the ppc64le packages tree)
 = Compilation process =

Nic-wiki: /* 8.2 - Install HIP */

2020-09-28T22:00:54Z

8.2 - Install HIP

← Older revision		Revision as of 22:00, 28 September 2020
Line 232:		Line 232:

	=== 8.2 - Install HIP ===		=== 8.2 - Install HIP ===

			spoiler - skip to end to bypass my stream of consciousness and get the build instructions

	Download code and set install options:		Download code and set install options:

Nic-wiki: /* 8 - Install HIP runtime */

2020-09-28T22:00:19Z

8 - Install HIP runtime

← Older revision		Revision as of 22:00, 28 September 2020
Line 285:		Line 285:
	set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${PROF_API_GEN} ${PROF_API_HDR} ${PROF_API_STR})</pre>		set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${PROF_API_GEN} ${PROF_API_HDR} ${PROF_API_STR})</pre>

			Relevant discovery after manual execution: File is hardcoded to '/usr/bin/python' and ubuntu20 has python3 binary.

			Changed ../hip_prof_gen.py interpreter line and retrying cmake.

			SUCCESS!

			Script to build/install:

			<pre>cd $TLD
			git clone https://github.com/ROCm-Developer-Tools/HIP.git
			cd HIP</pre>

			Edit hip_prof_gen.py line 1 to set interpreter to '/usr/bin/python3'

			<pre>mkdir -p build; cd build
			CC=clang CXX=clang++ cmake -DLLVM_DIR=/packages/hcc/git -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hip/git -DHSA_PATH=/packages/rocm/git/hsa -DROCM_PATH=/packages/git/rocm -DUSE_PROF_API=1 -DHIP_PLATFORM=hcc ..
			make -j8
			make install</pre>

			This directory is likely to be important: <pre>/packages/hip/git/cmake</pre>

			HIP modulefile:

			<pre>#%Module########################################
			##
			## HIP system
			##

			set _module_name [module-info name]

			module-whatis "The AMD HIP compiler via Git"

			proc ModulesHelp { } {
			global _module_name
			puts stderr "The $_module_name modulefile defines the default system paths"
			puts stderr "and environment variables needed to use HIP."
			}

			#module-verbosity on

			set _module_name [module-info name]
			set is_module_rm [module-info mode remove]
			set sys [uname sysname]
			set os [uname release]
			set mach [uname machine]

			if {[is-loaded gcc] && !$is_module_rm } { module unload gcc }

			module load rocm/git
			module load z3/git
			module load hcc/git_3.1

			set SWTOP /packages/hip/git

			prepend-path PATH $SWTOP/bin

			prepend-path LD_LIBRARY_PATH $SWTOP/lib

			prepend-path C_INCLUDE_PATH $SWTOP/include
			prepend-path CPLUS_INCLUDE_PATH $SWTOP/include</pre>

	== 9 - roctracer ==		== 9 - roctracer ==

Nic-wiki at 19:26, 28 September 2020

2020-09-28T19:26:25Z

@@ Line 4: / Line 4: @@
 = Development testbed parameters =
-The testbed that is being used to prepare this system is [[Compute:_Pike || Pike]]. This is currently our only Power9 machine that has an AMD GPU in it. It is running Ubuntu 20-LTS.
+The testbed that is being used to prepare this system is [[Compute:_Pike | Pike]]. This is currently our only Power9 machine that has an AMD GPU in it. It is running Ubuntu 20-LTS.
 The desired parameters of the installation are:
@@ Line 253: / Line 253: @@
 Further additions to the cmake configure line:
 <pre>CC=clang CXX=clang++ cmake -DLLVM_DIR=/packages/hcc/git -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hip/git -DHSA_PATH=/packages/rocm/git/hsa -DROCM_PATH=/packages/git/rocm -DUSE_PROF_API=1 -DHIP_PLATFORM=hcc ..</pre>
 == 9 - roctracer ==

Nic-wiki: /* 8.2 - Install HIP */

2020-09-27T06:17:29Z

8.2 - Install HIP

← Older revision		Revision as of 06:17, 27 September 2020
Line 243:		Line 243:
	There is a reason for -j1. It barfs because of the _Float16 problem in the installed header. Fix it:		There is a reason for -j1. It barfs because of the _Float16 problem in the installed header. Fix it:

	<~~code~~>vi /packages/hcc/git/include/kalmar_math.h		<pre>vi /packages/hcc/git/include/kalmar_math.h
	#define _Float16 __fp16 </~~code~~>		#define _Float16 __fp16 </pre>

	Oh, this is nice:		Oh, this is nice:

	<code>fatal error: 'hip/hcc_detail/hip_prof_str.h' file not found</code>		<code>fatal error: 'hip/hcc_detail/hip_prof_str.h' file not found</code>

			https://github.com/ROCm-Developer-Tools/HIP/issues/999

			Further additions to the cmake configure line:
			<pre>CC=clang CXX=clang++ cmake -DLLVM_DIR=/packages/hcc/git -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hip/git -DHSA_PATH=/packages/rocm/git/hsa -DROCM_PATH=/packages/git/rocm -DUSE_PROF_API=1 -DHIP_PLATFORM=hcc ..</pre>

	== 9 - roctracer ==		== 9 - roctracer ==

Nic-wiki: /* 8 - Install HIP runtime */

2020-09-27T06:15:08Z

8 - Install HIP runtime

← Older revision		Revision as of 06:15, 27 September 2020
Line 229:		Line 229:

	Note that building this includes building LLVM, which has some mind-blowingly massive link stages (as in using 10GB of ram each massive), so if the machine does not have a gigantic amount of memory, constraining -j may be recommended.		Note that building this includes building LLVM, which has some mind-blowingly massive link stages (as in using 10GB of ram each massive), so if the machine does not have a gigantic amount of memory, constraining -j may be recommended.


			=== 8.2 - Install HIP ===

			Download code and set install options:
			<pre>cd $TLD
			git clone https://github.com/ROCm-Developer-Tools/HIP.git
			cd HIP
			mkdir -p build; cd build
			CC=clang CXX=clang++ cmake -DLLVM_DIR=/packages/hcc/git -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hip/git -DHSA_PATH=/packages/rocm/git/hsa -DROCM_PATH=/packages/git/rocm ..
			make -j1</pre>

			There is a reason for -j1. It barfs because of the _Float16 problem in the installed header. Fix it:

			<code>vi /packages/hcc/git/include/kalmar_math.h
			#define _Float16 __fp16 </code>

			Oh, this is nice:

			<code>fatal error: 'hip/hcc_detail/hip_prof_str.h' file not found</code>

	== 9 - roctracer ==		== 9 - roctracer ==

Nic-wiki: /* 8 - Install HIP runtime */

2020-09-27T02:56:26Z

8 - Install HIP runtime

← Older revision		Revision as of 02:56, 27 September 2020
Line 156:		Line 156:
	cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/z3/git		cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/z3/git
	make -j24		make -j24
	~~cd ..; chgrp swmgr build-ppc64le -R; chmod g+w build-ppc64le -R~~		make install</pre>
	~~cd build-ppc64le;~~
	~~[swmgr] $~~ make install</pre>

	Z3 module for /packages/modulefiles/z3/git:		Z3 module for /packages/modulefiles/z3/git:
Line 212:		Line 210:
	HCC configurator is unsmart, tell it things:		HCC configurator is unsmart, tell it things:

	<pre>cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hcc/git -DHSA_HEADER=/packages/rocm/git/include/hsa/hsa.h -DHSA_LIBRARY=/packages/rocm/git/lib/libhsa-runtime64.so -DROCM_INSTALL_PATH=/packages/rocm/git -DROCM_ROOT=/packages/rocm/git		<pre>cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hcc/git -DHSA_HEADER=/packages/rocm/git/include/hsa/hsa.h -DHSA_LIBRARY=/packages/rocm/git/lib/libhsa-runtime64.so -DROCM_INSTALL_PATH=/packages/rocm/git -DROCM_ROOT=/packages/rocm/git</code>
	~~make -j8~~
	cd ..~~; chgrp swmgr build -R; chmod g+w build;~~		There are a few more edits that are evidently required:
	~~(sudo to swmgr)~~
	~~[swmgr] $~~ make ~~install~~ -j8</pre>		<code>build/include/kalmar_runtime.h:
			#define _Float16 __fp16 </code>

			from: https://reviews.llvm.org/D57188

			<code> /packages/rocm/git/hsa/include/hsa/hsa.h: insert
			#define LITTLEENDIAN_CPU
			</code>

			Without these compilation will fail riiiight at the veeeeery end.

			<code>make -j8
			make install</pre>

	Note that building this includes building LLVM, which has some mind-blowingly massive link stages (as in using 10GB of ram each massive), so if the machine does not have a gigantic amount of memory, constraining -j may be recommended.		Note that building this includes building LLVM, which has some mind-blowingly massive link stages (as in using 10GB of ram each massive), so if the machine does not have a gigantic amount of memory, constraining -j may be recommended.

Nic-wiki: /* 8 - Install HCC runtime */

2020-09-27T01:20:43Z

8 - Install HCC runtime

← Older revision		Revision as of 01:20, 27 September 2020
Line 148:		Line 148:
	== 8 - Install HIP runtime ==		== 8 - Install HIP runtime ==

	==~~= 8~~.1 - install ~~HCC ===~~		Prerequisites:
			* Recommend to install Z3 theorem prover (LLVM and by extension HCC want it).
			* apt install libxml2 libxml2-dev

			<pre>cd z3
			mkdir -p build-ppc64le; cd build-ppc64le
			cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/z3/git
			make -j24
			cd ..; chgrp swmgr build-ppc64le -R; chmod g+w build-ppc64le -R
			cd build-ppc64le;
			[swmgr] $ make install</pre>

			Z3 module for /packages/modulefiles/z3/git:

			<pre>#%Module########################################
			##
			## Z3 theorem prover
			##

			set _module_name [module-info name]

			module-whatis "The Z3 theorem prover from MS Research"

			proc ModulesHelp { } {
			global _module_name
			puts stderr "The $_module_name modulefile defines the default system paths"
			puts stderr "and environment variables needed to use Z3."
			}

			##module-verbosity on

	~~Based upon instruction at:~~		set _module_name [module-info name]
			set is_module_rm [module-info mode remove]
			set sys [uname sysname]
			set os [uname release]
			set mach [uname machine]

	~~https:~~//~~rocmdocs.amd.com~~/~~en/latest/Installation_Guide/HCC-Compiler.html~~		set SWTOP /packages
			set ZTHREE $SWTOP/z3/git

	~~Fetch code. Find a nice article to read, this is going to take a hot minute. Or several.~~		prepend-path PATH $ZTHREE/bin

	~~<code>cd~~ $~~TLD~~		prepend-path LD_LIBRARY_PATH $ZTHREE/lib
	~~git clone --recursive -b clang_tot_upgrade https://github.com/RadeonOpenCompute/hcc.git<~~/~~code>~~

			prepend-path C_INCLUDE_PATH $ZTHREE/include
			prepend-path CPLUS_INCLUDE_PATH $ZTHREE/include</pre>

			Note that /lib differs from path in x86_64 path which is /lib64.

			=== 8.1 - install HCC ===

			Based upon instruction at:

			https://rocmdocs.amd.com/en/latest/Installation_Guide/HCC-Compiler.html

			Fetch the code. Find a nice paper to read, this isn't going to take just one hot minute...

	git clone https://github.com/RadeonOpenCompute/hcc.git		<pre>cd $TLD
	~~cd hcc;~~ mkdir -p build; cd build</pre>		git clone --recursive -b clang_tot_upgrade https://github.com/RadeonOpenCompute/hcc.git
			mkdir -p build; cd build
			</pre>

	~~... and now that needs its own personal clang apparently.~~		HCC configurator is unsmart, tell it things:

	=== 8.2 - ~~install HIP?~~ ===		<pre>cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/packages/hcc/git -DHSA_HEADER=/packages/rocm/git/include/hsa/hsa.h -DHSA_LIBRARY=/packages/rocm/git/lib/libhsa-runtime64.so -DROCM_INSTALL_PATH=/packages/rocm/git -DROCM_ROOT=/packages/rocm/git
			make -j8
			cd ..; chgrp swmgr build -R; chmod g+w build;
			(sudo to swmgr)
			[swmgr] $ make install -j8</pre>

			Note that building this includes building LLVM, which has some mind-blowingly massive link stages (as in using 10GB of ram each massive), so if the machine does not have a gigantic amount of memory, constraining -j may be recommended.

	== 9 - roctracer ==		== 9 - roctracer ==

Nic-wiki: /* 8.1 - install HCC */

2020-09-27T00:53:50Z

8.1 - install HCC

← Older revision		Revision as of 00:53, 27 September 2020
Line 150:		Line 150:
	=== 8.1 - install HCC ===		=== 8.1 - install HCC ===

	Fetch ~~and install:~~		Based upon instruction at:

			https://rocmdocs.amd.com/en/latest/Installation_Guide/HCC-Compiler.html

			Fetch code. Find a nice article to read, this is going to take a hot minute. Or several.

	<code>cd $TLD		<code>cd $TLD
			git clone --recursive -b clang_tot_upgrade https://github.com/RadeonOpenCompute/hcc.git</code>






	git clone https://github.com/RadeonOpenCompute/hcc.git		git clone https://github.com/RadeonOpenCompute/hcc.git
	cd hcc; mkdir -p build; cd build</pre>		cd hcc; mkdir -p build; cd build</pre>