ANSYS FLUENT
ANSYS FLUENT is available on the cluster nodes but because of space constraints it's not available on the submit host. It can be invoked as fluent after using the module load ansys command.
Documentation
Documentation for ANSYS FLUENT is available via the ANSYS customer support portal (you must register to get access to the portal, and this registration requires FEUP's ANSYS account number which we can't freely distribute - contact us to get this).
Running FLUENT
If you need to use FLUENT interactively, you can use interactive jobs as documented at this page (basically use the command qsub -I).
In both interactive and batch mode, FLUENT should always be invoked with the fluent -g command (the -g disables the graphical display which isn't normally available on the cluster nodes). This will start the FLUENT program which will then prompt for the solver version to use:
$ fluent -g /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/bin/fluent -r18.2.0 -g /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/cortex/lnamd64/cortex.18.2.0 -f fluent -g (fluent " -alnamd64 -r18.2.0 -t0 -path/soft/ANSYS/18.2/v182/fluent -ssh") /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/bin/fluent -r18.2.0 -alnamd64 -t0 -path/soft/ANSYS/18.2/v182/fluent -ssh -cx cfpsmall05.grid.fe.up.pt:37347:34395 The versions available in /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/lnamd64 are: 2d 2d_node 2ddp_host 3d 3d_node 3ddp_host 2d_host 2ddp 2ddp_node 3d_host 3ddp 3ddp_node The fluent process could not be started. version>
You can skip this prompt by providing the solver on the command line:
$ fluent -g 2d
which will place you directly in the command line prompt for FLUENT. You can now type your commands to drive FLUENT, or you can use a pre-created journal file.
Journal Files for batch processing
ANSYS FLUENT can be controlled non-interactively through a simple file called the journal. The journal contains a series of instructions in the ANSYS FLUENT TUI language which is described in more detail in Chapter 3 of the ANSYS FLUENT User's Guide. You can either manually create a journal file (with a text editor), or use the FLUENT GUI on your workstation to record the commands to run your analysis on a smaller data set. You do this by first starting FLUENT and then opening the menu File→Write→Start Journal (as shown in the screenshot below) and selecting the file to save the commands in. All of the operations that you do after this will be saved to this file. Once you have finished recording the commands you go File→Write→Stop Journal to make sure they are all recorded in the journal file. You can now modify the saved journal file to use the final data files and copy it to the cluster to run your analysis.
The simplest journal (in the file simple.journal) is just:
exit
you can then reference the journal from the command line using the -i argument to FLUENT:
$ fluent -g 2d -i simple.journal
/soft/ANSYS/18.2/v182/fluent/fluent18.2.0/bin/fluent -r18.2.0 -g 2d -i simple.journal
/soft/ANSYS/18.2/v182/fluent/fluent18.2.0/cortex/lnamd64/cortex.18.2.0 -f fluent -g -i simple.journal (fluent "2d -pshmem -host -alnamd64 -r18.2.0 -t1 -mpi=ibmmpi -path/soft/ANSYS/18.2/v182/fluent -ssh")
/soft/ANSYS/18.2/v182/fluent/fluent18.2.0/bin/fluent -r18.2.0 2d -pshmem -host -alnamd64 -t1 -mpi=ibmmpi -path/soft/ANSYS/18.2/v182/fluent -ssh -cx cfpsmall05.grid.fe.up.pt:39633:44138
Starting /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/lnamd64/2d_host/fluent.18.2.0 host -cx cfpsmall05.grid.fe.up.pt:39633:44138 "(list (rpsetvar (QUOTE parallel/function) "fluent 2d -flux -node -alnamd64 -r18.2.0 -t1 -pshmem -mpi=ibmmpi -ssh") (rpsetvar (QUOTE parallel/rhost) "") (rpsetvar (QUOTE parallel/ruser) "") (rpsetvar (QUOTE parallel/nprocs_string) "1") (rpsetvar (QUOTE parallel/auto-spawn?) #t) (rpsetvar (QUOTE parallel/trace-level) 0) (rpsetvar (QUOTE parallel/remote-shell) 1) (rpsetvar (QUOTE parallel/path) "/soft/ANSYS/18.2/v182/fluent") (rpsetvar (QUOTE parallel/hostsfile) "") )"
Welcome to ANSYS Fluent Release 18.2
Copyright 2017 SAS IP, Inc. All Rights Reserved.
Unauthorized use, distribution or duplication is prohibited.
This product is subject to U.S. laws governing export and re-export.
For full Legal Notice, see documentation.
Build Time: Jul 25 2017 20:10:41 EDT Build Id: 10098
--------------------------------------------------------------
This is an academic version of ANSYS FLUENT. Usage of this product
license is limited to the terms and conditions specified in your ANSYS
license form, additional terms section.
--------------------------------------------------------------
Host spawning Node 0 on machine "cfpsmall05" (unix).
/soft/ANSYS/18.2/v182/fluent/fluent18.2.0/bin/fluent -r18.2.0 2d -flux -node -alnamd64 -t1 -pshmem -mpi=ibmmpi -ssh -mport 192.168.147.149:192.168.147.149:34345:0
Starting /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/multiport/mpi/lnamd64/ibmmpi/bin/mpirun -e MPI_IBV_NO_FORK_SAFE=1 -e MPI_USE_MALLOPT_MMAP_MAX=0 -np 1 /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/lnamd64/2d_node/fluent_mpi.18.2.0 node -mpiw ibmmpi -pic shmem -mport 192.168.147.149:192.168.147.149:34345:0
-------------------------------------------------------------------------------
ID Hostname Core O.S. PID Vendor
-------------------------------------------------------------------------------
n0 cfpsmall05 1/8 Linux-64 24620 Intel(R) Xeon(R) E5430
host cfpsmall05 Linux-64 24438 Intel(R) Xeon(R) E5430
MPI Option Selected: ibmmpi
-------------------------------------------------------------------------------
Cleanup script file is /homes/jbarber/cleanup-fluent-cfpsmall05-24438.sh
+------------------------------------------------------------------+
| ANSYS Product Improvement |
| |
| ANSYS Product Improvement Program helps improve ANSYS |
| products. Participating in this program is like filling out a |
| survey. Without interrupting your work, the software reports |
| anonymous usage information such as errors, machine and |
| solver statistics, features used, etc. to ANSYS. We never |
| use the data to identify or contact you. |
| The data does NOT contain: |
| - Any personally identifiable information including names, |
| IP addresses, file names, part names, etc. |
| - Any information about your geometry or design specific |
| inputs. |
| You can stop participation at any time. To change your |
| selection go to Help >> ANSYS Product Improvement Program |
| in the GUI. |
| For more information about the ANSYS Privacy Policy, please |
| check: http://www.ansys.com/privacy |
| |
+------------------------------------------------------------------+
>
> exit
$
Parallel Jobs
The above examples show FLUENT being used with as single process, it is also possible to use more processes to speed up the analysis.
You can use more processes by changing the command line to use the -t, -ssh, and (optional) -cnf argument to FLUENT, and by requesting more nodes and processes-per-node from the cluster scheduler (described in more detail here).
Let's first cover the FLUENT command. The -ssh argument is required whenever you also use the -t argument to run a parallel job. The -t argument is used to specify how many processes in total you want to use, so if you wanted to use 16 processes you would use -t16. Finally, the -cnf argument is used to specify a text file that contains the hosts to use. The values for -t and -cnf are supplied by the queuing system as environment variables, so you can use the command:
$ fluent -g 2d -ssh -t$PBS_NP -i test.journal
/soft/ANSYS/18.2/v182/fluent/fluent18.2.0/bin/fluent -r18.2.0 -g 2d -ssh -t1 -i test.journal
/soft/ANSYS/18.2/v182/fluent/fluent18.2.0/cortex/lnamd64/cortex.18.2.0 -f fluent -g -i test.journal (fluent "2d -pshmem -host -alnamd64 -r18.2.0 -t1 -mpi=ibmmpi -path/soft/ANSYS/18.2/v182/fluent -ssh")
/soft/ANSYS/18.2/v182/fluent/fluent18.2.0/bin/fluent -r18.2.0 2d -pshmem -host -alnamd64 -t1 -mpi=ibmmpi -path/soft/ANSYS/18.2/v182/fluent -ssh -cx cfpsmall05.grid.fe.up.pt:45127:41088
Starting /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/lnamd64/2d_host/fluent.18.2.0 host -cx cfpsmall05.grid.fe.up.pt:45127:41088 "(list (rpsetvar (QUOTE parallel/function) "fluent 2d -flux -node -alnamd64 -r18.2.0 -t1 -pshmem -mpi=ibmmpi -ssh") (rpsetvar (QUOTE parallel/rhost) "") (rpsetvar (QUOTE parallel/ruser) "") (rpsetvar (QUOTE parallel/nprocs_string) "1") (rpsetvar (QUOTE parallel/auto-spawn?) #t) (rpsetvar (QUOTE parallel/trace-level) 0) (rpsetvar (QUOTE parallel/remote-shell) 1) (rpsetvar (QUOTE parallel/path) "/soft/ANSYS/18.2/v182/fluent") (rpsetvar (QUOTE parallel/hostsfile) "") )"
Welcome to ANSYS Fluent Release 18.2
Copyright 2017 SAS IP, Inc. All Rights Reserved.
Unauthorized use, distribution or duplication is prohibited.
This product is subject to U.S. laws governing export and re-export.
For full Legal Notice, see documentation.
Build Time: Jul 25 2017 20:10:41 EDT Build Id: 10098
--------------------------------------------------------------
This is an academic version of ANSYS FLUENT. Usage of this product
license is limited to the terms and conditions specified in your ANSYS
license form, additional terms section.
--------------------------------------------------------------
Host spawning Node 0 on machine "cfpsmall05" (unix).
/soft/ANSYS/18.2/v182/fluent/fluent18.2.0/bin/fluent -r18.2.0 2d -flux -node -alnamd64 -t1 -pshmem -mpi=ibmmpi -ssh -mport 192.168.147.149:192.168.147.149:36981:0
Starting /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/multiport/mpi/lnamd64/ibmmpi/bin/mpirun -e MPI_IBV_NO_FORK_SAFE=1 -e MPI_USE_MALLOPT_MMAP_MAX=0 -np 1 /soft/ANSYS/18.2/v182/fluent/fluent18.2.0/lnamd64/2d_node/fluent_mpi.18.2.0 node -mpiw ibmmpi -pic shmem -mport 192.168.147.149:192.168.147.149:36981:0
-------------------------------------------------------------------------------
ID Hostname Core O.S. PID Vendor
-------------------------------------------------------------------------------
n0 cfpsmall05 1/8 Linux-64 25502 Intel(R) Xeon(R) E5430
host cfpsmall05 Linux-64 25320 Intel(R) Xeon(R) E5430
MPI Option Selected: ibmmpi
-------------------------------------------------------------------------------
Cleanup script file is /homes/jbarber/cleanup-fluent-cfpsmall05-25320.sh
+------------------------------------------------------------------+
| ANSYS Product Improvement |
| |
| ANSYS Product Improvement Program helps improve ANSYS |
| products. Participating in this program is like filling out a |
| survey. Without interrupting your work, the software reports |
| anonymous usage information such as errors, machine and |
| solver statistics, features used, etc. to ANSYS. We never |
| use the data to identify or contact you. |
| The data does NOT contain: |
| - Any personally identifiable information including names, |
| IP addresses, file names, part names, etc. |
| - Any information about your geometry or design specific |
| inputs. |
| You can stop participation at any time. To change your |
| selection go to Help >> ANSYS Product Improvement Program |
| in the GUI. |
| For more information about the ANSYS Privacy Policy, please |
| check: http://www.ansys.com/privacy |
| |
+------------------------------------------------------------------+
> exit
$
To do the same thing with a batch job and the scheduler, create a script as follows (called test.sh in this example):
#!/bin/bash module load ansys fluent -g 2d -ssh -t$PBS_NP -cnf=$PBS_NODEFILE -i test.journal
and on the submit host, run it on the cluster with the qsub command:
$ qsub -l nodes=1:ppn=2 test.sh
As you need more nodes and processes, you can modify the nodes=1 and ppn=2 arguments.
A more complicated Journal example
The following journal illustrates processing a “real” data set from the ANSYS examples:
/file/read-case-data /usr/local/ansys/v145/icemcfd/Samples/Visual3_Files/Visual3_Examples/Pipe_Network/pp /file/auto-save/root-name pp /file/auto-save/data-frequency 10 /solve/iterate 100 exit
The first line reads the case and data files, the second sets the output to be saved to files in the current directory starting with pp, the third sets the data to be saved every 10 iterations, the forth line starts the solver running for 100 iterations, and the final line stops FLUENT.
The journal file can be executed with the FLUENT 3D solver as follows:
$ fluent -g 3d -ssh -t$PBS_NP -cnf=$PBS_NODEFILE -i test.journal
Parallel Interactive Jobs
It is possible to run interactive jobs over multiple nodes, with a command such as follows:
$ fluent 3d -ssh -t$PBS_NP -cnf=$PBS_NODEFILE
Note that you have to know which FLUENT version you want to run - it's probably one of:
- 2d
- 2ddp
- 3d
- 3ddp