OpenFOAM is a suite of computational fluid dynamics applications. It contains myriad solvers, both compressible and incompressible, as well as many utilities and libraries.
Availability and Restrictions
Versions
The following versions of OpenFOAM are available on OSC clusters:
| Version | Owens | Pitzer | Cardinal |
|---|---|---|---|
|
4.1 |
X |
|
|
| 5.0 | X | X | |
| 7.0 | X* | X* | |
| 1906 |
X |
||
| 1912 | X | ||
| 2306 | X | X | |
| 2312 | X |
The location of OpenFOAM may be dependent on the compiler/MPI software stack, in that case, you should use one or both of the following commands (adjusting the version number) to learn how to load the appropriate modules:
module spider openfoam module spider openfoam/2306
Feel free to contact OSC Help if you need other versions for your work.
Access
OpenFOAM is available to all OSC users. If you have any questions, please contact OSC Help.
Publisher/Vendor/Repository and License Type
OpenFOAM Foundation, Open source
Basic Structure for an OpenFOAM Case
The basic directory structure for an OpenFOAM case is:
/home/yourusername/OpenFOAM_case |-- 0 |-- U |-- epsilon |-- k |-- p `-- nut |-- constant |-- RASProperties |-- polyMesh | |-- blockMeshDict | `-- boundary |-- transportProperties `-- turbulenceProperties |-- system |-- controlDict |-- fvSchemes |-- fvSolution `-- snappyHexMeshDict
IMPORTANT: To run in parallel, you need to also create the decomposeParDict file in the system directory. If you do not create this file, the decomposePar command will fail.
Usage
Usage on Owens
Setup on Owens
To configure the Owens cluster for the use of OpenFOAM 4.1, use the following commands:module load openmpi/1.10-hpcx # currently only 4.1 is installed using OpenMPI libraries module load openfoam/4.1
Batch Usage on Owens
Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems.
On Owens, refer to Queues and Reservations for Owens and Scheduling Policies and Limits for more info.
Interactive Batch Session
For an interactive batch session on Owens, one can run the following command:
sinteractive -A <project-account> -N 1 -n 40 -t 1:00:00
which gives you 28 cores (-N 1 -n 28) with 1 hour (-t 1:00:00). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script (job.txt) for a serial run:
#!/bin/bash #SBATCH --job-name serial_OpenFOAM #SBATCH --nodes=1 --ntasks-per-node=1 #SBATCH --time 24:00:00 #SBATCH --account <project-account> # Initialize OpenFOAM on Owens Cluster module load openmpi/1.10-hpcx module load openfoam # Copy files to $TMPDIR and move there to execute the program cp * $TMPDIR cd $TMPDIR # Mesh the geometry blockMesh # Run the solver icoFoam # Finally, copy files back to your home directory cp * $SLURM_SUBMIT_DIR
To run it via the batch system, submit the job.txt file with the following command:
sbatch job.txt
Non-interactive Batch Job (Parallel Run)
Below is the example batch script (job.txt) for a parallel run:
#!/bin/bash #SBATCH --job-name parallel_OpenFOAM #SBATCH --nodes=2 --ntasks-per-node=28 #SBATCH --time=6:00:00 #SBATCH --account <project-account> # Initialize OpenFOAM on Ruby Cluster # This only works if you are using default modules module load openmpi/1.10-hpcx module load openfoam/2.3.0 # Mesh the geometry blockMesh # Decompose the mesh for parallel run decomposePar # Run the solver mpiexec simpleFoam -parallel # Reconstruct the parallel results reconstructPar
Usage on Pitzer
Setup on Pitzer
To configure the Pitzer cluster for the use of OpenFOAM 5.0, use the following commands:module load openmpi/3.1.0-hpcx # currently only 5.0 is installed using OpenMPI libraries module load openfoam/5.0
Batch Usage on Pitzer
Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems.
On Pitzer, refer to Queues and Reservations for Pitzer and Scheduling Policies and Limits for more info.
Interactive Batch Session
For an interactive batch session on Owens, one can run the following command:
sinteractive -A <project-account> -N 1 -n 40 -t 1:00:00
which gives you 1 node (-N 1), 40 cores (-n 40) with 1 hour (-t 1:00:00). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script (job.txt) for a serial run:
#!/bin/bash #SBATCH --job-name serial_OpenFOAM #SBATCH --nodes=1 --ntasks-per-node=1 #SBATCH --time 24:00:00 #SBATCH --account <project-account> # Initialize OpenFOAM on Owens Cluster module load openmpi/3.1.0-hpcx module load openfoam # Copy files to $TMPDIR and move there to execute the program cp * $TMPDIR cd $TMPDIR # Mesh the geometry blockMesh # Run the solver icoFoam # Finally, copy files back to your home directory cp * $SLURM_SUBMIT_DIR
To run it via the batch system, submit the job.txt file with the following command:
sbatch job.txt
Non-interactive Batch Job (Parallel Run)
Below is the example batch script (job.txt) for a parallel run:
#!/bin/bash #SBATCH --job-name parallel_OpenFOAM #SBATCH --nodes=2 --ntasks-per-node=40 #SBATCH --time=6:00:00 #SBATCH --account <project-account> # Initialize OpenFOAM on Ruby Cluster # This only works if you are using default modules module load openmpi/3.1.0-hpcx module load openfoam/5.0 # Mesh the geometry blockMesh # Decompose the mesh for parallel run decomposePar # Run the solver mpiexec simpleFoam -parallel # Reconstruct the parallel results reconstructPar
