ParMETIS (Parallel Graph Partitioning and Fill-reducing Matrix Ordering) is an MPI-based parallel library that implements a variety of algorithms for partitioning unstructured graphs, meshes, and for computing fill-reducing orderings of sparse matrices. ParMETIS extends the functionality provided by METIS and includes routines that are especially suited for parallel AMR computations and large scale numerical simulations. The algorithms implemented in ParMETIS are based on the parallel multilevel k-way graph-partitioning, adaptive repartitioning, and parallel multi-constrained partitioning schemes developed in Karypis lab.
METIS (Serial Graph Partitioning and Fill-reducing Matrix Ordering) is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. The algorithms implemented in METIS are based on the multilevel recursive-bisection, multilevel k-way, and multi-constraint partitioning schemes developed in Karypis lab.
Availability and Restrictions
Versions
ParMETIS is available on Owens and Pitzer Clusters. The versions currently available at OSC are:
| Version | Owens | Pitzer |
|---|---|---|
| 4.0.3 | X* | X* |
METIS is available on Owens, and Pitzer Clusters. The versions currently available at OSC are:
| Version | Owens | Pitzer |
|---|---|---|
| 5.1.0 | X* | X* |
You can use module -r spider '.*metis.*' to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.
Access
ParMETIS / METIS is available to all OSC users. If you have any questions, please contact OSC Help.
Publisher/Vendor/Repository and License Type
University of Minnesota, Open source
Usage
Usage on Owens
Set-up
To load ParMETIS, run the following command: module load parmetis. To use the serial implementation, METIS, run the following command instead: module load metis. You can use module spider metis and module spider parmetis to view available modules. Use module spider metis/version and module spider parmetis/version to check what modules should be loaded before load ParMETIS / METIS.
Building With ParMETIS / METIS
With the ParMETIS library loaded, the following environment variables will be available for use:
| Variable | Use |
|---|---|
| $PARMETIS_CFLAGS | Use during your compilation step for C or C++ programs. |
| $PARMETIS_LIBS |
Use when linking your program to ParMETIS. |
Similarly, when the METIS module is loaded, the following environment variables will be available:
| VARIABLE | USE |
|---|---|
| $METIS_CFLAGS | Use during your compilation step for C programs. |
| $METIS_LIBS | Use when linking your program to METIS. |
For example, to build the code myprog.cc with the METIS library you would use:
g++ -o myprog myprog.cc $METIS_LIBS
Batch Usage
When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.
Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the ParMETIS / METIS module in your batch script before executing a program which is built with the ParMETIS / METIS library. Below is the example batch script that executes a program built with ParMETIS:#!/bin/bash #SBATCH --job-name=myprogJob #SBATCH --nodes=1 --ntasks-per-node=28 module load gnu/4.8.5 module load parmetis cp foo.dat $TMPDIR cd $TMPDIR myprog < foo.dat > foo.out cp foo.out $SLURM_SUBMIT_DIR
Usage on Pitzer
Set-up
To load ParMETIS, run the following command: module load parmetis. To use the serial implementation, METIS, run the following command instead: module load metis .
Building With ParMETIS / METIS
With the ParMETIS library loaded, the following environment variables will be available for use:
| VARIABLE | USE |
|---|---|
| $PARMETIS_CFLAGS | Use during your compilation step for C or C++ programs. |
| $PARMETIS_LIBS |
Use when linking your program to ParMETIS. |
Similarly, when the METIS module is loaded, the following environment variables will be available:
| VARIABLE | USE |
|---|---|
| $METIS_CFLAGS | Use during your compilation step for C programs. |
| $METIS_LIBS | Use when linking your program to METIS. |
For example, to build the code myprog.cc with the METIS library you would use:
g++ -o myprog myprog.cc $METIS_LIBS
