When you connect to one of the HPC systems at OSC, you are working on a login node. This document describes the computing environment on the login nodes.
Summary of OSC-specific information
If you are familiar with Linux, please at least read this section.
- Login nodes have a shell limit of 20 minutes CPU time. Use the batch system for serious computing. For large file transfers use sftp.osc.edu instead of a login node.
- Each HPC system at OSC has multiple login nodes. Home directories are visible everywhere.
- OSC projects correspond to Linux groups.
- OSC uses a module system for setting up the software environment.
Login session
When you first log in, you’ll see two important pieces of information. The first is the message of the day. The message of the day includes important system notices such as scheduled downtime. These notices are also posted on https://www.osc.edu/. You can display the motd again with the command cat /etc/motd.
The second piece of information displayed is information about your disk usage and quota. You have at least one and possibly more allocations of disk space associated with your user name. Each allocation has limits on the amount of space and the number of files you can use. This information is collected nightly and displayed when you log in. If you exceed your quota, further commands and jobs will fail, so keep an eye on it. You can display your current usage with the quota command.
The login nodes, like the compute nodes, run the Linux operating system. You will be greeted with a prompt, for example:
-bash-4.1$
This prompt comes from the shell, which is the command-line interpreter for Linux.
The logout or exit command terminates your login session.
Sample login session where the user does nothing except exit:
Last login: Mon Jun 25 16:02:13 2012 from 131.187.118.68
******************************************************************************
Welcome to the Ohio Supercomputer Center!
PLEASE SEE THE OSC TECHNICAL INFORMATION SERVER (http://www.osc.edu/hpc)
FOR ANNOUNCEMENTS/NOTICES AND USER INFORMATION.
For questions or assistance, contact oschelp@osc.edu.
*****************************************************************************
2012/05/01 All OSC systems are operating normally.
*****************************************************************************
As of 2012 Jun 25 04:02 userid usr1234 on /nfs/14 used 250GB of quota 500GB and 560634 files of quota 1000000 files
As of 2012 Jun 25 04:02 project/group G-2959 on /nfs/proj09 used 246GB of quota 2861GB and 1087715 files of quota 2000000 files
-bash-4.1$ exit
logout
Purpose of login nodes
The login nodes are the gateway to the HPC systems. They should never be used for serious computing. The login nodes are intended for tasks such as text editing, managing your files, submitting jobs to the batch system, and monitoring your jobs. They are also suitable for compiling and linking your code, unless the code is huge. Small-scale interactive work may also be acceptable if your resource requirements are minimal.
The login shell limits your computations to 20 minutes of CPU time. When you reach that limit your process will be killed with, unfortunately, no explanation. Remember: Computation should be done through the batch system, not on the login node.
Large file transfers sometimes hit the 20-minute limit because of the encryption overhead. See below for instructions on transferring large files.
Login node structure
Each HPC system at OSC has its own set of login nodes. Connections are done in a round-robin fashion. When you ssh to OSC system, you will actually be connected to one of its login nodes.
If you are just working with your files and directories it doesn’t matter which system you log into. All the login nodes on our systems have all the same file systems mounted. If you are submitting or monitoring batch jobs or building code, it does matter which system you are logged into, but not which login node on that system. You can find out which node you are on with the hostname command, although you rarely need to know.
Linux
Linux is a “Unix-like” operating system that is widely used in HPC. Your interaction with Linux is through a command-line shell, which displays a prompt and interprets the commands you type in.
You have a choice of several shells: bash, csh, tcsh, or ksh. The default shell on OSC systems is bash, but you can change it through my.osc.edu. The different shells are similar in concept but differ in some of their commands and syntax. If you don’t have a strong preference, we suggest you stay with bash.
This document is not intended as a Linux/Unix tutorial; there are plenty of those available. We describe just a few commands that you can’t live without.
Throughout this documentation, we will use the dollar sign ($) as a generic prompt in our examples. User input is shown in bold; output is shown in normal type.
One of the most important commands is man, which displays online manual pages. Most commands have a man page. For example, to see the man page for the mkdir command:
$ man mkdirMKDIR(1) User Commands MKDIR(1) NAME mkdir - make directories SYNOPSIS mkdir [OPTION]... DIRECTORY... DESCRIPTION Create the DIRECTORY(ies), if they do not already exist. [ remainder of output omitted ] $
Files and directories
When you log in to any of the login nodes you’ll be working in your home directory. Each user has a home directory that is shared across all OSC systems. Whatever system you connect to, you’ll have the same home directory. You can create files and subdirectories in your home directory.
File permissions in Unix/Linux are based on the concepts of “user”, “group”, and “other”. “User” is just you. “Other”, or “world”, in our case means everyone with an OSC user ID. Your primary “group” at OSC corresponds to the project your user ID is associated with. Groups are also used to control access to licensed software. You can set different permissions on your files and directories for each of these categories. The permissions you can set are read, write, and execute. The default permissions on your home directory allow anyone with an OSC HPC account to read your files, although only you have write permission. You can change the permissions with the chmod command if you want to restrict access.
You can transfer files between an outside computer and your home directory using any of these protocols: sftp, scp, rsync. These are all secure transfers, meaning that they encrypt your data. If you have large files, you may hit the 20-minute limit on CPU time on the login node. If this happens you may use sftp.osc.edu for your transfers instead of the login node. This server supports the same protocols as the login nodes, plus gridftp, but it doesn’t have the same shell limits. It is to be used only for file transfers.
Here are a few commands that you’re likely to use for managing your files. Look up the man pages for more information. Many other options are available. If this is new to you, be sure to read a Linux or Unix tutorial.
| Command | Description |
ls |
List the files in the current working directory or specified directory. With the -l (lowercase L, not number one) option, give a long listing. With the -a option include files starting with a period (“.”), which are called dot-files and are usually hidden. |
cd |
Change directory. If no directory is specified, change to the home directory. If a directory is specified, it can be either relative to the current directory or absolute. The “..” means the parent directory, up one level. |
pwd |
Print the full name of the current working directory. |
mkdir new_dir |
Make a new directory in the current working directory. |
cp filename newloc |
Copy a file to a new location or a new name in the current directory. |
mv filename newname |
Move a file to a new location or rename it in the current directory. |
cat filename |
Display a file. |
less filename |
Display a file with paging. |
rm filename |
Remove a file. |
chmod a+r filename |
Make a file world readable (add “read” permission for “all”) Make a file executable by me and my group (add “execute” permission for “user” and “group”) Remove read and execute permissions for “other” from this directory and all files and directories below it, recursively. |
tar -cf myfiles.tar dirname |
Save many files together into a single archive file; restore files from an archive. Compression options are available. Example is for archiving a directory. |
gzip filename |
Compress or uncompress a file. |
See the storage environment documentation for more information.
Graphics and GUIs
While Linux is primarily a command-line environment, many applications have graphical user interfaces (GUIs) based on a standard called X11. See the documentation for connecting to the OSC systems for instructions on getting graphics to work. A useful command for testing your X11 connection is xclock, which displays a clock (with hands that move) if everything is working properly. Two things to check if it doesn’t work: 1) make sure you have an X server running on your local machine; and 2) make sure your ssh connection is set up to forward or tunnel X11. These steps are described in the documentation described above.
Text editing
You can create and edit files using any of a number of text editors available in Linux. The traditional Unix editors vi and emacs are available; there are many books and web pages about vi and emacs if you’re interested.
A simpler editor to get you started is nano, a non-GUI editor that is fairly self-explanatory. Just type nano at the prompt.
If you prefer a GUI-based editor, you can use gedit. You must have X11 forwarding enabled and have an X11 server running on your system. (See other documentation.) Then type gedit at the prompt.
Yet another option is to edit your files on your own system and transfer them back and forth to the HPC system. If you do this you should be aware that text files created on Windows, Mac, and Linux all have different line endings, sometimes leading to difficult-to-diagnose problems.
Note to Windows users
The following command will convert your Windows text file to Unix format:
dos2unix filename
For conversion in the other direction:
unix2dos filename
On your Windows system, Wordpad will correctly display Unix files while Notepad will not.
Note to Mac users
The following command will convert your Mac text file to Unix format:
mac2unix filename
Environment variables
Your environment in Linux is controlled by a set of environment variables. Some of these variables are automatically set for you; others you control either directly or by loading environment modules. Environment variables are all upper case. To use their values you precede the name with a $.
A few of the environment variables you might need to know about are these:
$HOME |
The absolute path of your home directory |
$PATH |
List of directories where the system looks for programs to run |
$LD_LIBRARY_PATH |
List of directories where the system looks for shared libraries to load |
To display the value of an environment variable, use the echo command, for example:
$ echo $HOME
/users/PAZ0001/usr1234
(Recall that the $ at the beginning of the line in our examples represents the prompt and is not something that you type in.)
To go to your home directory:
cd $HOME
You can set the value of an environment variable. The following examples set the value of OMP_NUM_THREADS (an OpenMP parameter) to 12.
To set the value of an environment variable in bash:
export OMP_NUM_THREADS=12
To set the value of an environment variable in csh:
setenv OMP_NUM_THREADS 12
Running programs
To run a program under Linux, type the program name followed by any arguments. If the program executable is not in your path, you must include the path with the program name.
Example of running a program that is in your path:
vi myfile
Example of running a program that is not in your path:
$HOME/bin/myprog
If you want to run a program located in your current directory, you should precede its name with “./”. The dot (“.”) represents your current directory. For security reasons, it is generally not part of your path. Example:
./myprog2
You can add a directory to your path, with the syntax depending on which shell you use.
Example in bash:
export PATH=$HOME/bin:$PATH
Example in csh:
setenv PATH $HOME/bin:$PATH
Modules
OSC has dozens of software packages installed. Rather than clutter up your environment with all of them, and probably running into conflicts as a result, we provide a system of modules. When you load a module, it modifies or sets the environment variables necessary for running the corresponding software. The module system is available in both the login environment and the batch environment. It is documented in “Batch Processing at OSC”.
Initializing your environment
When a shell is started, it reads commands from a set of startup files to initialize itself. OSC-provided files are executed first to set up the basic OSC environment, including the module system. If you have startup files in your home directory, they will be executed next. You can use this mechanism to set paths or load modules that you always use, for example.
The shell startup files are often called “dot files” because their names start with a period or “dot”. They don’t show up when you list a directory using the ls command unless you add the option “-a”: “ls -a”.
A bash login shell reads commands from ~/.bash_profile. A bash non-login interactive shell reads commands from ~/.bashrc. Many people have their .bash_profile execute their .bashrc to keep their login and non-login environments the same. The csh or tcsh shell reads ~/.cshrc or ~/.tcshrc. (The symbol ~ refers to your home directory.)
