JLab Computing

Using ifarm Scientific Computing Resources

The JLab computer center manages the various ifarm clusters and data storage. These are some of the resources offered there (general guide):

• To log into the JLab servers first ssh into login.jlab.org, which requires a CUE JLab computer account. Then you can log into another node in the jlab network like the jlabl1 workstations or the ifarm via another ssh directly to those hostnames.
• Standard nuclear physics computing programs and analysis software can be initialized into your coding environment by executing the "production" scripts. To execute these scripts upon logging into the jlab servers place the following code or something similar in a file called .login or any .*rc file you prefer:

source /site/12gev_phys/softenv.csh 2.3

• If you have problems then make sure you are running the /bin/tcsh shell (execute "echo $SHELL" on the command line to see what shell you are running)
• Recently the production scripts have been updated to "softenv" scripts, and the newest version is 2.3
• The swif workflow program can manage running batch jobs on the Auger batch farm system for you.
• See the "job management" section of Ciprian Gal's prexSim simulation readme for specific details on how to get a swif workflow up and running for you.

Getting Account Access Permissions

• Register as a JLab user, undergrad, or grad student (here, or register from the "online" link in here).
  • After registration, you have to "Register New Visit" as user group "Remote Access," even though you aren't necessarily visiting, and you will need you to call the JLab helpdesk at some point.
  • While filling out the Registration form you can request an account on the JLab Common User Environment (CUE). You must include Bob Michaels (rom@jlab.org) as your JLab sponsor for the account - be sure to request access to a-parity and moller12gev user groups (here is a good starting link).
  • To set up your computing environment on the ifarm see above.
  • To get access to swif and scientific computing resources follow the instructions here.
  • Then to use swif see the guide below or readmes in relevant repositories.
• Jefferson Lab github access - Send an email with the following (and if this doesn't work ask one of the senior members of the collaboration to add you themselves):

Subject: Please add me to the JeffersonLab github organization
To: <helpdesk@jlab.org>
Hello,

I'm a JLab user and my JLab user name is _______.
Could you please add me to the JeffersonLab github organization?
My github username is ______ and account id is ______

Introduction to SWIF

To use the ifarm's batch submission system (online monitoring and documentation here) one option is to use the Auger batch system manager called "swif" (documented somewhat here).

• To use swif first you need access to the ifarm, then you need to create a certificate (see above)
• Execute

/site/bin/jcert -create

• To create a workflow on swif run

swif create -workflow WorkFlowName (where WorkFlowName is an identifier you give to it to monitor its progress)

• To monitor the workflow run

swif status -workflow Name

• To delete a workflow run

swif cancel -workflow Name
swif cancel -delete -workflow Name

• To add a job run

swif add-jsub -workflow Name -script jobScript.xml
swif run -workflow Name

• To create a script .xml file for running jobs see the description of its function and the python wrapper code included in Ciprian's prexSim code (https://github.com/cipriangal/prexSim) or Cameron's updated one to work with new remoll v2.0.0 data structures (jlabSubmit.py and its relatives)
• A suggested .login file for your ifarm uses (that allows for batch job submission) is:

source /site/env/syslogin
source /site/env/sysapps
if ( `hostname` !~ "jlabl"* && `hostname` !~ "adaq"* )  then
source /site/12gev_phys/softenv.csh 2.3 
endif

• A sample .tchsrc file for using the default ifarm tc shell is here:

# ~/.tcshrc: executed by tcsh(1) for non-login shells.
setenv PATH $PATH\:/site/bin 
set savehist = 100000
set histfile = ~/.tcsh_hist
alias root root -l
alias gits git status
alias swif /site/bin/swif
alias swifs swif status -workflow