Difference between revisions of "Setting up Postpan"

From PREX Wiki
Jump to: navigation, search
(Getting Started)
 
(17 intermediate revisions by 2 users not shown)
Line 1: Line 1:
* To run prex-prompt in your own folder
+
Following these steps will allow you to use existing 'pass1' japan output root files to perform regression using postpan. If you want to [[Running prompt|run postpan's ./prompt]], you will need your own working local version of japan, and some of the instructions below will change as well.
  
First login to apar account on aonl1
+
== Getting Started ==
ssh apar@aonl1
+
# Login to the apar account on aonl2: <code>ssh apar@aonl2 </code>
 +
# Create a directory for output file storage
 +
#* Making your own folder is important, otherwise whatever you change in your directory will be changed in main prex-prompt.
 +
#* Right now, aonl2 has space.
 +
#* <code> mkdir /aonl2/work1/my_directory </code>
 +
# Create a directory for the code
 +
## Go to your own personal apar directory. If you don't have one, make one.
 +
##* e.g. if your directory name is my_directory, then <code> cd ~/my_directory </code>
 +
## Make a new folder inside your directory
 +
##* <code> mkdir prompt </code>
 +
# Clone the code from github
 +
#* <code> cd prompt </code>
 +
#* <code> git clone https://github.com/JeffersonLab/prex-prompt.git </code>
 +
# Set the environment to use ROOT version 6
 +
#* Create a file called reset_env_postpan.csh in the new prex-prompt folder. Copy and paste the following code into it:
 +
setenv PATH `echo $PATH | sed 's/:/\n/g' | grep -v "4.34.36" |grep -v ROOT | awk 'NR==1{printf"%s",$1}; NR>1{printf":%s",$1}'`
 +
setenv LD_LIBRARY_PATH `echo $LD_LIBRARY_PATH | sed 's/:/\n/g' | grep -v "5.34.36" | grep -v ROOT | awk 'NR==1{printf"%s",$1}; NR>1{printf":%s",$1}'`
 +
setenv ROOTSYS /adaqfs/apps/ROOT/6.14-04 <\code>
  
Go to your directory. If you don’t have one, make one
+
== Setting up for analysis ==
e.g. if your directory name is my_directory then do
+
Currently, ther version of prex-prompt that you pull from github is missing files. There are also many symbolic links that need to be changed. Right now, the symbolic links point to the current working version of japan, and running postpan with these links unchanged could <span style="color:red"delete or replace files, upload unnecessary plots to the web, and cause a lot of headaches for the WAC. '''So don't skip these steps!''' </span>
cd <my_directory>
+
Make a new folder inside your directory
+
  mkdir prompt
+
Go to the new directory and do git clone
+
cd prompt
+
  
git clone https://github.com/JeffersonLab/prex-prompt.git
+
=== Getting the missing files ===
 +
Copy the following missing files and directories to your <code> prex-prompt/postpan </code> directory from the <code> ~/PREX/prompt/postpan </code> directory. Do '''NOT''' try to save time by deleting and replacing your entire postpan directory. Git tracking will be all messed up!
  
cd prex-prompt
+
'''Example:''' <code> cp ~/PREX/prompt/postpan/redana ~/my_dir/prompt/prex-prompt/postpan </code>
You will see several files inside the prex-prompt directory. Remove all the links (the one in cyan color. e.g Aggregator, BMODextractor, hallaweb_online
+
# conf/
etc). Don't directly delete the link file, it will not only be deleted from your directory but also from original prex-prompt. Use the following command to remove the lonk files
+
# scripts/
rm <folder_name> -i
+
# src/
e.g. rm Aggregator -i,
+
# redana
It will ask you yes or no to remove the file, type
+
# libRedAna.so
y
+
# RedDict_rdict.pcm
After you remove all the linked files from your directory, make a directories of
+
# rootlogon.C
the same names that you just removed from your prex-prompt in different location.
+
# Also copy device_list.h from the <code> ~/PREX/prompt/postpan/rootmacros </code> directory to your <code> prex-prompt/postpan/rootmacros </code> directory
mkdir /aonl2/work1/<my_directory>/<folder_name>
+
e.g mkdir /aonl2/work1/<my_directory>/Aggregator
+
  
Now change the directories in to a link. Use the following command to create a link
+
=== Fixing the symbolic links ===
ln -s /aonl2/work1/my_directory/<file_name> <file_name>  
+
You will see several symbolic links (the ones in cyan color) inside the <code> prex-prompt </code> directory.
e.g. ln -s /aonl2/work1/<my_directory>/Aggregator Aggregator.
+
# Remove the current symbolic links
 +
#* '''Example:''' <code> rm Aggregator -i </code>
 +
#* <span style="color:red"> '''PROCEED WITH CAUTION''': Typing <code> rm Aggregator/ -i </code> (with a slash '/' ) will delete EVERYTHING in the actual Aggregator folder. Please be careful, because autocomplete will put a slash - no slashes allowed! <span style="color:red">
 +
## Aggregator
 +
## BMODextractor
 +
## hallaweb_online
 +
## japanOutput
 +
## LogFiles
 +
## LRBOutput
 +
## results
 +
## SummaryPlots
 +
## SummaryText
 +
## results in the <code> postpan </code> directory
 +
#* You will be asked if you want to delete the symbolic link. Type <code> y </code> for yes.
 +
# Create new directories
 +
#* This is what your new output file directory <code> /aonl2/work1/my_directory </code> is for
 +
#* Create directories in this folder with the same names as the links that you just removed from your prex-prompt, '''EXCEPT japanOutput'''
 +
#* '''Example:''' <code> mkdir /aonl2/work1/<my_directory>/Aggregator </code>
 +
# Create new symbolic links pointing to the new directories you just made
 +
## Go into the directory you want to create the link in
 +
##* <code> cd ~/my_directory/prompt/prex-postpan/ </code>
 +
## Create a new symbolic link that points to your own personal folder
 +
##* '''Example:''' <code> ln -s /aonl2/work1/<my_directory>/Aggregator Aggregator </code>
 +
## We want to use the already-made japan 'pass1' root files so we don't have to re-run them. Point <code> japanOutput </code> to:
 +
##* <code> ln -s /chafs2/work1/apar/japanOutput japanOutput </code>
  
Do similarly for all the folders. Making your own folder is important otherwise whatever you will
+
== Running postpan ==
change in your directory will be changed in main prex-prompt.
+
At last, you are ready to run!
  
Go inside the postpan  directory and remove the link file called “results” similarly as you
+
=== Running with regression variables already in the root file ===
removed other links and make a new directory called results and convert it into link using above
+
# Go into your prex-prompt directory: <code> cd ~/my_directory/prompt/prex-prompt/ </code>
method. Basically you need to remove all the link folders that you see and make your own
+
# Source ROOT Version 6: <code> source reset_env_postpan.csh </code>
folder.
+
# Change the dependent and independent variables for the regression analysis, and add any additional cuts
 +
#* These can be found in <code> prex-prompt/postpan/conf/combo_reg.conf </code>
 +
#* Change variables: dv are the dependent varibles and iv are the independent variables
 +
#* Add your own custom cuts
 +
# Run postpan
 +
#* <code> ./auto_postpan.sh <run_number> </code>
  
* To run prompt-analysis
+
=== Running with regression variables not in the root file ===
Go to the prex-prompt directory and do,
+
Sometimes, you'll want to do regression with a linear combination of variables. This is a bit more complicated. You'll need to re-run japan on the raw data so the root tree has the variables you want to look at.
./prompt.sh <run_number>
+
# First, [[Setting up japan|set up japan]]
 +
# Source your <code> reset_env_japan.csh file </code>
 +
# Open the japan/Parity/prminput/prex_datahandler.<run#>.map file corresponding to the run you are going to analyse
 +
# See what file is used for  [ [ QwCombiner ] ]
 +
# Open that file and add your new variables
 +
#* There are examples in the file
 +
# Next, follow the instructions for [[Running prompt|setting up prompt for analysis]]
 +
#* Do NOT run prompt!
 +
# Run <code> ./qwparity -r <run_number> -c prex_prompt.conf </code>
 +
#* This will generate a pass1 japan output file in your japanOutput directory.
 +
# Change the dependent and independent variables for the regression analysis, and add any additional cuts
 +
#* These can be found in <code> prex-prompt/postpan/conf/combo_reg.conf </code>
 +
#* Change variables: dv are the dependent varibles and iv are the independent variables
 +
#* Add your own custom cuts
 +
# Run postpan
 +
#* <code> ./auto_postpan.sh <run_number> </code>
  
* Making summary plots for single run
+
== Where is the output? ==
./summary.sh <run_number>
+
Running postpan will generate 2 ouput files that will be saved in the <code> results </code> directory. Running <code> ./auto_postpan.sh </code> again on the same run number will create files with the same name. So rename your output files if you are going to do this, so you can keep track of them.
 
+
# '''prexPrompt_<run_number>_postpan_summary.txt'''
* For list of runs, make a list of runs in a file and do,
+
#* Raw and corrected means and RMSs of your dependent variables
./prompt_runlist.sh <runlist_filename>
+
#* Raw mean and RMS of your independent variables
 
+
#* Lists these statistics for the entire run as well as separate mini-runs
This will analyse and plot the run number that you gave. Your plots will be saved in
+
# '''prexPrompt_<run_number>_regress_postpan.root'''
"SummaryPlots" folder and text file will be saved at "SummaryText" folder. The root file will
+
#* Any cuts you put in the configuration file are applied by the cut <code> ok_cut == 1 </code>
be saved in "japanOutput" folder.
+
#* The reg tree contains dependent variables before and after correction, and the independent variables
 
+
#* The mini tree contains data that you can sort based on mini run number, <code> minirun </code>
* summary.sh script uploads plots and text summary to the webpage. So be mindful
+
when pushing this button.
+
 
+
 
+
* To change the dependent and independent variables (dv and iv) for the regression analysis,
+
and run postpan, go to postpan/cong directory and add a desired variables in combo_reg.conf
+
file. If the run has already analyzed in japanOutput file you can use the following command to
+
get a postpan results.
+
 
+
* To run the postpan
+
./auto_postpan.sh <run_number>
+
 
+
Postpan rootfiles will be saved in “results” directory.
+

Latest revision as of 15:26, 26 July 2019

Following these steps will allow you to use existing 'pass1' japan output root files to perform regression using postpan. If you want to run postpan's ./prompt, you will need your own working local version of japan, and some of the instructions below will change as well.

Getting Started

  1. Login to the apar account on aonl2: ssh apar@aonl2
  2. Create a directory for output file storage
    • Making your own folder is important, otherwise whatever you change in your directory will be changed in main prex-prompt.
    • Right now, aonl2 has space.
    • mkdir /aonl2/work1/my_directory
  3. Create a directory for the code
    1. Go to your own personal apar directory. If you don't have one, make one.
      • e.g. if your directory name is my_directory, then cd ~/my_directory
    2. Make a new folder inside your directory
      • mkdir prompt
  4. Clone the code from github
  5. Set the environment to use ROOT version 6
    • Create a file called reset_env_postpan.csh in the new prex-prompt folder. Copy and paste the following code into it:
setenv PATH `echo $PATH | sed 's/:/\n/g' | grep -v "4.34.36" |grep -v ROOT | awk 'NR==1{printf"%s",$1}; NR>1{printf":%s",$1}'`
setenv LD_LIBRARY_PATH `echo $LD_LIBRARY_PATH | sed 's/:/\n/g' | grep -v "5.34.36" | grep -v ROOT | awk 'NR==1{printf"%s",$1}; NR>1{printf":%s",$1}'`
setenv ROOTSYS /adaqfs/apps/ROOT/6.14-04 <\code>

Setting up for analysis

Currently, ther version of prex-prompt that you pull from github is missing files. There are also many symbolic links that need to be changed. Right now, the symbolic links point to the current working version of japan, and running postpan with these links unchanged could delete or replace files, upload unnecessary plots to the web, and cause a lot of headaches for the WAC. So don't skip these steps!

Getting the missing files

Copy the following missing files and directories to your prex-prompt/postpan directory from the ~/PREX/prompt/postpan directory. Do NOT try to save time by deleting and replacing your entire postpan directory. Git tracking will be all messed up!

Example: cp ~/PREX/prompt/postpan/redana ~/my_dir/prompt/prex-prompt/postpan

  1. conf/
  2. scripts/
  3. src/
  4. redana
  5. libRedAna.so
  6. RedDict_rdict.pcm
  7. rootlogon.C
  8. Also copy device_list.h from the ~/PREX/prompt/postpan/rootmacros directory to your prex-prompt/postpan/rootmacros directory

Fixing the symbolic links

You will see several symbolic links (the ones in cyan color) inside the prex-prompt directory.

  1. Remove the current symbolic links
    • Example: rm Aggregator -i
    • PROCEED WITH CAUTION: Typing rm Aggregator/ -i (with a slash '/' ) will delete EVERYTHING in the actual Aggregator folder. Please be careful, because autocomplete will put a slash - no slashes allowed!
    1. Aggregator
    2. BMODextractor
    3. hallaweb_online
    4. japanOutput
    5. LogFiles
    6. LRBOutput
    7. results
    8. SummaryPlots
    9. SummaryText
    10. results in the postpan directory
    • You will be asked if you want to delete the symbolic link. Type y for yes.
  2. Create new directories
    • This is what your new output file directory /aonl2/work1/my_directory is for
    • Create directories in this folder with the same names as the links that you just removed from your prex-prompt, EXCEPT japanOutput
    • Example: mkdir /aonl2/work1/<my_directory>/Aggregator
  3. Create new symbolic links pointing to the new directories you just made
    1. Go into the directory you want to create the link in
      • cd ~/my_directory/prompt/prex-postpan/
    2. Create a new symbolic link that points to your own personal folder
      • Example: ln -s /aonl2/work1/<my_directory>/Aggregator Aggregator
    3. We want to use the already-made japan 'pass1' root files so we don't have to re-run them. Point japanOutput to:
      • ln -s /chafs2/work1/apar/japanOutput japanOutput

Running postpan

At last, you are ready to run!

Running with regression variables already in the root file

  1. Go into your prex-prompt directory: cd ~/my_directory/prompt/prex-prompt/
  2. Source ROOT Version 6: source reset_env_postpan.csh
  3. Change the dependent and independent variables for the regression analysis, and add any additional cuts
    • These can be found in prex-prompt/postpan/conf/combo_reg.conf
    • Change variables: dv are the dependent varibles and iv are the independent variables
    • Add your own custom cuts
  4. Run postpan
    • ./auto_postpan.sh <run_number>

Running with regression variables not in the root file

Sometimes, you'll want to do regression with a linear combination of variables. This is a bit more complicated. You'll need to re-run japan on the raw data so the root tree has the variables you want to look at.

  1. First, set up japan
  2. Source your reset_env_japan.csh file
  3. Open the japan/Parity/prminput/prex_datahandler.<run#>.map file corresponding to the run you are going to analyse
  4. See what file is used for [ [ QwCombiner ] ]
  5. Open that file and add your new variables
    • There are examples in the file
  6. Next, follow the instructions for setting up prompt for analysis
    • Do NOT run prompt!
  7. Run ./qwparity -r <run_number> -c prex_prompt.conf
    • This will generate a pass1 japan output file in your japanOutput directory.
  8. Change the dependent and independent variables for the regression analysis, and add any additional cuts
    • These can be found in prex-prompt/postpan/conf/combo_reg.conf
    • Change variables: dv are the dependent varibles and iv are the independent variables
    • Add your own custom cuts
  9. Run postpan
    • ./auto_postpan.sh <run_number>

Where is the output?

Running postpan will generate 2 ouput files that will be saved in the results directory. Running ./auto_postpan.sh again on the same run number will create files with the same name. So rename your output files if you are going to do this, so you can keep track of them.

  1. prexPrompt_<run_number>_postpan_summary.txt
    • Raw and corrected means and RMSs of your dependent variables
    • Raw mean and RMS of your independent variables
    • Lists these statistics for the entire run as well as separate mini-runs
  2. prexPrompt_<run_number>_regress_postpan.root
    • Any cuts you put in the configuration file are applied by the cut ok_cut == 1
    • The reg tree contains dependent variables before and after correction, and the independent variables
    • The mini tree contains data that you can sort based on mini run number, minirun