|HOW TOs for shift crew|
|All Expert Contacts|
The WAC is primarily tasked with verifying the quality of the data taken during each shift - the WAC Notes pages are the ideal place for this information.
Set the RCND environment variables on an aonl computer with:
setenv RCDB_CONNECTION mysql://firstname.lastname@example.org:3306/a-rcdb
It is necessary that the WAC keep track of the various runs, conditions, cuts, channel maps, and pedestal changes as a function of time and file system.
Reanalyzing with updated cuts as needed
The WAC is responsible for updating the "official" rootfiles for everyone else to use (and everyone else is not permitted to change these root files without WAC permission).
Utilizing the "Prompt" WAC version of the "operations" branch of Japan is the ideal way to accomplish this.
The prompt code lives in the master branch of Prex-Prompt
An easy script to launch a new terminal, log into a new processor, and run a copy of prompt in the right environment is a cascade of shell scripts living in the ~/scripts/ folder Disclaimer: this technique is computationally intensive, though convenient, also the aggregator shell script appears to hate it...
[apar@aonl1 ~/scripts]$ ./terminal_prompt_1.sh adaq3 3602
Tracking Data over time
Helpful link for all steps of the process: https://docs.google.com/spreadsheets/d/1WVee1TgdSmqHRLoPxF7-2ty0YYNyqH3qhMulZXQItNQ/edit?usp=sharing
The run-wise plots are uploaded to the folder /u/group/halla/www/hallaweb/html/parity/prex/onlinePlots, which is visible from Haweb
The WAC is also responsible for keeping track of the data over the course of "slugs" (generally defined as in/out insertions of the IHWP or flips of the Wein).
- There are several codebases, initially we used Tao's postpan collection tool, the "Collector".
- This was then enhanced with a more general drawing and textfile printing macro.
- For general data analysis and daily aggregation, this has since been replaced with the root-file reading and histogram filling/parsing set of methods referred to as the "Aggregator"
- The Aggregator is designed for pulling the data out of rootfiles directly and calculating histogram based quantities off of that (rather than trusting intermediary programs to have the extensibility needed for arbitrary data manipulations)
The collector code lives in Tao's collector repository
Do all the Plots (Except grand_prototype)
Create a list of good production runs formatted one run number per line. Save the list in the run_list directory with a file name in the form:
Run this script, using the full file path to your newly created run list:
bash ~/PREX/prompt/EZ_WAC.sh ~/PREX/prompt/collector/run_list/slug##.list
This script will create these files and copy them to the online slug directories https://hallaweb.jlab.org/parity/prex/onlinePlots/slug/ :
o_slug##.txt slug##.list y_postPanPlots_slug##.pdf aggregate_maindet_slug##.pdf slug_charge_mon.pdf summary_minirun_slug##.pdf summary_minirun_slug##.txt summary_minirun_slug_linear##.txt total_charge_mon.pdf
To create the grand-aggregator plots you will need to run:
cd ~/PREX/prompt/Aggregator/drawPostpan setenv CAM_OUTPUTDIR /chafs2/work1/apar/aggRootfiles/slugRootfiles/grandRootfile root -l -b -q plotAgg.C'("aggRootfiles/slugRootfiles/minirun_slug","plots/summary_minirun_slug", slug, ihwp, wein, hrs)' The first parameter points to location of accumulated minirun files. The second parameter points to location of output plots and text file. The third parameter is the slug number and needs to be entered by user. Enter ihwp = 1 for in, 2 for out Enter wein = 1 for right, 2 for left Enter hrs = 0 for both, 1 for right only, 2 for left only
You will need to run the above command for *each* slug one at a time. Trying to run more than one simultaneously will not work.
After you have finished running the above commands, you can create the pdf with:
cd ~/PREX/prompt/Aggregator/drawPostpan root -l -b -q grandAgg.C'("/chafs2/work1/apar/aggRootfiles/slugRootfiles/grandRootfile/grand_aggregator.root","~/PREX/prompt/hallaweb_online/slug/slug_list/slug##/OUTFILE")'
Where OUTFILE is the string you want to use as your pdf file name without the ".pdf" Since this only uses slugs 26 onward, usually these a good choice is "slug_26-##"
Old Step-by-step Instructions
- login as apar@adaq3
- cd into ~/PREX/prompt/collector
- source ../../setup_japan.tcsh
- cd ~/PREX/prompt/collector
WAC List Creation
- create a list of run number you want to (see for example test.list)
- use PVDB/RCND/RCDB commands/website
- make a slug##.list in ~/PREX/prompt/collector/run_list/
- run the command: ./collector -d ../results/ -l run_list/slug##.list (Note:replace slug##.list with your file name)
- a root file named prexPrompt_slug##.root will be stored in the rootfiles directory
- to produce plots stay in the collector directory and run ./aggregate
- you will be asked to enter slug##, enter it and hit Enter
- a aggregate_plots_slug##.pdf file will be created in the plots directory.
Draw Post Pan:
- cd into drawPostpan
- [apar@adaq1 ~/PREX/prompt/collector/drawPostpan]$ root -l -b -q drawPostPan.C'("~/PREX/prompt/collector/rootfiles/prexPrompt_slug##.root","slug##","list.txt")'
To reanalyze and make new run and minirun files for each run
- Optional: Make your slug list, then if the run/minirun individual stub files haven't been made yet do.
- Once the aggregator is done running on all runs in a given slug you can hadd them together and add the units branch to them with
cd ~/PREX/prompt/Aggregator/drawPostpan/ ; ~/PREX/prompt/Aggregator/drawPostpan/accumulate_mini_aggFiles_list.sh slug##
- Then to make plots/csv file do
[apar@adaq2 ~/PREX/prompt/Aggregator/drawPostpan]$ setenv CAM_OUTPUTDIR /chafs2/work1/apar/aggRootfiles/slugRootfiles/grandRootfile [apar@adaq2 ~/PREX/prompt/Aggregator/drawPostpan]$ root -l -b -q plotAgg.C'("aggRootfiles/slugRootfiles/minirun_slug","plots/summary_minirun_slug", slug, ihwp, wein, hrs)' The first parameter points to location of accumulated minirun files. The second parameter points to location of output plots and text file. The third parameter is the slug number and needs to be entered by user. Enter ihwp = 1 for in, 2 for out Enter wein = 1 for right, 2 for left Enter hrs = 0 for both, 1 for right only, 2 for left only
- To look at the data by hand, find the output slug aggregated files and do:
root /chafs2/work1/apar/aggRootfiles/slugRootfiles/run_slug11.root root  agg->Draw("reg_asym_usl_mean:run_number","","*")
- To make the grand aggregator plots do
root -l -b -q grandAgg.C'("/chafs2/work1/apar/aggRootfiles/slugRootfiles/grandRootfile/grand_aggregator.root","outLocation...")'
Proposed Data Frame update
A Data frame is a useful ROOT 6.12 construct for managing data. It marries some of the Draw() data managing functionality into a MultiThread safe and TTree data traversal optimized framework:
ROOT::RDataFrame d("mul",_file0); or ROOT::RDataFrame d1(*tchainName) auto cutData = d.Filter("cuts"); auto hist1 = cutData.Define("defined1","math*branch1.leaf1").Histo1D("defined1"); auto hist2 = cutData.Define("defined2","math*branch2.leaf2").Histo1D("defined2"); hist1->Draw()
This executes all of the actual histo filling and math, traversing the tree, all at once (they are "Lazy"). See documentation for more.
Caveat: it needs the exact branch.leaf name, it won't search for the first leaf for you.
- Loop over a config input text file to get the draw commands, cuts and output names (if blank or NULL then just use the draw command as output name)
- Define all of those using .Define and .Filter and .Histo1D
- GetMean, RMS, Errors, and nEntries for each
- Make a totally fresh agg tree and entry with the same traditional structure/or add a line to be appended into agg file