Difference between revisions of "Weekly Analysis Coordinator"

From PREX Wiki
Jump to: navigation, search
(→‎Online analysis)
m (→‎WAC Notes PREX II | WAC Notes CREX)
 
(81 intermediate revisions by 11 users not shown)
Line 12: Line 12:
 
|}
 
|}
  
−
<span style="color:red">In the process of editing and updating this page. Please contact Juliette Mammei <B>[mailto:crowder@jlab.org?Subject=PREXwiki crowder@jlab.org]</B> with questions or suggestions.</span>
+
 
 
<br><br>
 
<br><br>
−
[[Main_Page|<B>PREX Main</B>]]
+
[[Main_Page|<B>PREX Main</B>]] >> [[WAC_Notes|WAC Notes Index]] | [[WAC_Notes_CREX|CREX WAC Notes Index]]
 
<br><br>
 
<br><br>
  
 
== Weekly Analysis Coordinator (WAC) ==
 
== Weekly Analysis Coordinator (WAC) ==
  
−
[[WAC|WAC instructions]]
+
'''Daily Meeting [https://bluejeans.com/454783784 BlueJeans]'''
 +
 
 +
Useful Links:
 +
* [[WAC_Notes|WAC Notes Index]] | [[WAC_Notes_CREX|CREX WAC Notes Index]]
 +
* [[WAC Collection]]
 +
* [[WAC|Old PREX I WAC instructions]]
 +
* [[WAC Re-prompt Device Additions|PREX II WAC Re-prompt Device Additions]]
 +
* [[WAC Respin 1|PREX II WAC Respin 1]]
 +
 
 
<table border="1">
 
<table border="1">
−
<caption> WAC schedule
+
<caption> WAC schedule - CREX
 
</caption>
 
</caption>
−
<th> Dates (2010) </th><th> WAC name
+
<th> CREX Dates (2019-2020) </th><th> WAC name
 +
</th>
 +
<tr><td> 2019 12/02 - 2020 1/29 </td><td> Cameron Clarke</td></tr>
 +
<tr><td> 2020 1/30 - 2/05 </td><td> Weibin Zhang </td></tr>
 +
<tr><td> 2020 2/6 </td><td> Adam Zec </td></tr>
 +
<tr><td> 2020 2/7 - 2/12 </td><td> Sakib Rahman </td></tr>
 +
<tr><td> 2020 2/13 - 2/17 </td><td> Devi Adhikari </td></tr>
 +
<tr><td> 2020 2/18 - 2/23 </td><td> Ryan Richards </td></tr>
 +
<tr><td> 2020 2/24 - 2/25 </td><td> Devi Adhikari </td></tr>
 +
<tr><td> 2020 2/26 - 3/3 </td><td> Victoria Owen </td></tr>
 +
<tr><td> 2020 3/4 - 3/10 </td><td> Adam Zec </td></tr>
 +
<tr><td> 2020 3/11 - 3/15 </td><td> Sakib Rahman </td></tr>
 +
<tr><td> 2020 3/16 - 3/24 </td><td> Amali Premathelake </td></tr>
 +
<tr><td>  </td><td>  </td></tr>
 +
<tr><td> 2020 8/10 - 8/17 </td><td> Cameron Clarke </td></tr>
 +
<tr><td> 2020 8/18 - 8/24 </td><td> Sakib Rahman </td></tr>
 +
<tr><td> 2020 8/25 - 8/31 </td><td> Weibin Zhang </td></tr>
 +
<tr><td> 2020 9/1 - 9/7 </td><td> Robert Radloff </td></tr>
 +
<tr><td> 2020 9/8 - 9/14 </td><td> Victoria Owen </td></tr>
 +
<tr><td> 2020 9/15 - 9/21 </td><td> Iris Halilovic </td></tr>
 +
</table>
 +
 
 +
 
 +
 
 +
<table border="1">
 +
<caption> WAC schedule - PREX II
 +
</caption>
 +
<th> PREX II Dates (2019) </th><th> WAC name
 
</th>
 
</th>
 
<tr>
 
<tr>
−
<td> 06/24 - 07/02 </td><td> Tao Ye
+
<td> 06/24 - 07/06 </td><td> Tao Ye
 
</td></tr>
 
</td></tr>
 
<tr>
 
<tr>
−
<td> 07/02 - 07/09 </td><td> Sakib Rahman
+
<td> 07/06 - 07/17 </td><td> Sakib Rahman
 
</td></tr>
 
</td></tr>
 
<tr>
 
<tr>
−
<td> 07/09 - 07/16 </td><td>  
+
<td> 07/17 - 07/23 </td><td> Cameron Clarke
 
</td></tr>
 
</td></tr>
 
<tr>
 
<tr>
−
<td> 07/16 - 07/23 </td><td>  
+
<td> 07/23 - 07/30 </td><td> Devi Adhikari
 
</td></tr>
 
</td></tr>
 
<tr>
 
<tr>
−
<td> 07/23 - 07/30 </td><td>  
+
<td> 07/30 - 08/06 </td><td> Victoria Owen
 
</td></tr>
 
</td></tr>
 
<tr>
 
<tr>
−
<td> 07/30 - 08/06 </td><td>  
+
<td> 08/06 - 08/13 </td><td> Adam Zec and Cameron Clarke
 
</td></tr>
 
</td></tr>
 
<tr>
 
<tr>
−
<td> 08/06 - 08/13 </td><td>  
+
<td> 08/13 - 08/20 </td><td> Robert Radloff
 
</td></tr>
 
</td></tr>
 
<tr>
 
<tr>
−
<td> 08/13 - 08/20 </td><td>  
+
<td> 08/20 - 08/27 </td><td> Ryan Richards
 +
</td></tr>
 +
<tr>
 +
<td> 08/27 - 09/03 </td><td> Adam Zec
 
</td></tr>
 
</td></tr>
 
</table>
 
</table>
Line 53: Line 91:
 
== WAC Tasks ==
 
== WAC Tasks ==
  
 +
# Use command "gowac" to get a quick run down on scripts and set up the environment
 +
# Calculate accumulated coulombs and alert shift how many coulombs there are are left to collect crew if IHWP flip is imminent
 +
# Coordinate with the [[Analysis Expert]]
 
# Run the scripts for looking at daily/weekly data
 
# Run the scripts for looking at daily/weekly data
−
# Look closely at runs marked "junk"
+
#* Daily: use ''gojapan ; cd ~/PREX/prompt ; ./prompt.sh runnumber&''
 +
#* Slugly: '''[[WAC_Collection#Tracking_Data_over_time|How To]]'''
 +
# Look closely at runs marked "junk" and update runs every time something is done in the analysis chain (use '''[[PVDB]]''' tools)
 +
# Collect each shift data - '''[[WAC_Collection#Daily|How To]]'''
 
# Develop scripts to monitor data
 
# Develop scripts to monitor data
−
 
+
# Monitor changes made to the regular japan operations branch. This is the directory that opens when the gojapan command is used at login. The regular japan operations branch is the engine for realtime analysis. So, shiftcrew plot macros should be updated in this branch. If map files are updated, propagate that into japan_wac_only folder.
 +
#* For a given change in the maps/pedestals/cuts, push it to operations and pull to wac only
 +
#* cd ~/PREX/japan/Parity/prminput
 +
#* git add [things that are new]
 +
#* git commit --author=[email]
 +
#* git push -u origin operations
 +
#* [enter username and password for github]
 +
#* cd ~/PREX/japan_WAC_only/Parity/prminput
 +
#* git pull
 +
#* redo prompts with cd ~/PREX/prompt ; ./prompt.sh [runnumber]
 +
# Pull in changes from regular japan operations branch into the japan_wac_only folder after reviewing the commits made.
 +
# Track developments and make presentations from '''[[WAC Notes]]'''
  
 
== [[Online analysis]] ==
 
== [[Online analysis]] ==
  
−
* [[Run Info]]
+
* [[Run Info|WAC Notes table template]]
 
* [[PVDB | PVDB parity run database ]]
 
* [[PVDB | PVDB parity run database ]]
 +
** [https://hallaweb.jlab.org/rcdb/ Go to DB Home]
 
** [[PVDB#How_to_add.2Fmodify_comments_for_the_DB | Run DB comment gui]]
 
** [[PVDB#How_to_add.2Fmodify_comments_for_the_DB | Run DB comment gui]]
 
** [[PVDB#WAC_DB_TOOL | Update Run type and Run flag]]
 
** [[PVDB#WAC_DB_TOOL | Update Run type and Run flag]]
−
* [https://hallaweb.jlab.org/parity/prex/onlinePlots/ onlinePlots]
+
** [[PVDB#Make_a_run_list_from_DB | Make a run list from DB]]
 +
** [[PVDB#Simple_update_script_for_many_runs.2Fconditions | Using python script to update conditions]]
 +
** [[PVDB#Update_Run_End_Information | Update run end info (run end time, # events)]]
 +
** [[PVDB#Accumulated_charge_monitor | Get accumulated charge]]
 +
* [https://hallaweb.jlab.org/parity/prex/onlinePlots/ PREX II onlinePlots], [https://prex.jlab.org/analysis/prex2/index.php?all=true PREX II list of all runs] - from prompt analysis
 +
* [https://prex.jlab.org/analysis/crex/ CREX onlinePlots], [https://prex.jlab.org/analysis/crex/index.php?all=true CREX list of all runs] - from prompt analysis
 +
* [https://docs.google.com/spreadsheets/d/1lPRPk-IzzBdXKH4nujMcFYRrRwcvfKOMtl-omLCYO7M/edit#gid=1686002474 List of Slow Tree EPICS variables]
 +
* [https://docs.google.com/spreadsheets/d/1zc3_XIiwO2gzZOC9s4OADb5hWb8D1z0ZqVKc1kHCDZM/edit#gid=1146168969 Stale PREX II ResultsSummary]
 +
 
 +
  A stable grandroot file can be found in /chafs2/work1/apar/aggRootfiles/slugRootfiles/grandRootfile/archive/grand_aggregator_1-30.root which has all slugs between 1-30.
 +
  Useful cuts: ihwp==1 (in) ihwp==2 (out)
 +
              wein==1 (right) wein==2 (left)
 +
              hrs==0 (both arm) hrs==1 (right arm only) hrs==2 (left arm only)
 +
  To get an idea about variable nomenclature, look in the root tree or the [https://prex.jlab.org/analysis/prex2/slug/slug_list/slug30-Left/grand_prototype.pdf grand_prototype.pdf]
 +
* [https://hallaweb.jlab.org/parity/prex/onlinePlots/comptonPlots comptonPlots]
 +
* Check the Feedback log: (/adaqfs/home/apar/PREX/japan_feedback/LogFiles/Feedback_PITA_log.txt)
  
 
== Accessing onsite computers ==
 
== Accessing onsite computers ==
Line 70: Line 141:
 
With the access to hallgw, one can have access to the onsite computers.
 
With the access to hallgw, one can have access to the onsite computers.
  
−
* Do not connect remotely to the adaq computers
+
* Do not connect remotely to the adaq1 computer unless necessary for some DAQ related process
−
* For special analysis, connect to a machine on the aonl cluster
+
* Focus on aonl[1-3], especially for analysis jobs
 +
* For large analysis jobs focus on the ifarm installation of JAPAN
  
  
Line 81: Line 153:
 
* With access granted, type in: ssh username@hallgw
 
* With access granted, type in: ssh username@hallgw
  
 +
== [[WAC Notes|WAC Notes PREX II]] | [[WAC Notes CREX]] ==
 +
* [https://logbooks.jlab.org/entry/3707841 HALOG] - Accumulated Charge counting procedure
 +
* [https://logbooks.jlab.org/entry/3707596 HALOG] - IHWP flipping procedure (preliminary)
 +
* [https://hallaweb.jlab.org/parity/prex/onlinePlots/slug/ Hall A Web] - Online Plots for Slugs
 +
* [https://logbooks.jlab.org/entry/3707759#comment-22762 HALOG] - Procedure for defining BPM stability cuts on one run
 +
* [https://github.com/leafybillow/collector Collector Github Repo]
 +
* [https://github.com/leafybillow/prex-prompt PREX Prompt Github Repo]
 +
* [https://docs.google.com/spreadsheets/d/1zc3_XIiwO2gzZOC9s4OADb5hWb8D1z0ZqVKc1kHCDZM/edit?usp=sharing Stale PREX II Grand Slug Summary Sheet]
 +
* [https://docs.google.com/spreadsheets/d/1mt39fCuIW3Pr8TO7fzhtFjmMFt4Op30jMm5wMQke6Xo/edit?usp=sharing CREX Calibrations history spreadsheet], [https://docs.google.com/spreadsheets/d/1IXYl7AnaxCbTfQwMgTbbUlGdmMqnLQiTY1WADiEt-yE/edit#gid=0 PQB notes]
 +
* [https://docs.google.com/spreadsheets/d/1l5qA0u9KhLIFA-MbcEpI3ezUD9saq6g1kzB_eTJD_po/edit#gid=0 CREX Run List spreadsheet]
 +
 +
== To calculate statistical precision so far ==
 +
 +
Exp error bar = sqrt(1/(sum(1/sigma^2)))
 +
 +
== Known Problems and Solutions ==
 +
 +
1) What to do when a run has two distinct bpm12X response regions (Example: run 7842)?
 +
 +
Ans: Look at the variable in the mul tree called "BurstCounter" which encodes minirun index.  For run 7842, you can look at mul->Draw("BurstCounter:yield_bpm12X","ErrorFlag==0","") to see that in the current root file, only BurstCounter==4 has the change in the 12X, and then mul->Scan("yield_bpm12X:pattern_number","ErrorFlag==0 && BurstCounter==4") lets you see that if you flag patterns between 50719 and 68050 (first and last entry in the scan output) as bad you'd cut out the events when the position is changing. You can multiply the pattern by 4 to get the event number range or use the event tree to get the range and put that in a prex_bad_events.$run_number$.map file. Then reprompt.
 +
 +
2) What to do when a target position is changed while a run is ongoing (Example: run 7860?
 +
 +
Ans:
 +
a)  Start by looking at the slow tree to know when the target moved:
 +
slow->Draw("pcrex90BDSPOS.VAL:Entry$","","*")
 +
For run 7860, by zooming in, I see the target position has stabilized by EPICS event 15.
 +
 +
b)  Check the beam current in the slow tree:
 +
slow->Draw("hac_bcm_average:Entry$","","*")
 +
I can see we didn't have current until EPICS event 45 or so, thus we know we only had current on one target.
 +
If we had a run with beam on one target then the target was moved and we wanted to analyze the data after the move, we' need to use an eventNumberCut file to exclude the early events.  Or if we only wanted the early events, we could exclude all the events after the target started moving.  To do that, you'd use the evt tree to find the CodaEventNumber of the event at which you wanted the cut.
 +
I'm going to only keep events after 30000, for this run.
 +
 +
c)  Create a special eventNumberCut.7860.conf file.
 +
The content should look like this, except the event range will be different for a different run.
 +
 +
# Cuts to keep only evnts with Ca-48 target in position
 +
 +
event=30000:
 +
 +
blinder.force-target-blindable=1
 +
 +
Then you should be able to replay the run.
 +
 +
3) How to monitor dpp drift?
 +
 +
https://userweb.jlab.org/~sanghwa/prex/ArchiveTools.pdf
 +
 +
4) How to make regression minrun summary and grandaggregator plots?
 +
 +
a) Login to apar@adaq1.
  
 +
b) gowac
  
 +
c) make file named slug$number$.list inside run_list folder or update it. Each entry in the file corresponds to a run in the slug
  
 +
d) ./EZ_WAC.sh run_list/slug$number$.list
  
 
[[Category:HOW TO]] [[Category:WAC]]
 
[[Category:HOW TO]] [[Category:WAC]]

Latest revision as of 11:26, 10 September 2020

Documentation
HOW TOs for shift crew
Expert Tools
All Expert Contacts




PREX Main >> WAC Notes Index | CREX WAC Notes Index

Weekly Analysis Coordinator (WAC)

Daily Meeting BlueJeans

Useful Links:

WAC schedule - CREX
CREX Dates (2019-2020) WAC name
2019 12/02 - 2020 1/29 Cameron Clarke
2020 1/30 - 2/05 Weibin Zhang
2020 2/6 Adam Zec
2020 2/7 - 2/12 Sakib Rahman
2020 2/13 - 2/17 Devi Adhikari
2020 2/18 - 2/23 Ryan Richards
2020 2/24 - 2/25 Devi Adhikari
2020 2/26 - 3/3 Victoria Owen
2020 3/4 - 3/10 Adam Zec
2020 3/11 - 3/15 Sakib Rahman
2020 3/16 - 3/24 Amali Premathelake
2020 8/10 - 8/17 Cameron Clarke
2020 8/18 - 8/24 Sakib Rahman
2020 8/25 - 8/31 Weibin Zhang
2020 9/1 - 9/7 Robert Radloff
2020 9/8 - 9/14 Victoria Owen
2020 9/15 - 9/21 Iris Halilovic


WAC schedule - PREX II
PREX II Dates (2019) WAC name
06/24 - 07/06 Tao Ye
07/06 - 07/17 Sakib Rahman
07/17 - 07/23 Cameron Clarke
07/23 - 07/30 Devi Adhikari
07/30 - 08/06 Victoria Owen
08/06 - 08/13 Adam Zec and Cameron Clarke
08/13 - 08/20 Robert Radloff
08/20 - 08/27 Ryan Richards
08/27 - 09/03 Adam Zec

WAC Tasks

  1. Use command "gowac" to get a quick run down on scripts and set up the environment
  2. Calculate accumulated coulombs and alert shift how many coulombs there are are left to collect crew if IHWP flip is imminent
  3. Coordinate with the Analysis Expert
  4. Run the scripts for looking at daily/weekly data
    • Daily: use gojapan ; cd ~/PREX/prompt ; ./prompt.sh runnumber&
    • Slugly: How To
  5. Look closely at runs marked "junk" and update runs every time something is done in the analysis chain (use PVDB tools)
  6. Collect each shift data - How To
  7. Develop scripts to monitor data
  8. Monitor changes made to the regular japan operations branch. This is the directory that opens when the gojapan command is used at login. The regular japan operations branch is the engine for realtime analysis. So, shiftcrew plot macros should be updated in this branch. If map files are updated, propagate that into japan_wac_only folder.
    • For a given change in the maps/pedestals/cuts, push it to operations and pull to wac only
    • cd ~/PREX/japan/Parity/prminput
    • git add [things that are new]
    • git commit --author=[email]
    • git push -u origin operations
    • [enter username and password for github]
    • cd ~/PREX/japan_WAC_only/Parity/prminput
    • git pull
    • redo prompts with cd ~/PREX/prompt ; ./prompt.sh [runnumber]
  9. Pull in changes from regular japan operations branch into the japan_wac_only folder after reviewing the commits made.
  10. Track developments and make presentations from WAC Notes

Online analysis

 A stable grandroot file can be found in /chafs2/work1/apar/aggRootfiles/slugRootfiles/grandRootfile/archive/grand_aggregator_1-30.root which has all slugs between 1-30. 
 Useful cuts: ihwp==1 (in) ihwp==2 (out)
              wein==1 (right) wein==2 (left)
              hrs==0 (both arm) hrs==1 (right arm only) hrs==2 (left arm only)
 To get an idea about variable nomenclature, look in the root tree or the grand_prototype.pdf
  • comptonPlots
  • Check the Feedback log: (/adaqfs/home/apar/PREX/japan_feedback/LogFiles/Feedback_PITA_log.txt)

Accessing onsite computers

With the access to hallgw, one can have access to the onsite computers.

  • Do not connect remotely to the adaq1 computer unless necessary for some DAQ related process
  • Focus on aonl[1-3], especially for analysis jobs
  • For large analysis jobs focus on the ifarm installation of JAPAN


To get access to hallgw:

  • Request to help desk for 2 factor authentication
    • If approved, you will receive an email guiding you through the steps for setting up the access pin and the pin generator app(MobilePASS)
  • In a terminal type in: ssh username@login.jlab.org
  • With access granted, type in: ssh username@hallgw

WAC Notes PREX II | WAC Notes CREX

To calculate statistical precision so far

Exp error bar = sqrt(1/(sum(1/sigma^2)))

Known Problems and Solutions

1) What to do when a run has two distinct bpm12X response regions (Example: run 7842)?

Ans: Look at the variable in the mul tree called "BurstCounter" which encodes minirun index. For run 7842, you can look at mul->Draw("BurstCounter:yield_bpm12X","ErrorFlag==0","") to see that in the current root file, only BurstCounter==4 has the change in the 12X, and then mul->Scan("yield_bpm12X:pattern_number","ErrorFlag==0 && BurstCounter==4") lets you see that if you flag patterns between 50719 and 68050 (first and last entry in the scan output) as bad you'd cut out the events when the position is changing. You can multiply the pattern by 4 to get the event number range or use the event tree to get the range and put that in a prex_bad_events.$run_number$.map file. Then reprompt.

2) What to do when a target position is changed while a run is ongoing (Example: run 7860?

Ans: a) Start by looking at the slow tree to know when the target moved: slow->Draw("pcrex90BDSPOS.VAL:Entry$","","*") For run 7860, by zooming in, I see the target position has stabilized by EPICS event 15.

b) Check the beam current in the slow tree: slow->Draw("hac_bcm_average:Entry$","","*") I can see we didn't have current until EPICS event 45 or so, thus we know we only had current on one target. If we had a run with beam on one target then the target was moved and we wanted to analyze the data after the move, we' need to use an eventNumberCut file to exclude the early events. Or if we only wanted the early events, we could exclude all the events after the target started moving. To do that, you'd use the evt tree to find the CodaEventNumber of the event at which you wanted the cut. I'm going to only keep events after 30000, for this run.

c) Create a special eventNumberCut.7860.conf file. The content should look like this, except the event range will be different for a different run. 

# Cuts to keep only evnts with Ca-48 target in position

event=30000:

blinder.force-target-blindable=1

Then you should be able to replay the run.

3) How to monitor dpp drift?

https://userweb.jlab.org/~sanghwa/prex/ArchiveTools.pdf

4) How to make regression minrun summary and grandaggregator plots?

a) Login to apar@adaq1.

b) gowac

c) make file named slug$number$.list inside run_list folder or update it. Each entry in the file corresponds to a run in the slug

d) ./EZ_WAC.sh run_list/slug$number$.list

Retrieved from "https://prex.jlab.org/wiki/index.php?title=Weekly_Analysis_Coordinator&oldid=11757"