Difference between revisions of "WAC Notes March 30 2021"

Revision as of 15:15, 30 March 2021

PREX Main << Weekly Analysis Coordinator << WAC Notes CREX << WAC Post-Run_Meeting Notes

WAC Notes March 23 2021 << >> WAC Notes April 6 2021

Organization

Date and time: March 30, 2021, 11am
Bluejeans ID: 564945377
Meeting link: https://bluejeans.com/564945377
Runlist spreadsheet

March 30th 2021

• BCM and BPM Pedestals get a once over: http://ace.phys.virginia.edu/HAPPEX/4507
• Ifarm prompt-install is set up for respin2 (almost finished - still need to pull crex-respin2-alpha and recompile JAPAN).
• Run lists updated - still need to cross check with python sql database interacting scripts
• Other items from before are all addressed (?)

New RCDB/runlist updates

• Cameron has added the label "Good" to a bunch of worthwhile calibration and pedestal runs that need to be kept around for the future (listed at the bottom)
• Cameron has change 5907 Test NeedCut to Production Suspicious (it is a slow target burn in run on Ca48 at low current).

March 23rd 2021

RCDB Update

• RCDB Values have been updated for the runs in question
  • Used /adaqfs/home/apar/pvdb/prex/scripts/update_conditions.py with the data fed into respin2_changes_description.csv and respin2_changes.csv
  • Matches the spreadsheet used over the last few months to track intended changes
  • The AT run period had a spurious newline character at the end of run_type and run_flag for all the runs, which is OK since it was consistent, but it broke the aggregator RCDB parser script... so I've replaced those entries with the non-newline character versions.

Cut maps Update

• Final cut maps have been committed

Beam off Pedestals

• Beam off pedestals have been calculated for all PMT devices - http://ace.phys.virginia.edu/HAPPEX/4502
  • Main and AT detectors determined
  • AT run period 6335-6408 can either use the finely segmented analysis determined during online analysis, or just a flat beam off pedestal that has no interesting features - I suggest the latter
  • The blinded tube data around 8444-8451 needs to be considered as well, likely it can just use the beam off values as well
  • There were some problems with the pedestals implemented for respin1
    • Most notably the blinded tube peds were accidentally applied for all devices and all runs > 8452 (when that should have only been the case for ATL2 and ATR2)
    • The runs < 7500 accidentally used stale pedestal numbers from the April pass of the pedestal analysis
    • So these are big changes, but hopefully small impacts
  • SAM pedestals are also be calculated using this method - there is some disagreement between techniques, so comments are appreciated - see HAPLOG 5403
• The AT period has noteworthy large difference between BCM fit pedestal and beam off pedestal - we will discuss at the 12 PM AT meeting
• Otherwise we are comfortable to use the beam off pedestals as given in the HAPLOGs going forward
• Should make another plot to determine significance of segment-averaging over the beam off pedestals (how wrong is my segment average?)
• Is beam off truly right compared to BCM fit peds? How much of an error bar is this choice introducing
• 3/26/2021 - Additional checks have been run (see new HAPLOG responses) and map files have been committed

BMOD Slopes

• BMOD slopes have been calculated using the simple analysis (5 BPMs, 13746 coil set), for both the standard BPM set and the eigenvector analysis (see recent haplogs). Either or both sets can be used for bmod calculations in JAPAN.
  • Doing eigenvector basis set and normal standard BMOD outputs into map files: /u/group/halla/parity/software/japan_offline/prompt/prex-prompt/bmodAna/plotMacros/maps/
  • Eigenvectors of the three parts of CREX (see http://ace.phys.virginia.edu/HAPPEX/4499) are added to mulc tree in those 3 parts
  • For respin2, should the normal 5bpm regression in JAPAN be replaced (replaced and moved to a "_simple" name) by the stable-eigenvector set determined at the beam corrections meeting last week?
    • Or should this new eigenvector one be added as a supplementary regression output and leave the normal 5 BPM one as it was?
    • If we want to swap over to the 5 stable-averaged evMon BPM set, we could do it for all of the versions of regression included in the tree
    • See also the next point one below
• 3/26/2021 - Additional checks have been run and map files have been committed

TFile truncation

• There are too many correlations and corrections analyses in the JAPAN trees. We should remove > half of them that nobody looks at (like the orig 4aX regression analyses).
  • List at the bottom
  • I suggest that we remove the targBPM and orig sets of analyses, and leave the others for now. A few all and allBPM analyses may even be missing that should be added in?
• 3/26/2021 - Done

New AT+main combos

• Adding in AT combos
  • New mapfiles temporarily live in /adaqfs/home/apar/cameronc/analyses/pedestal-calibrations/beam-off/local-maps-update before getting added into the repo... enough new changes that need to be verified before making large changes in github I feel
  • There is a massive host of potential ways of combining usl, usr, atl1, atl2, atr1, and atr2 - avg and dd-wise
  • us[l,r] vs. at[l,r][1,2] in avg and dd -> 16 possible combos
    • I've chosen to take the usl_atl* avg and dd (4 combos) and usr_atr* avg and dd (another 4). There are another 8 cross combos that are possible - doing usl_atr*, but I've left it out for now
  • us_[l,r avg or dd] vs. at[1,2, or 1+2 or 1-2]_[l,r avg or dd]
    • I've chosen to take us_avg and us_dd and combine it with at1_[l,r avg and l,r dd], but only taking avg with avg and dd with dd (no cross combo of us_avg vs at1_[l,r dd], etc.), which gives another 4 combos for at1s, and another 4 for the at2s - 8 combos
    • Similarly I've also done the at1+at2 case (but no at1-at2 case) -> another 4 combos like the ones right above here
  • See the full list at the bottom
  • The list could be 2 or 4x longer if more kinds of combos are wanted
• 3/26/2021 Done
  • These are also need to be added into the postpan maps

Prompt script, postpan etc.

A few things still need to be done for a respin2 to work

• Done: Run lists need to be regenerated
• Done: Save ifarm prompt respin 1 status
  • Done: Commit last respin1 ifarm prompt folder contents
  • Done: Document what was there
  • Tag as respin1 final
• Set up ifarm prompt for respin2
  • Update postpan maps to include the new combo us_at detectors (match the JAPAN list just committed)
  • Not done: Pull from remote
• Not Done: Update aggregator data list with new useful things (maybe abandon at some point and just use JAPAN's outputs?)

JLab CE 2.4

• Recompile JAPAN with JLab CE 2.4
  • Needs testing
  • Update lots of .sh and .py scripts that explicitly call 2.3
• Robert will test a new compilation

Run list considerations

• Run list comparisons:
  • What have we used for prompt?
    • Prompt exclusively used hand-written run lists, produced by the WACs, with some tendency to error
    • A lot of gaps and mistakes have been weeded out of these lists by subsequent RCDB-script magic
    • /adaqfs/home/apar/PREX/prompt/collector/run_list
    • These are NOT tracked on github, so every time a change has been made... it is kind of lost to time (but also the RCDB isn't tracked ...)
  • What have we used for respin1?
    • Respin1 relied on a combination of scripts parsing the RCDB and also those WAC made lists
    • For determining which runs to analyze - feeding into the respin itself, we looped over all runs and cut against target, production type, run type, etc. and made very generous cuts so that lots of calibration and auxiliary measurements would make it through
    • Respin1 utilized 1605 runs included in /u/group/halla/parity/software/japan_offline/prompt/prex-prompt/crex-runlist/simple_list/all_crex.list
      • Created with /u/group/halla/parity/software/japan_offline/prompt/prex-prompt/crex-runlist/MakeSimpleList_crex.C
      • This technique was also cross checked and exactly verified to match the bash-script technique independently written by Cameron (which is a bit easier to read and modify to do what you want)
      • Simple bash script example: /u/group/halla/parity/software/japan_offline/prompt/prex-prompt/WAC/auto_slug_list.sh
      • Python script that it calls: /u/group/halla/parity/software/japan_offline/prompt/prex-prompt/PVDB/prex/examples/make_run_list.py
    • The slug plots and grand plots however kept the WAC/run_lists sets, which was partially just because the scripts were still pointing to that folder and because it allowed for unambiguous comparisons between prompt and respin1 - this should be changed
  • What should we use for respin2?
    • Stop using the hand written slug lists
    • Shift over fully to using the prex-prompt/crex-runlist/MakeSimpleList_crex.C and prex-prompt/WAC/auto_slug_list.sh
    • Use both scripts and make sure the outputs are identical for identical input cuts
    • Do a comparison of the respin2 vs. respin1 and make sure differences are exactly due to the update of cut information: /adaqfs/home/apar/pvdb/prex/scripts/respin2_changes.csv, respin2_changes_description.csv
    • Do a comparison of these vs. the user-typed run-lists as well (any discrepancies must be explainable)

Offline analyses

• BMOD analysis
• Eigenvector analysis
• Aggregator
• Exact cut comparisons between prompt, respin1, and respin2

March 16th 2021

• Beam off pedestals (for PMT devices) - needs another pass - HAPLOG
  • Seems like a good idea to use bcm_dg_us.hw_sum_raw/num_samples < -150 or so (spot check 20 runs across CREX)
  • Also look at 1D histogram of BCMs
• Commit remaining map/cut updates from giant spreadsheet work
• Update pedestal maps with beam off pedestals, and double check values are all appropriate (Amali, Robert, Cameron)
  • Duplicate usl peds: prex_maindet_pedestal.8453-8506.map, prex_maindet_pedestal.8526-8549.map
• Update RCDB info
• Consider adding bmod slope map files for respin2
• Aim for Friday afternoon initial respin2 test slugs (need to make sure folders and disk usage are all in line too)
• Getting slug lists and such polished can happen after the respin is underway

March 9th 2021

Preparing for respin2 - mid March - 1 week from now

• Runlist finalization
  • Finish up runlist checking here
  • Aq issues and BPM 12X cut ranges
  • Github up to date
  • Apply RCDB updates
  • We need a standard set of cuts and procedure for getting slug lists, which can be deliberated after respin2 begins
• Pedestal updates
  • Use beam off for non-RF systems (does AT data need to have changed pedestals? At least double check them all)
  • Verify segmentation and values for RF systems
  • Refer to beam off pedestals for guidance on applicability ranges
  • Double check PITA scans from respin1 data and BPM calibrations separately
• Aq shift - solved
• Questions on AT data-set (similar to Aq issue)
• Run 8321 config file for file splits?
• Add combos to mulc and regression for main detector + AT combos (usl_atl1_dd, etc.)
• Include SAMs and ATs in BMOD calculations going forward

March 2nd 2021

Preparing for respin2

• Pedestals/new beam trip cut/Aq interrelation question
  • Aq post-trip studies: http://ace.phys.virginia.edu/HAPPEX/4483
    • Things look good when comparing the different time slices post-trip recovery - more plots are needed to determine how well the non-feedback data post-trip explains the shift in Aq from prompt to respin1 data set.
  • Pedestal comparisons for AT: http://ace.phys.virginia.edu/HAPPEX/4482
    • Changing from prompt (wrong) to respin1 (relative to bcm calibrated) pedestals changed the mean asyms by an expected ~3% per target, EXCEPT for Pb208, which changed by ~15%. This needs further investigation to understand why Pb pedestal change affected the asyms so much.

February 23rd 2021

Prior day's notes can be found here

List of objects in the TFile - minimize for sanity and disk space concerns

• There are several kinds of analyses included in the respin1 dataset. Some are holdovers from the prompt days, and most have been superseded or made irrelevant by subsequent offline analysis.
• The list of kinds of analyses are as follows:
  • The usual "lrb" for LinRegBlue
    • this is what we use for regression analysis
  • _alldet
    • These include all of the DVs (SAMs, ATs, maindets) and should be kept (and included for the other analyses below that are useful)
    • The main detectors and ATs have been promoted to global cuts, but the SAMs have not
    • Therefore any analysis that we care to look at for main + AT + SAMs should have a normal version and an ALL version
    • So we get the exact data set for mains + ATs with the normal one, and then a dataset suitable for SAMs with the ALL one.
  • ...lrb_orig
    • This uses BPM 4aX, as the original set of BPMs, but it has been superseded by the 1X based analysis which works for all runs and is better (less singular)
  • mulc_dit
    • This is the BMOD correction tree - it was trivial, left empty for respin1
    • But will be filled with meaningful slopes for respin2
  • ...targBPM
    • This uses linear combinations of the target BPMs to try to access a less singular basis set of BPMs
    • It has been superseded by the eigenvector based regression and BMOD analyses
  • ..._all
    • These analyses do overloaded regression using all 12 X and Y BPMs instrumented in the Hall A arc and in Hall A
    • They sould be left as is, as the overloaded regression results are interesting
    • Still remove the targBPM and orig versions
  • runwise (i.e. not _burst) versions
    • these use the full run to calculate the regression outputs
    • We are stuck with them I think, but nobody has really been locking at run-wise regression over burst-wise
    • It isn't necessary to remove (but may be a good thing to do if disk space concerns really do become an issue - I doubt they will)
  • block[0,1,2,3]
    • These use just the subblocks for the regression analyses. Nobody has actually used these yet, but we should, to make sure there isn't any noteworthy deviation as a function of sub-segmenting in time, or any helicity pickup from inadequate TSettle holdoff
    • These could be jettisoned if space issues become a real concern

 KEY: TList    5408.000_condition;1    object title
 KEY: TList    mapfiles;1      object title
 KEY: TDirectoryFile   evt_histo;1     evt_histo
 KEY: TDirectoryFile   mul_histo;1     mul_histo
 KEY: TDirectoryFile   burst_histo;1   burst_histo
 KEY: TDirectoryFile   objects;1       objects
 KEY: TTree    evt;1   MPS event data tree
 KEY: TTree    mul;1   Helicity event data tree
 KEY: TTree    pr;1    Pair tree
 KEY: TTree    slow;1  EPICS and slow control tree
 KEY: TTree    burst;1 Burst level data tree
 KEY: TTree    evt_bmw;1       Bmod Data
 KEY: TTree    mulc;1  Helicity event data tree (corrected)
 KEY: TTree    mulc_dit;1      Helicity event data tree (BMod corrected)
 KEY: TTree    mulc_dit_combo;1        Helicity event data tree (BMod corrected)
 KEY: TTree    mulc_lrb;1      Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    mulc_lrb_orig;1 Helicity event data tree (corrected by LinRegBlue) with 4aX
 KEY: TTree    mulc_lrb_targBPM;1      Helicity event data tree (corrected by LinRegBlue) with target BPMs
 KEY: TTree    mulc_lrb_alldet;1       Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    mulc_lrb_alldet_orig;1  Helicity event data tree (corrected by LinRegBlue) with 4aX
 KEY: TTree    mulc_lrb_alldet_targBPM;1       Helicity event data tree (corrected by LinRegBlue) with target BPMs
 KEY: TTree    mulc_lrb_all;1  Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    mulc_lrb_burst;1        Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    mulc_lrb_orig_burst;1   Helicity event data tree (corrected by LinRegBlue) with 4aX
 KEY: TTree    mulc_lrb_targBPM_burst;1        Helicity event data tree (corrected by LinRegBlue) with target BPMs
 KEY: TTree    mulc_lrb_alldet_burst;1 Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    mulc_lrb_alldet_orig_burst;1    Helicity event data tree (corrected by LinRegBlue) with 4aX
 KEY: TTree    mulc_lrb_alldet_targBPM_burst;1 Helicity event data tree (corrected by LinRegBlue) with target BPMs
 KEY: TTree    mulc_lrb_all_burst;1    Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    lrb_std;1       Correlations
 KEY: TTree    lrb_std_orig;1  Correlations with 4aX
 KEY: TTree    lrb_std_targBPM;1       Correlations with targetBPMs
 KEY: TTree    lrb_alldet;1    Correlations (All detector channels)
 KEY: TTree    lrb_alldet_orig;1       Correlations (All detector channels) with 4aX
 KEY: TTree    lrb_alldet_targBPM;1    Correlations (All detector channels) with target BPMs
 KEY: TTree    lrb_std_block0;1        Correlations (block0)
 KEY: TTree    lrb_std_block1;1        Correlations (block1)
 KEY: TTree    lrb_std_block2;1        Correlations (block2)
 KEY: TTree    lrb_std_block3;1        Correlations (block3)
 KEY: TTree    lrb_all;1       Correlations (all variables)
 KEY: TTree    burst_mulc;1    Helicity event data tree (corrected)
 KEY: TTree    burst_mulc_dit;1        Helicity event data tree (BMod corrected)
 KEY: TTree    burst_mulc_dit_combo;1  Helicity event data tree (BMod corrected)
 KEY: TTree    burst_mulc_lrb;1        Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    burst_mulc_lrb_orig;1   Helicity event data tree (corrected by LinRegBlue) with 4aX
 KEY: TTree    burst_mulc_lrb_targBPM;1        Helicity event data tree (corrected by LinRegBlue) with target BPMs
 KEY: TTree    burst_mulc_lrb_alldet;1 Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    burst_mulc_lrb_alldet_orig;1    Helicity event data tree (corrected by LinRegBlue) with 4aX
 KEY: TTree    burst_mulc_lrb_alldet_targBPM;1 Helicity event data tree (corrected by LinRegBlue) with target BPMs
 KEY: TTree    burst_mulc_lrb_all;1    Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    burst_mulc_lrb_burst;1  Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    burst_mulc_lrb_orig_burst;1     Helicity event data tree (corrected by LinRegBlue) with 4aX
 KEY: TTree    burst_mulc_lrb_targBPM_burst;1  Helicity event data tree (corrected by LinRegBlue) with target BPMs
 KEY: TTree    burst_mulc_lrb_alldet_burst;1   Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    burst_mulc_lrb_alldet_orig_burst;1      Helicity event data tree (corrected by LinRegBlue) with 4aX
 KEY: TTree    burst_mulc_lrb_alldet_targBPM_burst;1   Helicity event data tree (corrected by LinRegBlue) with target BPMs
 KEY: TTree    burst_mulc_lrb_all_burst;1      Helicity event data tree (corrected by LinRegBlue)
 KEY: TTree    burst_lrb_std;1 Correlations
 KEY: TTree    burst_lrb_std_orig;1    Correlations with 4aX
 KEY: TTree    burst_lrb_std_targBPM;1 Correlations with targetBPMs
 KEY: TTree    burst_lrb_alldet;1      Correlations (All detector channels)
 KEY: TTree    burst_lrb_alldet_orig;1 Correlations (All detector channels) with 4aX
 KEY: TTree    burst_lrb_alldet_targBPM;1      Correlations (All detector channels) with target BPMs
 KEY: TTree    burst_lrb_std_block0;1  Correlations (block0)
 KEY: TTree    burst_lrb_std_block1;1  Correlations (block1)
 KEY: TTree    burst_lrb_std_block2;1  Correlations (block2)
 KEY: TTree    burst_lrb_std_block3;1  Correlations (block3)
 KEY: TTree    burst_lrb_all;1 Correlations (all variables)
 KEY: TTree    evts;1  Running sum tree
 KEY: TTree    muls;1  Running sum tree
 KEY: TTree    bursts;1        Burst running sum tree

New Combinations of AT/US main detectors

 [asym:@asym_usl_atl1_avg]
 asym:usl, +0.5
 asym:atl1, +0.5
 [asym:@asym_usl_atl1_dd]
 asym:usl, +0.5
 asym:atl1, -0.5
 [asym:@asym_usl_atl2_avg]
 asym:usl, +0.5
 asym:atl2, +0.5
 [asym:@asym_usl_atl2_dd]
 asym:usl, +0.5
 asym:atl2, -0.5
 [asym:@asym_usr_atr1_avg]
 asym:usr, +0.5
 asym:atr1, +0.5
 [asym:@asym_usr_atr1_dd]
 asym:usr, +0.5
 asym:atr1, -0.5
 [asym:@asym_usr_atr2_avg]
 asym:usr, +0.5
 asym:atr2, +0.5
 [asym:@asym_usr_atr2_dd]
 asym:usr, +0.5
 asym:atr2, -0.5
 [asym:@asym_us_avg_at1lr_avg_avg]
 asym:usl, +0.5
 asym:usr, +0.5
 asym:atl1, +0.5
 asym:atr1, +0.5
 [asym:@asym_us_avg_at1lr_avg_dd]
 asym:usl, +0.5
 asym:usr, +0.5
 asym:atl1, -0.5
 asym:atr1, -0.5
 [asym:@asym_us_avg_at2lr_avg_avg]
 asym:usl, +0.5
 asym:usr, +0.5
 asym:atl2, +0.5
 asym:atr2, +0.5
 [asym:@asym_us_avg_at2lr_avg_dd]
 asym:usl, +0.5
 asym:usr, +0.5
 asym:atl2, -0.5
 asym:atr2, -0.5
 [asym:@asym_us_avg_atlr_avg_avg]
 asym:usl, +0.5
 asym:usr, +0.5
 asym:atl1, +0.25
 asym:atr1, +0.25
 asym:atl2, +0.25
 asym:atr2, +0.25
 [asym:@asym_us_avg_atlr_avg_dd]
 asym:usl, +0.5
 asym:usr, +0.5
 asym:atl1, -0.25
 asym:atr1, -0.25
 asym:atl2, -0.25
 asym:atr2, -0.25
 [asym:@asym_us_dd_at1lr_dd_avg]
 asym:usl, +0.5
 asym:usr, -0.5
 asym:atl1, +0.5
 asym:atr1, -0.5
 [asym:@asym_us_dd_at1lr_dd_dd]
 asym:usl, +0.5
 asym:usr, -0.5
 asym:atl1, -0.5
 asym:atr1, +0.5
 [asym:@asym_us_dd_at2lr_dd_avg]
 asym:usl, +0.5
 asym:usr, -0.5
 asym:atl2, +0.5
 asym:atr2, -0.5
 [asym:@asym_us_dd_at2lr_dd_dd]
 asym:usl, +0.5
 asym:usr, -0.5
 asym:atl2, -0.5
 asym:atr2, +0.5
 [asym:@asym_us_dd_atlr_dd_avg]
 asym:usl, +0.5
 asym:usr, -0.5
 asym:atl1, +0.25
 asym:atr1, -0.25
 asym:atl2, +0.25
 asym:atr2, -0.25
 [asym:@asym_us_dd_atlr_dd_dd]
 asym:usl, +0.5
 asym:usr, -0.5
 asym:atl1, -0.25
 asym:atr1, +0.25
 asym:atl2, -0.25
 asym:atr2, +0.25

Attendence

Cameron, Victoria, Paul King, Allison Zec, Juliette, Weibin, Ryan, Devi, Kent