Compton DAQ HOWTO
This page is intended for PREX-II/CREX shift crew and instructs the correct usage of the compton DAQ, as well as some troubleshooting procedures.
- 1 Things to Check Before Running Compton
- 2 Compton DAQ CODA
- 3 ROC6 Hard Reboot
- 4 Compton Monitoring
- 5 Troubleshooting Common Problems
Things to Check Before Running Compton
Check that the Compton laser table rates (look for ComptonUSbg and ComptonDSbg on the EPICS computer strip charts) are acceptable.
Values will be alarmed and will go off in case of quick spikes. If you don't know what the "acceptable" values are, then ask at the start of your shift.
Compton DAQ CODA
- Log in to compton vnc port 9 with
vncviewer compton:9 --shared. Passwd is in the usual place.
- If CODA is not open in the vnc, then in the ssh terminal, and run kcoda (to be safe) and then startcoda
- The Compton CODA looks almost exactly like the parity DAQ CODA with one extra window: the blue terminal window titled "ROC6." When you start up the Compton CODA if ROC6 keeps spitting out an error called udpSend: write failure then you've done it correctly.
- With the compton CODA opened in the VNC window, first click Platform -> Connect.
- Afterwards click "Configure."
- The final line in the ROC6 window should be an error message saying callbackControl: Do not understand the command: session/control/setSession. If you see this error, then you've done it correctly.
- Then click "Download."
- If you've done it correctly, then ER1, EB6, and ROC6 will all read "downloaded" as a state.
- If ROC6 begins showing errors other than the ones mentioned above, then contact the RC immediately.
IMPORTANT: Before Starting a Compton Production Run
- Check the EPICS plot of the variable "ComptonCentralRate"
- If this rate is exactly zero then the photon detector high voltage is off.
- If a compton expert is present, ask them for permission to turn it on
- If no compton expert is present then call Adam to have them help determine if it is safe to turn on
- VERY IMPORTANT: do not turn on photon detector without authorization from a compton expert.
- If photon detector HV is already on, then it's okay to proceed.
Starting and Stopping a Compton Run
- Once CODA is open, configured, and the ROCs downloaded to, simply click "Start" to start a compton run.
- Comment with the run configuration, and the current status of the beam.
- If possible have all compton runs last approximately 120 minutes. At 120 Hz that means once every other parity run that you start, you should also start a new compton run.
- To stop a compton run, simply click "Stop."
- After stopping each run, follow the steps to analyze it below:
Analyzing Compton Data
The compton has a much simpler and faster analysis procedure than the parity DAQ. To analyze a compton run:
- Open a new terminal window.
- Enter command
- Enter command
./online.sh -r <run_num>
- The script run without any flags will automatically place plot PDFs in the hallaweb comptonPlots directory, but they won't appear on the webpage until the compton shifter later decides which snail they're in.
- Not necessary but if you're curious you can run
./online.sh --helpto learn what the flags are and what they do.
That's it! If you see any errors that indicate replaying did not end successfully, then inform the compton expert on-call.
ROC6 Hard Reboot
The following instructions detail how to restart ROC6 if/when it dies during compton operation:
- In either the vnc or on the compton machine open a Firefox window. Go to the page hareboot4.jlab.org or [follow this link].
- Login with the correct hlauser credentials.
- Go to outlet 8 and do "Immediate off."
- Wait at least 30 seconds
- Select "Immediate On" for outlet 8.
- Check CODA to see if ROC6 is responding.
- Enter the command
telnet hatsv5 2005after selecting "Immediate On" to monitor ROC6 startup.
- If you see the VXWorks logo in ascii-art pop up, startup is proceeding regularly.
- Exit the environment with "Ctrl+]" and
IOC hacp Reboot
Have the RC contact the compton laser expert on call. This reboot should be entirely expert-driven.
Monitoring a device as sensitive as the compton polarimeter can be a daunting task, but we've done as much as possible to streamline the process for shifters. Monitoring the compton simply means paying occasional attention to our StripCharts on the EPICS computer. The EPICS variables to monitor are in the following table:
|EPICS Name||Variable Explanation||Alarmed?|
|IBC1H04CRCUR2||Hall A Beam Current|
|ComptonUSbg1||Upstream background detector 1||Yes|
|ComptonUSbg2||Upstream background detector 2||Yes|
|ComptonDSbg1||Downstream background detector 1||Yes|
|ComptonDSbg2||Downstream background detector 2||Yes|
|ComptonCentralRate||Photon detector rate||No|
|COMPTON_PW1PCAV_ca||In-cavity laser power||Parity Alarms|
|MCP1P01M||The compton dipole current||No|
|IPM1P02A.XPOS||BPM 2A x position||No|
|IPM1P02B.XPOS||BPM 2B x position||No|
|IPM1P02A.YPOS||BPM 2A y position||Parity Alarms|
|IPM1P02B.YPOS||BPM 2B y position||Parity Alarms|
Compton Background Rates
These are simply the rates of background radiation on our laser table. The lower the better, but we can't possibly expect zero. They're set to alarm if they exceed 7 kHz, so if they go off that'll be the primary indicator of a problem. A couple of things:
- The rates will spike when starting a new compton run because the scaler gets cleared. If this happens just clear alarms, it's not a problem. It's perfectly normal behavior. Just clear the alarms and continue.
- The rates will spike during a harp scan along the 1C line. These are normal, alarms from this can be cleared.
- NOTE: MCC has been told to warn us before doing harp scans on the 1C line. If you get such a warning you should shut off the photon detector until they are done.
If rates are too high call MCC to get them brought back down.
Compton Photon Detector Rate
The metric we use is ComptonCentralRate which tracks roughly as beamCurrent*(laserPower + someBackground). So you should see ComptonCentralRate track with the laser power while the beam is up, and when it's down you should see it fall to approximately 0. There are a number of problems that can impact this measure, and as such, ruin our data. They are:
- Beam is going through the chicane without position lock on or locked in a bad position such that we miss the laser
- The laser autolock has failed
- Photon detector has been pulled out of the path of the photon beam
- The beam tune going through the chicane is not tight enough
If you suspect something is wrong with ComptonCentralRate first ask: is the central rate very clearly tracking with laser state? If yes, then you are not missing the laser. Has the laser increased to at least 2 kW in the past ten minutes? If yes, the laser cycling is fine. Consider this handy table when deciding if something is wrong with the beam configuration in the chicane:
|If ComptonCentralRate is...||then I should...|
|>200 kHz when laser is locked||not worry.|
|170-200 kHz when laser is locked||not take any action yet, but continue to monitor to see if it gets worse.|
|140-170 kHz when laser is locked||attempt a steering only if you are familiar with the procedure and you have the RC's permission. Otherwise, continue to monitor.|
|<140 kHz when laser is locked||call compton expert immediately to request an investigation/potential steering.|
|not tracking with laser state at all||call compton expert immediately to request an investigation/potential steering.|
|exactly zero regardless of laser or beam state||check the compton HV settings to see if the photon detector is off.|
Compton Positions & HallAComLock
The compton has two BPMs on the laser table: IPM1P02A and IPM1P02B. During production both these BPMs should have a steady position lock engaged. The y-position is much more critical for production running than the x-position.
The name of the position lock we run for production is HallAComLock and it locks ONLY to IPM1P02B.YPOS. Because the other three BPM positions are allowed to drift we will expect the optimal position for HallAComLock to change over long time intervals, or when coming back from long downs/re-tuning. IF this lock is not engaged, or is in a bad position then the beam may partially, or wholly miss the laser.
2By appears to drift, call MCC and ask that HallAComLock be re-engaged. If HallAComLock is already engaged then it's probably running at a bad position, and you should call a compton expert for advice. In that case they will ask you to lower the current to 10 uA or something similar and enter a new value for HallAComLock and monitor ComptonCentralRate to see if it improves. Shifters should generally not attempt this without a compton expert guiding them, but may be allowed to under exceptional circumstances or if the RC gives their explicit permission.
The laser cavity locks itself on and off automatically for brief intervals. Laser off periods lasting greater than 10 minutes (while taking production) should be cause for concern. If that is the case contact compton expert immediately. Untrained people should not attempt to do any laser operations.
The HV is controlled with a GUI similar to the main detectors. Instructions for accessing it are found at this wiki page here. IF you're even slightly unsure if the PMT is supposed to be on or off please call Adam immediately.
And that's it! Rates and position locks are the most critical. Keep a sharp eye out!
Troubleshooting Common Problems
A few common problems have occurred when running the compton and should have easy fixes. A few of them are:
"udpSend: write failure" shows up in the ROC6 terminal
- "udpSend: write failure" is actually a good thing, because it means your telnet connection has succeeded, but the ROC recognizes that you have not yet restarted a new run and hasn't initiated the normal protocol with CODA.
- See for reference a Compton DAQ restart that looks bad, but is actually just a CODA session in need of an extra try on the start buttons: https://logbooks.jlab.org/entry/3806858
- Coming from a Compton DAQ crash which was recovered just with kcoda and startcoda: https://logbooks.jlab.org/entry/3806857
CODA is saying something disconnected and is asking me if I want to reset.
- This is a common occurrence. Proceed with the reset. Re-configure and re-download when you're done. If CODA still doesn't behave after that, then run
startcoda(up to five times if necessary) to attempt to recover.
CODA hangs at the end of a run in a "Waiting for..." state indefinitely.
- Reset CODA if the option is available to you. If the reset works, try configuring and downloading again. If that doesn't work, or you cannot reset CODA, then simply run
kcoda(multiple times if necessary) and then
CODA says it can't communicate with ROC6.
- First attempt a reset. If that fails, click on the blue "ROC6" window. Press enter a few times. If a new line pops up, simply
startcodaagain. If you cannot get a new line on that window then call the RC and instruct them to call Adam Zec and tell them that ROC6 needs a manual power cycle.
The VNC crashed on me!
- First, make sure you can still ssh into the compton computer. If yes, then it's likely just a network hiccup and you can start running again. If you can't, briefly walk over to the compton terminal (the one farthest from the door in the counting house back room) to make sure it's still running. If not, then contact the RC who should then contact a compton expert.
The CentralRate and/or the background rates are stuck at 0 or they don't seem to be updating!
- This is probably just ROC6 getting stuck in a bad state. Power cycle it with the above instructions. If problem persists contact compton expert.
The compton rates are too low/not tracking with laser state!
- See #Compton Photon Detector Rate for advice.
The background rates are too high at full current and they won't go down!
- Call RC. Tell them what the background rates are at full current. They may decide to lower the current for the time being. They may request a re-tune. Follow the RC's instructions.