Difference between revisions of "DAQ Troubleshooting"

From PREX Wiki
Jump to: navigation, search
(For Experts)
Line 34: Line 34:
*[[DAQ Network Ethernet]] - exists but empty
*[[DAQ Network Ethernet]] - exists but empty
*[[DAQ Network Portserver]] - exists but empty
*[[DAQ Network Portserver]] - exists but empty
*adaq1:/tmp fills up :  see https://logbooks.jlab.org/entry/3724325#comment-23569
** DAQ Documentation [[DAQ_Doc_Portal | Portal]]
** DAQ Documentation [[DAQ_Doc_Portal | Portal]]

Revision as of 11:13, 26 August 2019

In the process of editing and updating this page. Please contact Juliette Mammei crowder@jlab.org with questions or suggestions.


For shift workers

Figure 1 - CODA window showing EB1 disconnected.
Figure 2 - If you attempt to kill CODA while is it running, this window pops up. Type Y if you really want to kill it.
Figure 3 - If a ROC becomes disconnected, you will see warning messages in the message window.
Figure 4 - There may be a report that a particular ROC is disconnected.
Figure 5 - If the ROC's console is responding, you can type reboot.

  • Either the Event Builder (EB1) or Event Recorder (ER1) are disconnected (see Figure 1)
    • Type kcoda in a terminal, say Y(see Figure 2)
    • startcoda (follow instructions for starting CODA)
    • If that fails try again at least once before moving on to other fixes
    • It is common for the ER1 to fail when going from Configure to Download in the Run Control GUI too quickly, so patience is advised
  • If a ROC is disconnected (see Figures 3 and 4)
    1. Try to reset; if successful you should get Client is back message and everything should report Configured
    2. If a ROC is still reporting a problem - note which ROC is reporting the problem
      1. If the xterm console for that ROC is responding type reboot and enter (see Figure 5)
        1. If the problem persists after reboot try rebooting the other ROCs too, try to restart CODA twice, and call the RC
        2. Also make sure that the Red "platform" GUI has connected before doing ROC reboots
        3. Sometimes it takes several reboots of CODA and ROCs to work - going slow and having patience is key
        4. If the ROCs don't even connect and are unresponsive to the terminal/telnet session then remotely power cycling is necessary
      2. If the xterm console does not respond to input then try to ping the hostname
        1. Hostnames are CH: halladaq6.jlab.org, Injector: qweak1.jlab.org, RHRS: hallavme14.jlab.org, LHRS: happex7.jlab.org
        2. If the ping fails to see the hostname then the ROC is turned off or the network switch and console servers are off too and something needs to be rebooted remotely or manually
      3. Check with an available expert in the Counting House or call the RC

For Experts