Commissioning and running of sPHENIX

[Stefan Bathe <stefan.bathe AT baruch.cuny.edu>]

Minutes shift change 2023-05-07

general
—————
from Rob:  
Shift crew should NOT be filling the chiller.   That is for the expert crew.  
If it needs filling, can call shana in an emergency through frank.  She is 
willing to come in off hours.   

Again.  Shift crew should NOT be doing any work of any kind in these systems. 
  Read only.  

So chillers are open reservoir. As we work air out of the system they will 
need to have fluid added. After a week or so I expect this to not happen 
anymore. I suspect the chiller that needed adding was the new one we put on 
on Friday which would be expected since it was definitely some air introduced 
when we shut things down and replace the chiller.

evening:  Xuan
- series of chiller audio alarms during first 5 hours, EMCal NW and SW 
  - filled according to Sean’s instructions
- contacted Sean and Lee, Lee rebooted shift GUI
- EMCal expert had requested to tune alarm threshold, which might have caused 
more alarms
  - shift didn’t know threshold had been changed

night:  Hanpu
- also several alarms for NE chiller, went to check, warning symbol on 
chiller: "auto advanced filter removed"
  - longstanding alarm—>should be fixed
- shift checklist/alarm panel reading reading regarding ECW chiller platform 
T:  F/C unit mismatch
- 4am INTT finished test, turned off LV, HV
- 1am:  plugged in same HCal Stefan had plugged in earlier (had tested the 
wrong channel, so not clear whether it slipped back out)

day:  Sookhyun
- four audible alarms that reset themselves
- checked chiller level on vision:  one was low, so refilled until everything 
was 75—>don’t!  Chillers are expert only!
- INTT experts working on fiber communication

MBD (Abdul)
- nothing new

HCal (Hanpu)
- checked TP and LED
  - iHCal West no signal again
    - unplugged/plugged cable from clockmaster to TP
      - don’t think it’s connection issue—>need to fix before door closes 
(maybe try different TP unit or make East/West swaps to identify problem)
- plugged in iHCal cable in bore
- two outer HCal channels have half-high signal, one fixed by unplugging, 
other still issue

EMCal (John H)
- working on readout of bias voltages and control boards, logging, 
identifying bias channels with problems
  - 6 IF boards have difficulties with bias voltage—> investigating

TPOT (Hugo via email)
- John K. and Martin connected TPOT to vGTM12 yesterday, which is confirmed 
to work, so TPOT again able to take noise/pedestal data.
- We did take some data yesterday which are yet to be analized. 
- will do so maybe today or tomorrow, then do some stress test on the whole 
DAQ chain

TPC (Jin)
    • Active readout for TPC pedestal data most time of the day. Shift crew 
do not need to monitor the TPC readout, just following standard TPC standing 
order for detector protection.
    • After John K. updated the kernel driver, stress testing the readout at 
31GB/s (no file logging) since 10PM last night. The patch on kernel driver 
seems quite successful and all sectors have been stable for 16hr+ non-stop 
data taking. https://sphenix-intra.sdcc.bnl.gov/WWW/elog/TPC/71
    • Continued stress test, identifying and mitigating bottlenecks in the 
readout pipeline in coming days.
 
INTT (Rachid)
- after success with decoder Raul took PRDF files with RC_DAQ
  - Martin:  data have very few hits, haven’t seen entire phase space
    - Rachid will ask Raul to pass on recent run numbers
- tomorrow will work on PRDF to online monitoring chain
- one module has issue with bias voltage, (took bias filter to CERN, when 
came back there was this issue)—>debugging
- working on fiber communication with servers
- tomorrow will release LV to shift crew (permanently on)
- bias will be operated by shift crew as per expert request

MVTX  (Zhaozhong)
- no update

DAQ (Martin)
- handed HCal back last night at 8:30pm
- set up two more SEBs
  - 19 for Dan for LLvl1 
  - other one not connected, for Chris P
- TPOT:  now temporarily on vGTM 12
  - blocks 8,9,11 have issues, firmware updated will come next week, then 
reassign vGTMs
- later will ask for another 3 hours with HCal, will talk to Blair and Hanpu


- 
---------------------------------------------------------------------------------
Stefan Bathe
Professor of Physics
Baruch College, CUNY
and RIKEN Visiting Scientist

Baruch:                                     BNL:
17 Lexington Ave                      Bldg. 510
office 940                                  office 2-229
phone 646-660-6272                phone 631-344-8490
----------------------------------------------------------------------------------