Commissioning and running of sPHENIX

[Stefan Bathe <stefan.bathe AT baruch.cuny.edu>]

Minutes shift change 2023-05-06

general
—————
- from Rob:  Shift crew should not be leaving the control window for the TPC 
or INTTs open in the control room.  It’s is preferable that the crew never 
pull this window up.  It is too easy to accidentally hit stop. 

- Kin:  CAS watch increased from 1 to 2 (running mode)
  - MCR not staffed this weekend (talk to CAS watch instead of MCR if you 
need them)
    - ATR check list finished, but RHIC check list not finished, probably 
will be finished Monday

evening:  Xuan
- GUI issue with monitor shift leader TPC TPOT CDU chiller window shut, Rob 
rebooted GUI, also contacted Lee, resolved at 5:40pm
- EMCal expert power cycled 1W3
- EMCal SW LV was off, turned on (not clear why LV was off)
- 7-10:40pm 10 times alarms chiller TPC TPOT, minor issue
- EMCal NW NE alarms
- INTT DAQ work
—> follow up with Joel and Rob Monday about spurious alarms

night:  Hanpu
- spurious alarms like :  Aggregate chiller audio alarm detected, 
acknowledged.  "Durachill NW encountered a new error.”
- 2 kinds:  
  - 1st no input to overhead panel, ring for 10 s, if not acknowledged, 
message:  “all system functioning with normal parameters: then stop
  - 2nd:  old problem, needs to be acknowledged
- 4:00am Martin handed HCal back to Hanpu, after that cosmic run
- 7:00am police came to check if doors were locked
  - but no doors were locked
  —> Martin:  add this to checklist of evening shift to lock doors at 5:00pm

day:  Sookhyun
- TPOT gas back to normal
- TPOT gas bottle replaced but not notified shift crew
- alarms similar to what Hanpu described
- John K worked on TPOT grounding
- coolant additive empty, Rob:  not important


MBD (Abdul)
- nothing new

HCal (Hanpu)
- cosmics from 4:00am
- Blair took cosmics during day
- Martin communicated with Blair, Martin took over
  - currently in Martin’s hands

EMCal (John)
- Tim and John ran control cable to sector 22, cured IF board problem
- Eric installed MPOD controller for last crate of 8 crates of EMCal bias 
supply
- energized all bias for entire detector
- LED run, looking at current (I) characteristics

TPOT (John K)
- fixed grounding connections in the bore, will test the impact on noise as 
soon as we get a chance
- all grounds solid now
- once John K and Martin sort out GTM:  Hugo will take noise data 

TPC (Jin)
    • Apparent chiller communication issue, which I assume it is low priority 
and Rob would check it next 
week:https://sphenix-intra.sdcc.bnl.gov/WWW/elog/Run+2023+Log/593
    • We were logging data with >99.5% of FEEs last night for a 12hr-long 
pedestal run. Electronics are quite stable. 
https://sphenix-intra.sdcc.bnl.gov/WWW/elog/TPC/67
    • One known problem is DMA buffer wrap around error, which John K. 
patched, in test and released to half of the fleet. 
https://sphenix-intra.sdcc.bnl.gov/WWW/elog/TPC/69
    • Continue working on straighten out bottle necks in the data pipeline to 
reach max readout :https://sphenix-intra.sdcc.bnl.gov/WWW/elog/TPC/66
    • Offline analysis by Thomas and Adi (UCLA) showed new noise sources that 
is concentrated to several GEM modules. Under investigation. 
https://sphenix-intra.sdcc.bnl.gov/WWW/elog/TPC/67
- Hugo:  what is reliable number of samples?  John K:  depends on firmware 
version; 32 is OK for TPOT for current setup, will be higher after firmware 
and kernel update

INTT (Martin)
- have working version of decoder
- about to check in
- last night Raul and Martin took data with Raul’s latest version of 
firmware, progress with that


MVTX  (Zhaozhong)
- 2 spare datas cable replaced
- not plugged into 2E1 rack and patch panel
- problem with CAEN system
  - needs to be solved first:  waiting for Joe Schambach, Yasser

DAQ 
- Martin has been working on calorimeter readout
- see DAQ e-log entry 17; need Chi’s help
  - Stefan:  how do we get Chi’s help on this?
  - John H:  needs more information for Chi to debug
    - John didn’t see this before in highbay
   - can this be reproduced in a small system?
   - can we live with this?
- Martin worked on INTT decoder
- next TPOT GTM
- John H:  goal for next week:  need released version where we normally run 
run control and have people take data from everything that stands still 
during the day with regular starts, stops and get statistics of how reliable 
that is
- development:  external triggers
  - Martin:  works
- John K:  finished updating other half of EBDC servers, no problems anymore
  - still issue with TPOT FEE firmware, will be done after FELIX firmware 
update 


- 
---------------------------------------------------------------------------------
Stefan Bathe
Professor of Physics
Baruch College, CUNY
and RIKEN Visiting Scientist

Baruch:                                     BNL:
17 Lexington Ave                      Bldg. 510
office 940                                  office 2-229
phone 646-660-6272                phone 631-344-8490
----------------------------------------------------------------------------------