Commissioning and running of sPHENIX

[Stefan Bathe <stefan.bathe AT baruch.cuny.edu>]

Minutes shift change 2023-04-30

evening:  Genki
- water leak found in IR, came from roof, Chris called CAS watch
  - Chris will talk to Marianna tomorrow
- TPC expert turned LV on
- EMCal:  value for one corner on LV GUI turned to red, expert (Sean) knows, 
will work on it tomorrow
  - probably problem with network connection
- HCal took cosmic data overnight
- TPC warning on cooling monitor for coolant (same situation in past)
  - checked coolant, didn’t look empty
  - —> Stefan got in touch with Rob —> Rob says ignore

night:  David
- passed on conditions continued on through night
- temporarily (10 minutes) lost elog and run web site 

day:  Zhaozhong
- police stopped by to check if doors are locked
  - Martin:  door across from trailer must be locked off hours, but not 
emergency door (glass door)
- TPC warning persisted
  - flow rate 58 l/min (slightly less than minimum 60) -> Rob says it’s fine 
as long as there’s no interlock and no overheating
- Sean called in today asking about status of EMCal LV SE side
  - from control panel see red from interlock and occupancy box
  - voltages and currents look OK, vision monitor OK, rack OK
- TPC telemetry lost, Jin said not big problem

MBD (Mickey)
- today measurements on electronics board needed to set gain
- tomorrow new subsystem expert:  Abdul

HCal (Hanpu)
- yesterday working on trying to take cosmics with just east side, not stuck 
anymore, only two runs stuck at large number, other runs finish (long runs) 
with 1.5 kHz scheduler
- haven’t tried running only west b/c of digitizer trigger not functioning 
(error message)
  - Martin:  set rollover limit at 20 GB (for transfer to SDCC):  
rcdaq_client daq_set_rollover_limit 20
- Emma:  when to connect to GTM:  error connection refused
  - now back
- Martin advises to use standard setup file instead of test setup file, which 
was only meant for temporary use

EMCal
- no report

TPOT (Hugo by email)
- Nothing to report from TPOT
- The detector FEE has been Off since yesterday noon, following to chiller 
problem that was fixed later on. 
- Hugo turned it ON this morning.
- he will discuss next week with the TPC crew about turning TPOT back on 
together with the TPC when necessary, since they are in the same racks, uses 
the same electronics, power supply and chiller. 
-  turned TPOT off again because there are some problems with the telemetry 
(no access to FEE temperature) He'll check that again with Takao and Jin 
later today.
- Jin will try to recover TPOT telemetry

TPC (Jin)
    • After Rob gave clear for LV operation, Takao turned on the FEEs. 96% of 
the FEE has link at power on, and we decided it do not worth the effort to 
recover links to the last 4% FEEs during the weekend. 
https://sphenix-intra.sdcc.bnl.gov/WWW/elog/TPC/44
    • Pedestal data taking for 1.5 hrs around mid-night, with 22 out of 24 
TPC sectors. Two servers did not observe GTM trigger, including the expected 
missing sector 13 whose GTM connection was used for TPOT temporarily. When 
stopping the run, there was a GTM RPC error. 
https://sphenix-intra.sdcc.bnl.gov/WWW/elog/TPC/46
    • Around 11:20AM, the 22 server who could take data with GTM lost FEE 
telemetry with FEE communication logic stuck in FELIX. This event was 
correlated with a GTM reboot at 11:24AM. Shift crew and I cannot figure out 
who rebooted it. https://sphenix-intra.sdcc.bnl.gov/WWW/elog/TPC/48
    • To shift crew: we do not reply on FEE telemetry for the safety of TPC 
operation. Loss of telemetry is an inconvenience but not a big problem. 
Noting major changes in TPC ELog is nonetheless very useful and thanks to 
Emma to do so!

INTT (Genki)
- activity ongoing as planned
  - made new LV GUI:  will ask shift crew to monitor only that one from Monday
  - Raul and Genki worked on pedestal runs
  - Itaru:  started working on masking hot channels, 80 % done, 20 % to go, 
will continue this work

MVTX (Zhaozhong)
- nothing

sEPD
- no report

DAQ (Martin)
- daq elog:  new gtm_show_modebits command
- Jin:  GTM was rebooted this morning, unclear who did it
  - Martin:  tomorrow make it so it always restarts when it crashes
                 - sometimes when rebooted doesn’t see RHIC clock (noise?)
                 - don’t reboot if not absolutely necessary (only if you 
can’t log in)
- Martin wants to run HCal in about half an hours

Magnet (Kin)
- nothing

Plan of the day
- DAQ development
- subsystem testing

- 
---------------------------------------------------------------------------------
Stefan Bathe
Professor of Physics
Baruch College, CUNY
and RIKEN Visiting Scientist

Baruch:                                     BNL:
17 Lexington Ave                      Bldg. 510
office 940                                  office 2-229
phone 646-660-6272                phone 631-344-8490
----------------------------------------------------------------------------------