Commissioning and running of sPHENIX

[Jamie Nagle <jamie.nagle AT colorado.edu>]

As linked here:   https://docs.google.com/document/d/1HMFx_exZSKlwKSqXyA99afs5rd2jnFbmdlOTJWSRgJQ/edit

--------

General (Stefan/Kin)

• One year ago today:

Current Priority Items (PC)

• 
  

Work Control Coordinators (Chris/Joel)

• 
  

Plan of the Day (Stefan/PC/all–to be revisited at end of meeting)

• Current plan - shift crew & experts running for the weekend
  111x111, 2 mrad, 8 hour stores, field on, global mode, MBD,ZDC,LL1,HCal,EMCal(½)

→ continue updating instruction manual, recovery directions
→ experts requested to check online monitoring and update (MBD, HCal, EMCal)

• Expect 4 hour access sometime on Monday (not yet determined)
  → 4 hours to remove TPC one-laser (Azmoun + Frank)
  → ? MBD to check one cable , test cable/channel swap
  → check out HCal ADC crate (potential voltage/backplane issue)
      4 hours if we need to swap that crate (or may need to plan later)
  → check  out HCal pre-amp issue (maybe 2 hours)
  Other?  – please double check and update Access Request page (snapshot below)
  
  
  
  

Monday-Tuesday this  coming week

• Acquire input from TPC group about next 2 weeks of potential zero field running
  Also, input from Kin on magnet, how long to leave on for now

• Assess information from analysis of MVTX with single beam tests etc., and then follow up  with meeting with machine people

• MBD performance update → coordinate with  Commissioning Task Force

• Priority items for next week:   GL1/GTM firmware update, GL1 readout, RevTick implementation, DAQ monitor, further checks on ADC/XMIT issues
  
  

===================================================

Evening (Derek Anderson, Sijan Regmi, Justin Bryan)

• Inherited fill 33898 from day shift (111x111, 2 mrad, leveled to 100 Hz)

• TPC crew finished at 6:20 pm

• MCR steered out of leveling, and dumped blue beam for MVTX test

• Yellow beam dumped at 8:03 pm after MVTX crew finished

Fill 33899 (111x111, 2 mrad)

• Physics declared at 8:44 

• Afterwards, John and Jaebom worked on running in global mode with MBD+LL1+HCAL+EMCAL (mostly)+ZDC all the way up to the Owl Shift

Night (John Lajoie and Justin Frantz)

• (12:55AM) Store lost due to cryo issues (Cryo comm fault), MCR Called they have someone coming in but it might be a while. INTT expert doing development work.

• (6:41AM) New store 33901, 111x111, 2mrad, ZDC ~17kHz. Called HCAL expert per instructions from previous shift crew

• (7:45AM) Run 14232 ended automatically (sort of) with 1M events!

• 
  

Day (Charles Hughes, Alex Holt, Pedro Nieto)

• (8:00 AM) inherited beam from owl shift

• (8 - 9 AM) owl shift passed over new instructions for DAQ

• Showed how to set up DAQ, start/stop runs/troubleshoot

• John Haggerty started a global mode run w/ EMCAL + HCAL + MBD + LL1 + ZDC

(9:00 AM) Detector Issues

• 1 TPOT FEE card lost link - called Hugo, he fixed it

• EMCAL channels - 1 too high, 1 tripped to 0 

(10:00 AM) DAQ issues

• Tried to troubleshoot issues w/ Jamie Nagle

• Issues related to not seeing live triggers (we do see raw)

• John H. and Jaebeom worked on it

(12:15 PM) DAQ issues *resolved

• Busy mask for HCAL not properly set

• No live triggers for an unknown reason

• UPDATED DAQ INSTRUCTIONS (even from OWL shift)

(01:35 PM) John H. finished up work on ZDC in 1008-B

Very brief interruption to ZDC rate, now back as it was

• (02:45 PM) Planned beam dump for Fill 33901

• EMCAL trips from 9:00 AM recovered by Pedro w/ help of Tim Rinn/John H.

(3:30 PM) Preparing for Next Fill 33902

=============================================================

Magnet (Kin)

• Magnet still at full-field; otherwise nothing to report.
  
  

MBD (Mickey, Lameck, Alex)

• At about 11:00 AM we scaled down the MBD rate (500-1000 Hz) 2mRad xing angle. ZDC 3.5 kHz, and also at 1PM the MBD was further scaled down to (500-800 Hz). while ZDC remained at 3.5kHz.

• The MBD continues to fluctuate (even during a single run) from sometimes showing a clean Z-vertex peak centered near zero to having additional distributions near +- 250 cm we suspect these might be events going to the online monitor missing one packet (1001,1002) or all have zeros.

Trigger (Dan)

• 
  

GTM/GL1 (Martin/Dan/John K)

• 
  

DAQ (Martin)

• I made some updates and clarifications to the current DAQ instructions regarding teh GUI, and some additions

• Note that the LED running for HCal/Emcal (which is still under expert control) cannot be done from our current choice (seb10) of DAQ running host - Virginia and I have been in contact and this remains an expert task for now.  

MVTX (Yasser/Hao-Ren)

(https://www.dropbox.com/s/sft7rhqw75vhkh7/BeamBackground_MVTXerrors_Correlation.pdf?dl=0)

• We start to look at the data with a single beam test.

• Errors are correlated with high hit occupancy events produced by possible nuclear interaction on the beam direction, like Au nucleus? (more examples are in the elog entry: https://sphenix-intra.sdcc.bnl.gov/WWW/elog/MVTX/221)
  

• We also observed a clear correlation of error/nuclear interaction and possible wide beam profiles. (left: blue beam only, right: yellow beam only)

• We removed the entries in the 1008 IR/tunnel Access Requests spreadsheet as the task was completed 

TPC (Jin, Takao, Evgeny, Charles)

• CDU 300 flow rate shows low values (< 60 LPM) sometimes

• No interlocks/alarms

• Asked Rob - he says it's alright (due to weird way flow is measured) …

Yesterday, 1st run with full field + 100 Hz collisions

• Full Drift - 45 kV

• GM HV: 4.45 kV (~ 90% ops)

• No shorts - few trips, overall fairly stable

• (LocalMaxADCinWF - pedestal) -> saturation peaks

• Analysis Ongoing

• Planning for the future

• Call has gone out to plan ahead for “NO MAGNET” runs

• Bob Azmoun wants to use Monday’s access to remove diffuse laser head

• Preparing with him and Frank Toldo

Planning for future collisions - TBD

HCal (Virginia)

• Thanks to shift crew for calling me last night/this morning- looks like we got some good data! (see Mattermost for first looks)

• Online monitoring updated- should now be easy for shift crew to check HCal timing instead of calling the expert. Instructions updated on HCal page and I can show day/evening shift crews after the meeting what to check

• Testing LED running- not ready to be done by shift crew yet

EMCal ()

• 
  

TPOT (Hugo)

• I was called around 9AM due to one lost Link. Fixed by power cycling the relevant FEE. However I noticed that I could not reinitialize the FEE after that (nor get proper telemetry reading). Likely a problem with FELIX board. It however ‘fixed itself’ later on in the morning. Will keep an eye on it. 

• Not clear whether TPOT was included or not during data taking overnight. I noticed that multiple time vgtm12 was ON (together with vgtm0 and vgtm2, which are the TPC ones) and receiving triggers, but without an ongoing rcdaq run. As a result, the data collected by the FELIX board was never flushed by rcdaq, resulting in the buffers getting full. This will need to be clarified at some point (e.g next week). 

• Nothing else to report on the detector side

INTT ()

• 
  

sEPD()

• 
  

Gas/Cooling ()

• 
  

ZDC (John H./Ejiro)

• I changed the Steve splitter/summer module in 1008B to #2.  I did not screw up the coincidence sent to MCR by doing this:

• All 16 analog signals are ok now (see ZDC elog: https://sphenix-intra.sdcc.bnl.gov/WWW/elog/ZDC/230624_145103/allzdc_seb14-00014266-0000.png)

• Timing in the N/S sum channels look good.

Background Counters ()

• 
  

Online Monitoring (Chris)

||------------------------------------------------------------------------------------------
|| James L. Nagle   
|| Professor of Physics, University of Colorado Boulder

|| EMAIL:   jamie.nagle AT colorado.edu
|| SKYPE:  jamie-nagle        
|| WEB:      http://spot.colorado.edu/~naglej 
||------------------------------------------------------------------------------------------