Commissioning and running of sPHENIX

["Huang, Jin" <jhuang AT bnl.gov>]

I believe we had a productive discussion today with multiple action item and consensus which has the potential to significantly improve the data logging, in particular for the SDCC. 

• HF Physics gain is proportional to the MB events recorded → max data pipeline usage at the stability limit
• Diffuse laser [reference]: 

• Currently firing at fixed rate relative to GL1 trigger: few kHz; can have very short spacing
• Goal: implementing local vGTM14 busy will allow for laser trigger throttle (+busy recording) 
• Action Item: 

• (after vacation) Joe Mead implementing MEB=0 for standalone vGTMs, which allow for local busy of TPC laser vGTM14
• (Workaround right now): Use GTM1 to run TPC laser -> using front LEMO trigger input to receive sPHENIX GTM trigger -> generate local busy on GTM1 to throttle laser triggering.  

ZS noise suppression tuning: on-going by Christopher Platte, Takao, Charles

• Action Item: deploy ZS threshold tuned to threshold from cutting on noise CCDF 

Pre-post sample reduction: Done Aug-15. Require offline perf. study

RCDAQ compression

• Bug fix, passing standalone test -> ready for deployment
• Action Item:

• Deploy to TPC
• Test on MVTX too
• Require careful monitoring in coming days

Buffer box IO tuning

• We have tuned Lustre stripe-count (=n_RAID) and stripe-size (=RAID chunk * n_disk_inRAID)
• Can we throttle SDCC copying during the data taking? 

• Action Item: place a throttling wait at beginning of the condor copying job to stop copying during data taking

Single disk reading for copying to SDCC?:  

• Martin put in a ton of work, testing new copying scheme; using 240 thread (See above) 

Can we add 64GB mem. buffer between RCDAQ and BBox to ride through short disk high IO?

• Action Item: we will add 750MB buffer per RCDAQ threshold x 80 thread

Shall we remove much of the files in bbox so we sit blow 50% utilization?

• Need to deal with the backlog copying

______________________________

 

Jin HUANG

 

Physicist, Ph.D.

Brookhaven National Laboratory

Physics Department, Bldg 510 C

Upton, NY 11973-5000

 

Office: 631-344-5898

Cell:   757-604-9946

______________________________

 

---

From: Huang, Jin <jhuang AT bnl.gov>
Sent: Monday, August 19, 2024 12:55 PM
To: Sphenix-trigger-l AT lists.bnl.gov <Sphenix-trigger-l AT lists.bnl.gov>; sphenix-tpc-l AT lists.bnl.gov <sphenix-tpc-l AT lists.bnl.gov>
Cc: sphenix-run-l AT lists.bnl.gov <sphenix-run-l AT lists.bnl.gov>
Subject: Workfest on DAQ/Bandwidth

 

Hi Everyone

As mentioned in the weekend shift change meetings, we will have a workfest today 2PM discussing optimization of the DAQ pipeline with the large TPC data stream. 

 

Agenda and connection info at https://indico.bnl.gov/event/24567/ . For those at BNL, we will meet in the 1008 conference room. 

Cheers

Jin

 

______________________________

 

Jin HUANG

 

Physicist, Ph.D.

Brookhaven National Laboratory

Physics Department, Bldg 510 C

Upton, NY 11973-5000

 

Office: 631-344-5898

Cell:   757-604-9946

______________________________

 

---

From: Sphenix-tpc-l <sphenix-tpc-l-bounces AT lists.bnl.gov> on behalf of Huang, Jin <jhuang AT bnl.gov>
Sent: Friday, March 22, 2024 11:18 AM
To: Purschke, Martin <purschke AT bnl.gov>; Sphenix-trigger-l AT lists.bnl.gov <Sphenix-trigger-l AT lists.bnl.gov>; Mead, Joseph <mead AT bnl.gov>
Cc: sphenix-tracking-l AT lists.bnl.gov <sphenix-tracking-l AT lists.bnl.gov>; sphenix-tpc-l AT lists.bnl.gov <sphenix-tpc-l AT lists.bnl.gov>
Subject: [Sphenix-tpc-l] TPC diffuse implementation in GTM

 

Dear DAQ team

As my homework from the Tue DAQ meeting, a consensus has reached about how we would like to operate the TPC diffuse laser after discussion with the TPC operation team at 1008, distortion team, and tracking team. The details are summarized in this slide deck discussed in the tracking meeting today: https://indico.bnl.gov/event/21877/

In quick summary, the laser is fired after a GL1 trigger with a tunable standalone busy, which provide a throttling for lower laser rate than GL1 rate. It would likely require to a readout vGTM busy bits in the GL1 packet, so in offline we can reliably tell whether a laser fired following a GL1 trigger or been busy throttled. Details on the proposed change is summarized as option 1 in Page 5.

Please let us know what you think and anything I can help to move forward.

Cheers

Jin



______________________________

Jin HUANG

Physicist, Ph.D.
Brookhaven National Laboratory
Physics Department, Bldg 510 C
Upton, NY 11973-5000

Office: 631-344-5898
Cell:   757-604-9946
______________________________

-----Original Message-----
From: sPHENIX-trigger-l <sphenix-trigger-l-bounces AT lists.bnl.gov> On Behalf Of Martin Purschke via sPHENIX-trigger-l
Sent: Monday, March 18, 2024 6:49 PM
To: Sphenix-trigger-l AT lists.bnl.gov; Mead, Joseph <mead AT bnl.gov>
Subject: [Sphenix-trigger-l] GL1/GTM-focused DAQ meeting tomorrow (Tuesday Mar 19), 2pm


Dear all, especially Joe,

we want to pow-wow about what we have and what might still be missing with the current GL1/GTM firmware.

Here's the Indico - https://indico.bnl.gov/event/22784/

There are a few others, but central are 2 points -

- there are inconsistencies in how the busy is applied and honored in the various modes.
When a vGTM is in local mode, it appears to issue FAs unconditionally.
We also see that if it is set to accept L1s (in local mode with the "GL1_PERMIT" 0xD reg), it doesn't honor its own busy. The desired behavior is that in local mode it looks at its own DCM_BUSY_MASK and suppresses a L1 from being sent out if we are busy, for both FAs or a GL1-L1.

- the freezing of the BCO values that are sent out with FAs We see this with the TPC (only the FELIX-detectors can see it in the first place) that is running FAs a lot. Joe said he knows why. The desired behavior is that any issued L1 (from FA or GL1-L1) transmits the current BCO.

Other items -
I received v55 that gives us the GL1-P info in the readout data stream, as well as a different busy scheme that I'm working to adapt to. It also has the 2nd switchyard that allows us to cherry-pick the triggers that go on the 16 GL1-P scalers. That needs testing, obviously. Joe also implemented a way to better test the "bunch number sensitivity" - I had earlier tested this in a poor man's fashion by using a delayed fiducial tick copy as a Lemo trigger that so triggers only on the given bunch, depending on the delay.

There is a new issue that (after we never got vGTM1 to work for reasons
unknown) we now see that on our main unit, vGTM3 / INTT behaves erratically. I need a way to see this more directly with some sort of probe, and if we cannot resolve this, I probably need Joe's help to get
  to the bottom of this.

Ok. We will start with this. If we are done in time before 3:30, we may have time to talk about other, general DAQ things.

See you tomorrow,

        Martin


--
Martin L. Purschke, Ph.D.        ;   purschke AT bnl.gov
                                  ;   http://www.phenix.bnl.gov/~purschke
                                  ;
Brookhaven National Laboratory   ;   phone: +1-631-344-5244
Physics Department Bldg 510 C    ;   fax:   +1-631-344-3253
Upton, NY 11973-5000             ;   skype: mpurschke
-----------------------------------------------------------------------
_______________________________________________
sPHENIX-trigger-l mailing list
sPHENIX-trigger-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/sphenix-trigger-l
_______________________________________________
Sphenix-tpc-l mailing list
Sphenix-tpc-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/sphenix-tpc-l