sPHENIX tracking discussion

[Anthony Frawley <afrawley AT fsu.edu>]

To be clear, those hits are still deleted. It is just the hit-truth map entries that are not deleted.

Tony

---

From: Anthony Frawley <afrawley AT fsu.edu>
Sent: Tuesday, September 6, 2022 11:05 PM
To: pinkenburg <pinkenburg AT bnl.gov>; Christof Roland <christof.roland AT cern.ch>
Cc: sphenix-tracking <sphenix-tracking-l AT lists.bnl.gov>
Subject: Re: [Sphenix-tracking-l] PHG4TpcDigitizer timing

 

I just made a PR with a change to PHG4TpcDigitizer that removes the deletion of hit-truth associations from the map for hits with ADC value below threshold. This speeds the digitizer up by more than a factor of 3 in 1000 pion events. As far as I can see, there is no effect on tracking performance or resident memory.

The track evaluator seems to be slower by ~ 15% due to this change. The output file size may be larger also, Chris can check. Ultimately, we plan to dump this map.

See what you think Chris.

Tony

---

From: Anthony Frawley <afrawley AT fsu.edu>
Sent: Monday, September 5, 2022 3:12 PM
To: pinkenburg <pinkenburg AT bnl.gov>; Christof Roland <christof.roland AT cern.ch>
Cc: sphenix-tracking <sphenix-tracking-l AT lists.bnl.gov>
Subject: Re: [Sphenix-tracking-l] PHG4TpcDigitizer timing

 

Hello Christof,

We have already decided to move away from using the hit-truth association map.

I can't do it today, but I will look at the code tomorrow to see if there is a quick way to speed it up.

Cheers

Tony

---

From: sPHENIX-tracking-l <sphenix-tracking-l-bounces AT lists.bnl.gov> on behalf of Christof Roland via sPHENIX-tracking-l <sphenix-tracking-l AT lists.bnl.gov>
Sent: Monday, September 5, 2022 10:18 AM
To: pinkenburg <pinkenburg AT bnl.gov>
Cc: sphenix-tracking <sphenix-tracking-l AT lists.bnl.gov>
Subject: Re: [Sphenix-tracking-l] PHG4TpcDigitizer timing

 

Hi Tony and Chris, 

we should really revisit the logic with adding and removing to (multi)maps, especially when dealing with very large 

sets of hits stored in the maps. We have learned alread a while ago that multimaps don't scale nicely.

We should talk about this in the next simulations or tracking meeting. 

cheers

   Christof 

 

On 5. Sep 2022, at 16:05, pinkenburg via sPHENIX-tracking-l <sphenix-tracking-l AT lists.bnl.gov> wrote:

HI Tony,

I ran callgrind over it, by adding macros one can restrict it to small parts of the code:
https://wiki.sphenix.bnl.gov/index.php/Tools#Advanced_Usage

It spends 77% of its time in

void
TrkrHitTruthAssocv1::removeAssoc(const TrkrDefs::hitsetkey hitsetkey, const TrkrDefs::hitkey hitkey)

 which is 6 lines of code removing an entry from a multimap. Does anyone know how to speed this up? This is by far the most severe limitation on how many events we can run through the simulation chain till the run starts.

Chris

On 9/4/2022 11:29 AM, Anthony Frawley wrote:

Hi Chris,

This may be due to a change I made to the TPC digitizer to properly handle non-zero bunch crossings. It was merged on August 4, I think. The digitizer previously assumed that if a hit had a nominal z position that was negative, it was read out on the south side of the TPC. When I switched to storing TPC time of arrival, I also modified the digitizer code to not make that assumption - the assumption is just wrong if the crossing number is not zero. Now the hits are assigned during hit reconstruction to the readout direction corresponding to their true location in the TPC, and they keep that forever.

The result is that the digitizer previously looped over half of the TPC time bin range for a given hit. The half was chosen based on its nominal z position. Now it loops over the entire range of time(z) bins, so the loop is twice as long. This change is needed to get the correct response for out of time events, which occur even in triggered Au+Au running.

It is very possible that the digitization loop over time bins could be better optimized.

Tony

---

From: sPHENIX-tracking-l <sphenix-tracking-l-bounces AT lists.bnl.gov> on behalf of pinkenburg via sPHENIX-tracking-l <sphenix-tracking-l AT lists.bnl.gov>
Sent: Sunday, September 4, 2022 9:24 AM
To: sphenix-tracking <sphenix-tracking-l AT lists.bnl.gov>
Subject: [Sphenix-tracking-l] PHG4TpcDigitizer timing

 

Hi folks,

until fairly recently the clustering (which I tend to call electron
drift) took about as much time as the G4 simulations themselves. As of
recently the time of this pass has increased by at least a factor of 2
and I see hijing jobs with 400 MinBias events exceeding the 3 day limit
which evicts those jobs so we only get calorimeter clusters from these
sims. You can see the timers in a log from a jobs which finished today:
/direct/sphenix+u/sphnxpro/MDC2/submit/JS_pp200_signal/pass3trk_embed/condor/log/condor_Jet30-0000000040-00175.out

The PHG4TpcDigitizer is by far the biggest contributor now.

Chris

--
*************************************************************

Christopher H. Pinkenburg       ;    pinkenburg AT bnl.gov
                                ;    https://urldefense.com/v3/__http://www.phenix.bnl.gov/*pinkenbu__;fg!!PhOWcWs!xe8FYrNmsJulJp9oBu13WteweyGO1vSpcnB1taOCGMEm8CZmAW-zqiaFMW5F9QITuO1kZ3Z6bazeMWDFX4Uz7NQaAkyskfRdMw$ 

Brookhaven National Laboratory  ;    phone: (631) 344-5692
Physics Department Bldg 510 C   ;    fax:   (631) 344-3253
Upton, NY 11973-5000

*************************************************************

_______________________________________________
sPHENIX-tracking-l mailing list
sPHENIX-tracking-l AT lists.bnl.gov
https://urldefense.com/v3/__https://lists.bnl.gov/mailman/listinfo/sphenix-tracking-l__;!!PhOWcWs!xe8FYrNmsJulJp9oBu13WteweyGO1vSpcnB1taOCGMEm8CZmAW-zqiaFMW5F9QITuO1kZ3Z6bazeMWDFX4Uz7NQaAky4M3DbZA$ 

-- 
*************************************************************

Christopher H. Pinkenburg	;    pinkenburg AT bnl.gov
				;    http://www.phenix.bnl.gov/~pinkenbu

Brookhaven National Laboratory	;    phone: (631) 344-5692
Physics Department Bldg 510 C	;    fax:   (631) 344-3253
Upton, NY 11973-5000

*************************************************************

<output.png>_______________________________________________
sPHENIX-tracking-l mailing list
sPHENIX-tracking-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/sphenix-tracking-l