sPHENIX discussion of software

[Martin Purschke <purschke AT bnl.gov>]

All,

with the current TPC hitformat 121 and the assorted v4 of the decoder 
class, getting values from the class has been slowing to a crawl. I have 
written all but a handful of our decoders, with this one among the ones 
that are maintained by the detector group. My attitude is is, "you 
change it, you maintain it". But this current implementation is not 
getting us anywhere.

121 introduced a variable-length waveform, with typically not more than 
a dozen or so values above threshold. The decoding implementation that 
was chosen makes each possible waveform the max length of 1024 elements, 
and preset all 1024 elements with an impossible value of 65,000, then 
overwrite the actually present elements with their actual values. So you 
typically see a few values in a sea of 65000 values.

I'll refrain from commenting on that approach. A large fraction of the 
time spent in the analysis is for filling and eventually deleting those 
waveforms.

Initially the setting up of those 65,000-valued vector elements was done 
by 1024 individual push_back calls, which is glacially slow. I believe 
it was Chris who changed this to a resize(1024) operation, followed by a 
faster std:fill. Even faster is vector.resize(1024, 65000) that does 
that in one fell swoop and saves 1024 value-changes - resize(1024) sets 
all elements to 0 and we then set 65,000). I do that in the TPC pool and 
it's saves time - there are 1000s of waveforms, and it adds up.

However, I'm not going on about this to actually save this approach. I 
will change this to a more reasonable implementation that only brings up 
the elements that are actually present. Instead of having the values 
buried in a sea of 65000s

>         65000 65000 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
. . .   many lines deleted ....


> 65000 65000 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000     143     451
> 449     182 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
>                                                        65000 65000 
> 65000 65000 65000 65000 65000 65000 65000 65000
. . . more lines deleted ...

We just bring out the 4 samples  143   451  449  182 (looks like the 
laser, not sure) with an index, and are done.

I'm doing just that in the TPC pool and I feel we can gain significant 
factors of speed-up - I would then back-port this to the idtpcv4 class.

The other improvement is "selective" decoding that Chris has already 
started. Until recently, all decoder classes would either decode or not, 
depending on what you are asking of it. For example, if you just make a 
"dlist", there is nothing that triggers the decoding because we are not 
asking for any decoded value. Typically, the first time you make an 
iValue or similar call, the decoding will be triggered (and cached - 
it's only done once per packet),

Most decoders are fast enough that there is one decoding process that 
does it all. Here, we could have different levels of decoding - you ask 
for this aspect, there is no need to also decode the entire thing, etc. 
That will take some doing. Again, the pool is the first to get that, and 
gets back-ported then.

Anyway, in the time slices I have left I'll work on this. We are 
currently in a situation that we cannot verify the validity of the TPC 
data systematically and quickly due to the excessive processing times, 
and that has to change.

Best,

    Martin

--
Martin L. Purschke, Ph.D.        ;   purschke AT bnl.gov
                                 ;   http://www.phenix.bnl.gov/~purschke
                                 ;
Brookhaven National Laboratory   ;   phone: +1-631-344-5244
Physics Department Bldg 510 C    ;   fax:   +1-631-344-3253
Upton, NY 11973-5000             ;   skype: mpurschke
-----------------------------------------------------------------------