sPHENIX discussion of software

[Christof E Roland <cer AT mit.edu>]

Hi Hugo, 

> On 8. Apr 2022, at 19:37, Hugo Pereira Da Costa 
> <hugo.pereira-da-costa AT cea.fr> wrote:
> 
> 
> On 4/8/22 11:31, Christof E Roland wrote:
>> Hi Hugo,
>> 
>> yes, that would definitely help.
>> I have this on my todo list as well for the time being, but there are a 
>> significant number of this that I have to get sorted before attacking this.
>> If you can contribute here that would be cool.
>> 
>> Lets discuss on monday.
>> 
>> Cheers
>> 
>>    Christof
> 
> Of course, since then, I have thought about complications :)
> 
> If we use a vector, the TpcClusterCleaner will put "holes" in the vector, 
> unless we change the "good" cluster position in the vector, and thus change 
> their key.

I think holes can be avoided easily. First of all we should not delete 
clusters, but set an invalid flag. TrkrClusterv4 has that. 
If we have a vector of pointers to clusters we just have to make sure that if 
we make a copy of the ClusterContainer (e.g. for corrected clusters) we 
allocate an equal size 
vector filled with nullptr where we so not have a cluster.

> Worse, the TPCClusterMover which takes cluster from the original map that 
> are associated to tracks, apply distortion corrections, put them back on 
> TPC surfaces, and copy them in another map/vector, will also either 
> introduce many holes, or change the cluster key.

This is a problem for the current layout as well. we have to see if this is 
acutally happening or how to avoid it.

cheers

   Christof 


> Definitly, there is room for discussion !
> 
> 
> Hugo
> 
> 
>> 
>>> On 8. Apr 2022, at 16:23, Hugo Pereira Da Costa via sPHENIX-software-l 
>>> <sphenix-software-l AT lists.bnl.gov> wrote:
>>> 
>>> Hi Christoph,
>>> 
>>> Thanks for the investigation.
>>> 
>>> If no one else volunteer I can work on replacing the 
>>> std::map<cluster_key, cluster_ptr> by a vector as discussed in another 
>>> email. I don't expect it to be more than a day or two of work.
>>> 
>>> Maybe we can discuss this on monday ?
>>> 
>>> Hugo
>>> 
>>> 
>>> On 4/8/22 05:46, Christof E Roland via sPHENIX-software-l wrote:
>>>> Hi,
>>>> 
>>>> following our discussion on Tuesday i did a quick study on our storage 
>>>> containers, especially the ones using maps, i.e.  the hitsetcontainer.
>>>> The hitsetcontainer is the construct storing what is, from the reco 
>>>> software point of view, the closest representation of raw data we have 
>>>> defined.
>>>> 
>>>> I filled the hitsetcontainer with either 5 k or ~5.5 million fake hits 
>>>> per event and looked at the evolution of the memory footprint of the 
>>>> jobs using
>>>> the prmon tool. From the difference between the low and high occupancy 
>>>> case I canlculate the job memory "price tag" of storing a hit, i.e. 3 
>>>> unint32 - 12 bytes.
>>>> 
>>>> Storing 5 million extra hits in a std map increases the job memory by 
>>>> 0.5GB. This corresponds to a ~100byte effective memory price tag 
>>>> including all the overhead
>>>> and pre allocated memory around stl maps.
>>>> 
>>>> Using TObjArrays its about 60 byte
>>>> 
>>>> For stl vectors its ~14 which, given the precision with which you can 
>>>> read off the GB job memory scale in prmon, is equal to the 12 bytes you 
>>>> expect.
>>>> 
>>>> I guess we should investigate to which extend we can get rid of maps 
>>>> wherever we can.
>>>> 
>>>> Cheers
>>>> 
>>>>    Christof
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> sPHENIX-software-l mailing list
>>>> sPHENIX-software-l AT lists.bnl.gov
>>>> https://lists.bnl.gov/mailman/listinfo/sphenix-software-l
>>> _______________________________________________
>>> sPHENIX-software-l mailing list
>>> sPHENIX-software-l AT lists.bnl.gov
>>> https://lists.bnl.gov/mailman/listinfo/sphenix-software-l

Attachment: smime.p7s
Description: S/MIME cryptographic signature