sphenix-software-l AT lists.bnl.gov
Subject: sPHENIX discussion of software
List archive
Re: [Sphenix-software-l] Quick study on storage containers
- From: Christof E Roland <cer AT mit.edu>
- To: Hugo Pereira Da Costa <hugo.pereira-da-costa AT cea.fr>
- Cc: "sphenix-software-l AT lists.bnl.gov" <sphenix-software-l AT lists.bnl.gov>
- Subject: Re: [Sphenix-software-l] Quick study on storage containers
- Date: Fri, 8 Apr 2022 17:31:49 +0000
Hi Hugo,
yes, that would definitely help.
I have this on my todo list as well for the time being, but there are a
significant number of this that I have to get sorted before attacking this.
If you can contribute here that would be cool.
Lets discuss on monday.
Cheers
Christof
> On 8. Apr 2022, at 16:23, Hugo Pereira Da Costa via sPHENIX-software-l
> <sphenix-software-l AT lists.bnl.gov> wrote:
>
> Hi Christoph,
>
> Thanks for the investigation.
>
> If no one else volunteer I can work on replacing the std::map<cluster_key,
> cluster_ptr> by a vector as discussed in another email. I don't expect it
> to be more than a day or two of work.
>
> Maybe we can discuss this on monday ?
>
> Hugo
>
>
> On 4/8/22 05:46, Christof E Roland via sPHENIX-software-l wrote:
>> Hi,
>>
>> following our discussion on Tuesday i did a quick study on our storage
>> containers, especially the ones using maps, i.e. the hitsetcontainer.
>> The hitsetcontainer is the construct storing what is, from the reco
>> software point of view, the closest representation of raw data we have
>> defined.
>>
>> I filled the hitsetcontainer with either 5 k or ~5.5 million fake hits per
>> event and looked at the evolution of the memory footprint of the jobs using
>> the prmon tool. From the difference between the low and high occupancy
>> case I canlculate the job memory "price tag" of storing a hit, i.e. 3
>> unint32 - 12 bytes.
>>
>> Storing 5 million extra hits in a std map increases the job memory by
>> 0.5GB. This corresponds to a ~100byte effective memory price tag including
>> all the overhead
>> and pre allocated memory around stl maps.
>>
>> Using TObjArrays its about 60 byte
>>
>> For stl vectors its ~14 which, given the precision with which you can read
>> off the GB job memory scale in prmon, is equal to the 12 bytes you expect.
>>
>> I guess we should investigate to which extend we can get rid of maps
>> wherever we can.
>>
>> Cheers
>>
>> Christof
>>
>>
>>
>> _______________________________________________
>> sPHENIX-software-l mailing list
>> sPHENIX-software-l AT lists.bnl.gov
>> https://lists.bnl.gov/mailman/listinfo/sphenix-software-l
> _______________________________________________
> sPHENIX-software-l mailing list
> sPHENIX-software-l AT lists.bnl.gov
> https://lists.bnl.gov/mailman/listinfo/sphenix-software-l
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
-
[Sphenix-software-l] Quick study on storage containers,
Christof E Roland, 04/08/2022
-
Re: [Sphenix-software-l] Quick study on storage containers,
Hugo Pereira Da Costa, 04/08/2022
-
Re: [Sphenix-software-l] Quick study on storage containers,
Christof E Roland, 04/08/2022
-
Re: [Sphenix-software-l] Quick study on storage containers,
Hugo Pereira Da Costa, 04/08/2022
- Re: [Sphenix-software-l] Quick study on storage containers, Christof E Roland, 04/08/2022
-
Re: [Sphenix-software-l] Quick study on storage containers,
Hugo Pereira Da Costa, 04/08/2022
-
Re: [Sphenix-software-l] Quick study on storage containers,
Christof E Roland, 04/08/2022
- Re: [Sphenix-software-l] Quick study on storage containers, Huang, Jin, 04/08/2022
-
Re: [Sphenix-software-l] Quick study on storage containers,
Hugo Pereira Da Costa, 04/08/2022
Archive powered by MHonArc 2.6.24.