sphenix-software-l AT lists.bnl.gov
Subject: sPHENIX discussion of software
List archive
- From: pinkenburg <pinkenburg AT bnl.gov>
- To: "sphenix-software-l AT lists.bnl.gov" <sphenix-software-l AT lists.bnl.gov>
- Subject: [Sphenix-software-l] access speed to lustre
- Date: Fri, 4 Mar 2022 15:50:33 -0500
Hi folks,
looks like the 5 servers didn't make a dent - I see transfer rates of 500kB/s for a file. The size of the file I am looking at is 12GB which means it'll take 4 hours give or take.
In the general meeting I mentioned that the truth info is our biggest file and its content is a complete overkill for physics analysis. If we ever want a chance to analyze the data the TG's need to come up with a solution for their truth info needs - we cannot solve this by increasing the number of servers by feasible amounts.
To illustrate this, analyzing this combination:
DST_TRUTH_pythia8_Jet04_3MHz-0000000003-00000.root
DST_TRACK_pythia8_Jet04_3MHz-0000000003-00000.root
DST_CALO_CLUSTER_pythia8_Jet04_3MHz-0000000003-00000.root
is
12GB for the truth file
2GB for the tracks (not great either)
180MB for the calorimeter
The vast majority of the truth info is hits which nobody cares about in analysis.
Chris
--
*************************************************************
Christopher H. Pinkenburg ; pinkenburg AT bnl.gov
; http://www.phenix.bnl.gov/~pinkenbu
Brookhaven National Laboratory ; phone: (631) 344-5692
Physics Department Bldg 510 C ; fax: (631) 344-3253
Upton, NY 11973-5000
*************************************************************
-
[Sphenix-software-l] access speed to lustre,
pinkenburg, 03/04/2022
-
Re: [Sphenix-software-l] [EXTERNAL] access speed to lustre,
Osborn, Joe, 03/04/2022
- Re: [Sphenix-software-l] [EXTERNAL] access speed to lustre, pinkenburg, 03/04/2022
-
Re: [Sphenix-software-l] [EXTERNAL] access speed to lustre,
Osborn, Joe, 03/04/2022
Archive powered by MHonArc 2.6.24.