Skip to Content.
Sympa Menu

sphenix-software-l - [Sphenix-software-l] access speed to lustre

sphenix-software-l AT lists.bnl.gov

Subject: sPHENIX discussion of software

List archive

Chronological Thread  
  • From: pinkenburg <pinkenburg AT bnl.gov>
  • To: "sphenix-software-l AT lists.bnl.gov" <sphenix-software-l AT lists.bnl.gov>
  • Subject: [Sphenix-software-l] access speed to lustre
  • Date: Fri, 4 Mar 2022 15:50:33 -0500

Hi folks,

looks like the 5 servers didn't make a dent - I see transfer rates of 500kB/s for a file. The size of the file I am looking at is 12GB which means it'll take 4 hours give or take.

In the general meeting I mentioned that the truth info is our biggest file and its content is a complete overkill for physics analysis. If we ever want a chance to analyze the data the TG's need to come up with a solution for their truth info needs - we cannot solve this by increasing the number of servers by feasible amounts.

To illustrate this, analyzing this combination:
DST_TRUTH_pythia8_Jet04_3MHz-0000000003-00000.root
DST_TRACK_pythia8_Jet04_3MHz-0000000003-00000.root
DST_CALO_CLUSTER_pythia8_Jet04_3MHz-0000000003-00000.root

is

12GB for the truth file
 2GB for the tracks (not great either)
180MB for the calorimeter

The vast majority of the truth info is hits which nobody cares about in analysis.

Chris

--
*************************************************************

Christopher H. Pinkenburg ; pinkenburg AT bnl.gov
; http://www.phenix.bnl.gov/~pinkenbu

Brookhaven National Laboratory ; phone: (631) 344-5692
Physics Department Bldg 510 C ; fax: (631) 344-3253
Upton, NY 11973-5000

*************************************************************





Archive powered by MHonArc 2.6.24.

Top of Page