sphenix-software-l AT lists.bnl.gov
Subject: sPHENIX discussion of software
List archive
[[Sphenix-software-l] ] copying files to the local disk
- From: pinkenburg <pinkenburg AT bnl.gov>
- To: "sphenix-software-l AT lists.bnl.gov" <sphenix-software-l AT lists.bnl.gov>
- Subject: [[Sphenix-software-l] ] copying files to the local disk
- Date: Thu, 3 Oct 2024 16:49:01 -0400
Hi folks,
when promoting the copying of files to the local disk in condor jobs I forgot to give a recipe how to actually do this painlessly (means not figuring out where a file is actually located so you can copy it). I installed a script from our simulation production (getinputfiles.pl) which can do this for you for files which are in our file catalog (all production and simulation output). It's argument is either a filename (like HIST_DST_TRKR_HIT_run2pp_new_2024p007-00053018-00016.root) or you can give it a list of files like
getinputfiles.pl --filelist mylist.txt
It checks the size of the file and the md5sum to ensure data integrity. The script is a bit on the complicated side since it has evolved over time to be able to use the various copy mechanisms we had, by default it uses cp which is the best choice. I just don't want to throw out the others in case we ever need them.
I updated our condor wiki to show how to add this to the condor script:
https://wiki.sphenix.bnl.gov/index.php?title=Condor#Minimizing_Disk_Ops
But you can use this script to copy files interactively (not sure why you would need it but sometimes you want a file locally)
Chris
--
*************************************************************
Christopher H. Pinkenburg ; pinkenburg AT bnl.gov
; http://www.phenix.bnl.gov/~pinkenbu
Brookhaven National Laboratory ; phone: (631) 344-5692
Physics Department Bldg 510 C ; fax: (631) 344-3253
Upton, NY 11973-5000
*************************************************************
- [[Sphenix-software-l] ] copying files to the local disk, pinkenburg, 10/03/2024
Archive powered by MHonArc 2.6.24.