Skip to Content.
Sympa Menu

usatlas-hllhc-computing-l - Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group

usatlas-hllhc-computing-l AT lists.bnl.gov

Subject: US ATLAS HL-LHC computing discussion

List archive

Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group

Chronological Thread  
    < Chronological >       < Thread >
  • From: "Dong, Zhihua" <zdong AT bnl.gov>
  • To: Heather Gray <heather.gray AT cern.ch>, A A Hasib <a.hasib AT cern.ch>
  • Cc: "usatlas-hllhc-computing-l AT lists.bnl.gov" <usatlas-hllhc-computing-l AT lists.bnl.gov>
  • Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group
  • Date: Wed, 12 Sep 2018 13:36:24 +0000

Thanks for the files.


I think we are still missing some files.

...

Error in <TFile::TFile>: file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_020_025/wiggle_i
nput_deriv_Sampling_1.ver03.root does not exist
Wiggle file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_020_025/wiggle_input_deriv_Samplin
g_1.ver03.root not found
Error in <TFile::TFile>: file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_020_025/wiggle_i
nput_deriv_Sampling_2.ver03.root does not exist
Wiggle file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_020_025/wiggle_input_deriv_Samplin
g_2.ver03.root not found
Error in <TFile::TFile>: file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_020_025/wiggle_i
nput_deriv_Sampling_3.ver03.root does not exist
Wiggle file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_020_025/wiggle_input_deriv_Samplin
g_3.ver03.root not found
Error in <TFile::TFile>: file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_200_205/wiggle_i
nput_deriv_Sampling_5.ver03.root does not exist
Wiggle file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_200_205/wiggle_input_deriv_Sampling_5.ver03.root not found
Error in <TFile::TFile>: file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_200_205/wiggle_input_deriv_Sampling_6.ver03.root does not exist
Wiggle file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_200_205/wiggle_input_deriv_Sampling_6.ver03.root not found
Error in <TFile::TFile>: file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_280_285/wiggle_input_deriv_Sampling_6.ver03.root does not exist
Wiggle file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_280_285/wiggle_input_deriv_Sampling_6.ver03.root not found
Error in <TFile::TFile>: file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_200_205/wiggle_input_deriv_Sampling_7.ver03.root does not exist
Wiggle file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_200_205/wiggle_input_deriv_Sampling_7.ver03.root not found
Error in <TFile::TFile>: file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_280_285/wiggle_input_deriv_Sampling_7.ver03.root does not exist
Wiggle file /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_280_285/wiggle_input_deriv_Sampling_7.ver03.root not found

...



Also our standard installation of ROOT maybe missing something ? any advise?

Error in <TROOT::LoadMacro>: macro ../../ISF_FastCaloSimEvent/src/TFCSHistoLateralShapeWeight.cxx not found in path .:/hpcgpfs01/software/root/v6.14.04/macros


Without those file , code stuck in : Line 814 forever

analyze->LoopEvents(-1);

 

creating big "OutputAvgShape.root" file  with output  printing lines like:
....
Line: 430000 line  id 8ddb4b0 hash_id 148027568
Line: 440000 line  id 8ddb4b0 hash_id 148027568
Line: 450000 line  id 8ddb4b0 hash_id 148027568
Line: 460000 line  id 8ddb4b0 hash_id 148027568
Line: 470000 line  id 8ddb4b0 hash_id 148027568

...



Zhihua Dong
Computational Science Initiative
Brookhaven National Laboratory
Phone: 631-344-4141

From: Heather Gray <heather.gray AT cern.ch>
Sent: Tuesday, September 11, 2018 2:58:36 PM
To: A A Hasib
Cc: Dong, Zhihua; usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group
 
Hi All,

Thanks to Alex, the files are now available here at BNL:

The four files are in /afs/usatlas.bnl.gov/software/downloads//gitrel/samples/

 

-rw-r--r-- 1 undrus usatlas 2731676318 Sep 11 13:45 NTUP_FCS.14707881._000001.pool.root.1

-rw-r--r-- 1 undrus usatlas     430022 Sep 11 13:51 mc16_13TeV.431004.ParticleGun_pid22_E65536_disj_eta_m25_m20_20_25_zv_0.firstPCA_App.ver03.root

-rw-r--r-- 1 undrus usatlas     166680 Sep 11 13:52 mc16_13TeV.431004.ParticleGun_pid22_E65536_disj_eta_m25_m20_20_25_zv_0.secondPCA.ver03.root

-rw-r--r-- 1 undrus usatlas    1027441 Sep 11 13:53 mc16_13TeV.431004.ParticleGun_pid22_E65536_disj_eta_m25_m20_20_25_zv_0.shapepara.ver03.root


Cheers

Heather


On Tue, 11 Sep 2018 at 15:49, Hasib Ahmed <a.hasib AT cern.ch> wrote:
Hi Zhihua,

I don't have access to BNL and they can't be sent via email unfortunately. Perhaps someone in this list with access to both eos and BNL disk space would volunteer to copy these files for you ?

Thanks,
Hasib
------------------------------------
Hasib Ahmed
Research Associate
University of Edinburgh
ATLAS Experiment
-------------------------------------


On Tue, Sep 11, 2018 at 2:37 PM Dong, Zhihua <zdong AT bnl.gov> wrote:

Dear Hasib


Thank you for the info. 


I don't have access to /eos


Can you send me those required files ?  or copy it somewhere in BNL.

Are those files big ?



Thanks.


Dong


Zhihua Dong
Computational Science Initiative
Brookhaven National Laboratory
Phone: 631-344-4141

From: Hasib Ahmed <a.hasib AT cern.ch>
Sent: Tuesday, September 11, 2018 7:28:25 AM
To: Dong, Zhihua
Cc: usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group
 
Dear Zhihua,

Apologies for the delay. Just to be clear, for the actual  simulation we only need the big parametrization file located: 

However, this particular macro `runTFCSShapeValidation.cxx` makes a comparison of the simulated showers to the original G4 showers, hence it requires the input G4 sample. 

by the following samples:

* inputSample = /eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesSummer18Complete/mc16_13TeV.431004.ParticleGun_pid22_E65536_disj_eta_m25_m20_20_25_zv_0.deriv.NTUP_FCS.e6556_e5984_s3259_r10283_p3589/NTUP_FCS.14707881._000001.pool.root.1


The following files really required. We would need to clean up that code at a later time:
* pcaSample = /eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer03/mc16_13TeV.431004.ParticleGun_pid22_E65536_disj_eta_m25_m20_20_25_zv_0.firstPCA_App.ver03.root

* shapefile = eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer03/mc16_13TeV.431004.ParticleGun_pid22_E65536_disj_eta_m25_m20_20_25_zv_0.shapepara.ver03.root  (the file is not really needed but the name is used in the constructor of a class) 

* energyfile = eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer03/mc16_13TeV.431004.ParticleGun_pid22_E65536_disj_eta_m25_m20_20_25_zv_0.secondPCA.ver03.root  (not required)

* avgSample = Not required even though it would complain it cannot find the file. 


In terms of the run time, it should only take few minutes. But since the code creates lots of TCanvas, you should run it batch mode.

> root -b
> .x initTFCSAnalyzer.C
> .x runTFCSShapeValidation.cxx


Hope all these are clear. Let me know if you have further question. 

Best,
Hasib 
------------------------------------
Hasib Ahmed
Research Associate
University of Edinburgh
ATLAS Experiment
-------------------------------------


On Mon, Sep 10, 2018 at 9:19 PM Dong, Zhihua <zdong AT bnl.gov> wrote:

Dear Hasib


Could you send us the needed files on /eos/... 

The code without those file hangs.


Dong


Zhihua Dong
Computational Science Initiative
Brookhaven National Laboratory
Phone: 631-344-4141

From: Usatlas-hllhc-computing-l <usatlas-hllhc-computing-l-bounces AT lists.bnl.gov> on behalf of Dong, Zhihua <zdong AT bnl.gov>
Sent: Friday, September 7, 2018 1:29:06 PM
To: Hasib Ahmed
Cc: usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group
 


PLease provide those files someway.  


Also how long do we expect this run will take ?



Zhihua Dong
Computational Science Initiative
Brookhaven National Laboratory
Phone: 631-344-4141

From: Hasib Ahmed <a.hasib AT cern.ch>
Sent: Friday, September 7, 2018 11:42:18 AM
To: Dong, Zhihua
Cc: Heather Gray; usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group
 
Hi Zhihua, 

Sorry my bad, the code also run the original G4 samples and some additional files for comparison which are read bia FCS_dsid class:

We can provide you those file for this particle pdgid, energy and eta point that you can use locally. Without these files, the program should still run (I would think) but the plots it produces won't be what we expect. 

Thanks,
Hasib
------------------------------------
Hasib Ahmed
Research Associate
University of Edinburgh
ATLAS Experiment
-------------------------------------


On Fri, Sep 7, 2018 at 4:32 PM Dong, Zhihua <zdong AT bnl.gov> wrote:


I did get some permission denied message  related to /eos  by just run


Although it does not stop the program yet, no message after that for a while...



...

...

initialising FCS_dsid...
FCS_dsid ready
(int) 2
* Running on linux system
sh: /FCS_ls.696.list: Permission denied
Temporary file list for selection:/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesProdsysProduction/mc16_13TeV.431004.ParticleGun_pid22_E65536_disj_eta_m25_m20_20_25_zv_0.deriv.NTUP_FCS.*/NTUP_FCS.*.pool.root.* : /FCS_ls.696.list




Zhihua Dong
Computational Science Initiative
Brookhaven National Laboratory
Phone: 631-344-4141

From: Hasib Ahmed <a.hasib AT cern.ch>
Sent: Friday, September 7, 2018 11:15:56 AM
To: Dong, Zhihua
Cc: Heather Gray; usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group
 
Dear Zhihua, 

The relevant steering macro for your use case is the following:

which does not require any other external file apart from the parametrization file that was mentioned earlier. 


Cheers,
Hasib 
------------------------------------
Hasib Ahmed
Research Associate
University of Edinburgh
ATLAS Experiment
-------------------------------------


On Fri, Sep 7, 2018 at 4:05 PM Dong, Zhihua <zdong AT bnl.gov> wrote:


It seems code also depend on "/eos/atlas/...."

Does that directory need to be setup someway ?


[zdong@icsubmit01 macro]$ grep eos *.cxx *.C |grep -v "//"
MakeTFCSEventAnalyzer.cxx:    topDir   = "/eos/atlas/atlascerngroupdisk/proj-simul/";
MakeTFCSEventAnalyzer.cxx:    plotDir = "/eos/project/a/atlas-fastcalosim/www/";
MakeTFCSNtupleMaker.cxx:    topDir   = "/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesLocalProd2017/rel_21_0_42/Samples/";
runTFCSCreateParametrization.cxx:  TString EinterfileName = Form("%s/mc16_13TeV.pid%i.Einterpol.ver01.root", "/eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/", pdgid);
runTFCSCreateParametrization.cxx:  TFCSParametrizationBase* para_photon_simple=NewParametrizationSimple("/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesLocalProd2017/rel_21_0_62/energyPara/epara_photon_E65_eta02_21061.root","/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesLocalProd2017/rel_21_0_62/shapePara/mc16_13TeV.photon.E65536.eta020_025.merged_default_z0.shapepara.root",22,65536,0.2,0.25);
runTFCSCreateParametrization.cxx:  TFCSParametrizationBase* para_pion_simple=NewParametrizationSimple("/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesLocalProd2017/rel_21_0_62/energyPara/epara_pion_E65_eta02_21061.root","/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesLocalProd2017/rel_21_0_62/shapePara/mc16_13TeV.pion.E65536.eta020_025.merged_default_z0.shapepara.root",211,65536,0.2,0.25);
runTFCSShapeValidation.cxx:    TFile* wigglefile = TFile::Open("/eos/atlas/user/a/ahasib/public/Simul-FastCalo/ParametrizationProductionVer02/Wiggle/eta_020_025/wiggle_input_deriv_Sampling_2.ver02.root");
test_TFCS2DFunctionHistogram.cxx:  TFile* file=TFile::Open("/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesLocalProd2017/rel_21_0_62/shapePara/mc16_13TeV.pion.E65536.eta020_025.merged_default_z0.shapepara.root");
runTFCSEnergyInterpolationTGraph.C:void runTFCSEnergyInterpolationTGraph(int pid = 22, float etamin = 0., float etamax = 5., bool useFit = false, bool doSpline = true, std::string inputDir = "/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesSummer18Complete/", std::string plotDir = "plot_Einterpol", std::string ver = "ver03")
testMaxrz.C: std::string file = "/eos/atlas/atlascerngroupdisk/proj-simul/InputSamplesProdsysProduction/mc16_13TeV.431202.ParticleGun_pid22_E262144_disj_eta_m15_m10_10_15_zv_0.deriv.NTUP_FCS.e6556_e5984_s3259_r10283_p3449/NTUP_FCS.13289425._000001.pool.root.1";
[zdong@icsubmit01 macro]$


Zhihua Dong
Computational Science Initiative
Brookhaven National Laboratory
Phone: 631-344-4141

From: Hasib Ahmed <a.hasib AT cern.ch>
Sent: Friday, September 7, 2018 4:39:10 AM
To: Heather Gray
Cc: Dong, Zhihua; usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group
 
Hello, 
I just started received email from the mailing list. Yes, what Heather said is indeed correct. This is the parametrization file used for fast simulation which is essential. Just using a copy from your local disk should do the trick. 

Thanks,
Hasib 
------------------------------------
Hasib Ahmed
Research Associate
University of Edinburgh
ATLAS Experiment
-------------------------------------


On Thu, Sep 6, 2018 at 11:42 PM Heather Gray <heather.gray AT cern.ch> wrote:
Hi Zhihua,

Ah, I know what this is. This is the fill that contains the parametrisation used to describe the calorimeter response -- i.e. it's the meat for FCS. I think we'll need to copy it somewhere local at BNL and then edit the code so that it points at it instead of the afs location.

Cheers
Heather

On Thu, 6 Sep 2018 at 12:21, Dong, Zhihua <zdong AT bnl.gov> wrote:

Even with Xin's help setup relocated cvms . 

Still get the same error.


Maybe somewhere the code still depends on non-exiting /afs on IC

/afs/cern.ch/atlas/groups/Simulation/FastCaloSimV2/TFCSparam_v005.root 



Zhihua Dong
Computational Science Initiative
Brookhaven National Laboratory
Phone: 631-344-4141

From: Lancon, Eric
Sent: Thursday, September 6, 2018 1:17:01 PM
To: Dong, Zhihua; Lin, Meifeng; Snyder,Scott S
Cc: usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group
 

OK good!

Xin did something to overcome the cvmfs issue I believe, when he tested KNLs

Eric

 

From: "Dong, Zhihua" <zdong AT bnl.gov>
Date: Thursday, September 6, 2018 at 12:51 PM
To: "Lancon, Eric" <elancon AT bnl.gov>, "Lin, Meifeng" <mlin AT bnl.gov>, "Snyder,Scott S" <snyder AT bnl.gov>
Cc: "usatlas-hllhc-computing-l AT lists.bnl.gov" <usatlas-hllhc-computing-l AT lists.bnl.gov>
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group

 

Dear Eric

 

I don't have CERN account.  Costin helped me downloaded it.

 

Also  we installed ROOT on IC.

 

But the code seems depends on /afs or /cvmfs  which does not exist on IC.

 

Maybe it's just setup issue . Current setup.sh :

 

[zdong@icsubmit01 FastCaloSimAnalyzer]$ pwd

[zdong@icsubmit01 FastCaloSimAnalyzer]$ cat setup.sh

if [ -z $ATLAS_LOCAL_ASETUP_VERSION ];

then

   # do it only if it has not been done yet

   export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase

   alias setupATLAS='source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh'

   source $ATLAS_LOCAL_ROOT_BASE/user/atlasLocalSetup.sh

else

   echo "Atlas environment already configured."

fi

lsetup root

lsetup git

[zdong@icsubmit01 FastCaloSimAnalyzer]$

 

 

Ignore it , get errors  in   ".x  runTFCSShapeValidation.cxx"

...

...

hit_to_cell_mapping_23                        INFO   PDGID: all ; Ekin=all ; eta=all

hit_to_cell_mapping_23                        INFO   Ekin_bin=all ; calosample=23

hit_to_cell_mapping_23                        INFO   geo=0

Error in <TFile::TFile>: file /afs/cern.ch/atlas/groups/Simulation/FastCaloSimV2/TFCSparam_v005.root does not exist

...

 

 

 

Zhihua Dong

Computational Science Initiative

Brookhaven National Laboratory

Phone: 631-344-4141

 

From: Lancon, Eric
Sent: Thursday, September 6, 2018 12:27:49 PM
To: Dong, Zhihua; Lin, Meifeng; Snyder,Scott S
Cc: usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group

 

Hello Zhihua,

 

I tested with my google account, access does not work,

Do you have a CERN account?

 

Yes it needs ROOT

his package is standalone and thus does NOT require athena BUT only requires root 6.08 or later versions.

 

Eric

 

From: Usatlas-hllhc-computing-l <usatlas-hllhc-computing-l-bounces AT lists.bnl.gov> on behalf of "Dong, Zhihua" <zdong AT bnl.gov>
Date: Thursday, September 6, 2018 at 11:46 AM
To: "Lin, Meifeng" <mlin AT bnl.gov>, "Snyder,Scott S" <snyder AT bnl.gov>
Cc: "usatlas-hllhc-computing-l AT lists.bnl.gov" <usatlas-hllhc-computing-l AT lists.bnl.gov>
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group

 

This might be a stupid question .

 Do I need CERN account to download the 

https://gitlab.cern.ch/atlas-simulation-fastcalosim/FastCaloSimAnalyzer

 

Trying login with Google or other public account does not work for me.

 

Also does it require ROOT to run the code ?

 

 

Best

 

 

 

Zhihua Dong

Computational Science Initiative

Brookhaven National Laboratory

Phone: 631-344-4141

 

From: Usatlas-hllhc-computing-l <usatlas-hllhc-computing-l-bounces AT lists.bnl.gov> on behalf of scott snyder <scott.snyder AT cern.ch>
Sent: Wednesday, September 5, 2018 2:40:35 PM
To: Lin, Meifeng
Cc: usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] Fast Simulation Working Group

 

hi Meifeng -

>Which server at BNL did you put the files on? Most of us at CSI only have access to the BNL
>Institutional Cluster nodes (icsubmit or knlsubmit), and I couldn't seem to find the directories you
>referred to in your doc.

I copied the tree /afs/usatlas.bnl.gov/software/downloads/DB/21.0
to ~snyder/21.0.tar.bz2  --- i think you should be able to read that.
Let me know when you've copied it so i can remove it from there.
thanks,
sss

_______________________________________________
Usatlas-hllhc-computing-l mailing list
Usatlas-hllhc-computing-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-computing-l

_______________________________________________
Usatlas-hllhc-computing-l mailing list
Usatlas-hllhc-computing-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-computing-l
_______________________________________________
Usatlas-hllhc-computing-l mailing list
Usatlas-hllhc-computing-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-computing-l
_______________________________________________
Usatlas-hllhc-computing-l mailing list
Usatlas-hllhc-computing-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-computing-l

Archive powered by MHonArc 2.6.24.

Top of Page