Atlas-connect-l mailing list

["Dr. Harinder Singh Bawa" <harinder.singh.bawa AT gmail.com>]

Hello  Vukotic,

Thanks for your message and tutorial. Before I apply your tutorial into my work, I wanted to test the same as it is. I copied "SkimSlimT3Large" and changed the inputfiles path according to what I see in my FAX. Below are the steps I did:

[bawa@t3nfs Tutorial]$ dq2-ls -r user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00: 

INCOMPLETE:

COMPLETE: MWT2_DATADISK

[bawa@t3nfs Tutorial]$ dq2-list-files -p user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00

root://tier2-03.ochep.ou.edu:1094//atlas/dq2/user/flegger/MWT2/user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00/NTUP_SMWZ.01007411._000113.MWT2.root.1

I checked by copying one such file(working)

=============================

[bawa@t3nfs Tutorial]$ xrdcp root://tier2-03.ochep.ou.edu:1094//atlas/dq2/user/flegger/MWT2/user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00/NTUP_SMWZ.01007411._000113.MWT2.root.1 .

Now, when i submit using jobOption :

==================================

Jobs = 10

getenv         = False

executable     = SkimSlimLarge.sh

output         = output/SkimSlimLarge.out.$(Process)

error          = error/SkimSlimLarge.error.$(Process)

log            = log/SkimSlimLarge.log.$(Process)

arguments = $(Process) $(Jobs)

transfer_input_files = filter-and-merge-d3pd.py,/tmp/x509up_u1065,inputFileListLarge,branchesList,cutCode

universe       = vanilla

#Requirements   = (regexp("^uc3.*", TARGET.Machine,"IM") == True)

#Requirements   = HAS_CVMFS =?= True

requirements              = (Arch == "INTEL"||Arch == "X86_64")&& \

                            ((Machine == "pt3wrk0.atlas.csufresno.edu")|| \

                             (Machine == "pt3wrk1.atlas.csufresno.edu")|| \

                             (Machine == "pt3wrk2.atlas.csufresno.edu")|| \

                             (Machine == "pt3wrk3.atlas.csufresno.edu")|| \

                             (Machine == "pt3wrk4.atlas.csufresno.edu"))

WhenToTransferOutput = ON_EXIT

queue $(Jobs)

Now, When I submit the job, I am getting error messages as

*******************************************************************88

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<bool>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<char>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<short>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<long>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<unsigned-char>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<unsigned-short>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<unsigned-int>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<unsigned-long>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<float>=vector.dll vectorbool.dll> for level 0; ignored

Warning in <TEnvRec::ChangeValue>: duplicate entry <Library.vector<double>=vector.dll vectorbool.dll> for level 0; ignored

XrdSec: No authentication protocols are available.

140109 16:12:31 001 Xrd: GoToAnotherServer: Error handshaking to [atl-prod10.slac.stanford.edu:2094]

140109 16:12:31 001 Xrd: Open: Error in handling a redirection.

140109 16:12:31 001 Xrd: CheckErrorStatus: Error while being redirected for request Open

140109 16:12:31 001 Xrd: Open: Error opening the file /atlas/dq2/user/flegger/MWT2/user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00/NTUP_SMWZ.01007411._000113.MWT2.root.1 on host atl-prod09.slac.stanford.edu:1094

Error in <TXNetFile::CreateXClient>: open attempt failed on root://atl-prod09.slac.stanford.edu//atlas/dq2/user/flegger/MWT2/user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00/NTUP_SMWZ.01007411._000113.MWT2.root.1

Traceback (most recent call last):

  File "filter-and-merge-d3pd.py", line 965, in <module>

    sys.exit(main())

  File "filter-and-merge-d3pd.py", line 673, in main

    raise RuntimeError("no such file [%s]" % fname)

RuntimeError: no such file [root://atl-prod09.slac.stanford.edu:1094//atlas/dq2/user/flegger/MWT2/user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00/NTUP_SMWZ.01007411._000113.MWT2.root.1

******************************88

It seems, I am not able to access the input file..AM I correct? If yes, then I am wondering where do I set 

localSetupFAX on each node. Does condor will do for me ? What does the above error indicates?

Harinder

On Thu, Jan 9, 2014 at 12:08 PM, Ilija Vukotic <ivukotic AT uchicago.edu> wrote:

Hi,

Yes the standard way to do this is to send a proxy together with the job to a worker node.

If you look at the tutorial (you can set it up using command: tutorial FAX)

you will see a relevant example will be in the directory SkimSlimT3Large. 

There job submission script has this line:

transfer_input_files = filter-and-merge-d3pd.py,/tmp/x509up_u21183,inputFileListLarge,branchesList,cutCode

it is important that script executed has line equivalent to this:

export X509_USER_PROXY=x509up_u21183

so that the job knows where to look for the proxy.

Best regards,
Ilija

On Jan 9, 2014, at 12:47 , Dr. Harinder Singh Bawa <harinder.singh.bawa AT gmail.com> wrote:

Hello all,

Happy New year.

I have a question if somebody can answer would be appreciated. Currently I have a condor script which I run on Fresno-T3 which setup the environment on each node(ROOT) and compiles and run the job and send me output back to my area. The point is that my input and output data path is pointing to local dir.

The same I am trying to run on ATLASConnect. I thought I can use FAX system to transfer input file from the grid to the node where job is running BUT I see there would be a problem that to do this, I need voms-proxy to be setup on each node which requires passphrase. 

Or I can do is to include my proxy in the path and send it with my job. $X509_USER_PROXY

Do you have any better/simpler  idea than this?

Thanks

--

Dr. Harinder Singh Bawa

                                          
[web][facebook][youtube][twitter]

_______________________________________________
Atlas-connect-l mailing list
Atlas-connect-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/atlas-connect-l

--

Dr. Harinder Singh Bawa

                                          
[web][facebook][youtube][twitter]