Atlas-connect-l mailing list

["Dr. Harinder Singh Bawa" <harinder.singh.bawa AT gmail.com>]

Hi Ilija

Thanks! I understood and tested too that whatever be the server name ahead of the gLFN , if my job want to access,  server will check if it has file, find that it does not, and redirect the client to glrd server which will redirect it to another one & this is done behind the curtains :)

Harinder

On Thu, Apr 24, 2014 at 10:47 AM, Ilija Vukotic <Ilija.Vukotic AT cern.ch> wrote:

Hi,

Seems my explanations were too brief...

Your dataset exists at only one place: mwt2.
It can be accessed using any of the fax endpoints (servers) in the world.

If your job uses slac server, slac server will check if it has file, find that it does not, and redirect the client to glrd server which will redirect it to another one and that one to mwt2. Than finally your job will get the data directly from the server that has the file. All this redirection is fast and happens behind the curtain.

So your jobs should always use the server recommended by localSetupFAX. In most cases that provides optimal bandwidth.

If you really want to force your jobs to use a specific fax endpoint you may send the full input file list with the job.

Ilija

On Apr 24, 2014 12:36 PM, "Dr. Harinder Singh Bawa" <harinder.singh.bawa AT gmail.com> wrote:

Excuse me for not explaining well. Yes, gLFN is same

 //atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000113.MWT2.root.1

But server name is different as from where these LFN are accessed.

ex: If I run on MWT2: I have to use the path : root://atl-prod09.slac.stanford.edu:1094//atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000113.MWT2.root.1

and if I run on Fresno: I have to use path to inputfile as :  root://fax.mwt2.org:1094//atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000113.MWT2.root.1

I dont see my jobs access that particular file without giving server name in front of LFN. (I hope I could explain my problem)

Harinder

On Thu, Apr 24, 2014 at 10:28 AM, Ilija Vukotic <Ilija.Vukotic AT cern.ch> wrote:

Hi Harinder,

gLFNs are the same in both cases: //atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000113.MWT2.root.1

There is just one file like that in the whole world.

what is different is FAX access point: fax.mwt2.org or atl-prod09.slac.stanford.edu

whichever your jobs choose you will still get the same file - since it’s unique. 

Cheers,
Ilija

On Apr 24, 2014, at 12:19 , Dr. Harinder Singh Bawa <harinder.singh.bawa AT gmail.com> wrote:

Hi Ilija ,

Yes, its correct that  localSetupFAX calculated that the closest FAX endpoint in one case is WT2 (SLAC), and in other case MWT2.

This is the thing I am facing. If I submit my condor job in "atlasconnect" where there are few nodes on MWT2 and few nodes on Fresno T3 (For both LFN are different). I need to give path to my inputfiles before sending condor jobs but I dont know where my job will run (MWT2 or Fresno).

I can restrict in condor to run on either Fresno or on MWT2 putting condition in requirement file and give LFN of that particular site.

Like I can use inputfile as

[bawa@t3nfs SkimSlimT3Large]$  fax-get-gLFNs user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00

root://atl-prod09.slac.stanford.edu:1094//atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000113.MWT2.root.1

while putting restriction on jobs to run on Fresno T3. This inputfile cannot be run on MWT2 since on that site , LFN is different.

Exploring how I can use both sites together.

Harinder

On Thu, Apr 24, 2014 at 10:12 AM, Ilija Vukotic <Ilija.Vukotic AT cern.ch> wrote:

Hi,

reason why the LFN’s are different is because localSetupFAX calculated that the closest FAX endpoint in one case is WT2 (SLAC), and in other case MWT2.

That’s exactly how it should be. 

Your specific dataset exist only at MWT2 and nowhere else so independently what FAX endpoint is used to access them, you still get the same files.

Concerning “restrictions”: can you clarify what are you trying to do?

Cheers,

Ilija

--

Ilija Vukotic • Skype ivukotic • +1 872 230 6435 • University of Chicago

On Apr 24, 2014, at 12:03 , Dr. Harinder Singh Bawa <harinder.singh.bawa AT gmail.com> wrote:

Hi all,

I am trying to use FAX in my condor jobs to access input data to my root code. The problem I am see-ing at the moment is 

**********

If I see LFN for particular dataset in myFresno T-3 (Its from SLAC server):

[bawa@t3nfs SkimSlimT3Large]$  fax-get-gLFNs user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00

root://atl-prod09.slac.stanford.edu:1094//atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000113.MWT2.root.1

root://atl-prod09.slac.stanford.edu:1094//atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000102.MWT2.root.2

xxxxxxxxxxxxxxxxxxxxxxxxx

But If I see it from atlasconnect (Its from MWT2 server):

[hbawa@login SkimSlimT3Large]$ fax-get-gLFNs user.flegger.MWT2.data12_8TeV.00212172.physics_Muons.merge.NTUP_SMWZ.f479_m1228_p1067_p1141_tid01007411_00

root://fax.mwt2.org:1094//atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000113.MWT2.root.1

root://fax.mwt2.org:1094//atlas/rucio/user/flegger:NTUP_SMWZ.01007411._000102.MWT2.root.2

&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&

I can run on particular site after cross-checking path and make my input file accordingly.

But obviously I can run without any restriction on site since every site have different LFN.

I am trying to find a way to submit without any restrictions. Any suggestion is appreciated.

Thanks

Harinder

--

Dr. Harinder Singh Bawa

                                          
[web][facebook][youtube][twitter]

--

Dr. Harinder Singh Bawa

                                          
[web][facebook][youtube][twitter]

--

Dr. Harinder Singh Bawa

                                          
[web][facebook][youtube][twitter]