US ATLAS HL-LHC computing discussion

["Malik,Abid" <amalik AT bnl.gov>]

I think most people are not available. I will put my update on the google 
doc. We can meet next week same time if it works for everyone in the group.

Thanks,

Abid M. Malik
Computational Science Initiative (CSI)
Bld-725, 2-131
Phone#6313444657

________________________________________
From: Torre Wenaus <wenaus AT gmail.com>
Sent: Tuesday, August 21, 2018 12:16 PM
To: Malik,Abid
Cc: Rob Gardner; usatlas-hllhc-computing-l AT lists.bnl.gov
Subject: Re: [Usatlas-hllhc-computing-l] First meeting of the distributed 
training working group

You can, but I'm in another meeting that came up since. Sorry about that

On Tue, Aug 21, 2018 at 6:15 PM Malik,Abid 
<amalik AT bnl.gov<mailto:amalik AT bnl.gov>> wrote:
Can we use the same link for discussion?

Thanks

Abid M. Malik
Computational Science Initiative (CSI)
Bld-725, 2-131
Phone#6313444657

________________________________________
From: Torre Wenaus <wenaus AT gmail.com<mailto:wenaus AT gmail.com>>
Sent: Tuesday, August 21, 2018 12:12 PM
To: Malik,Abid
Cc: Rob Gardner; 
usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov>
Subject: Re: [Usatlas-hllhc-computing-l] First meeting of the distributed 
training working group

Oops... I did nothing to set one up... I have 10 meetings today, lost track 
of that one
  Torre

On Tue, Aug 21, 2018 at 6:06 PM Malik,Abid 
<amalik AT bnl.gov<mailto:amalik AT bnl.gov><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov>>>
 wrote:
Hi Torre,

Do we have a meeting today?

Thanks,

Abid M. Malik
Computational Science Initiative (CSI)
Bld-725, 2-131
Phone#6313444657

________________________________________
From: Torre Wenaus 
<wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>
Sent: Tuesday, August 14, 2018 1:23 PM
To: Malik,Abid
Cc: Rob Gardner; 
usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov>>
Subject: Re: [Usatlas-hllhc-computing-l] First meeting of the distributed 
training working group

We had an interesting meeting, see the live notes...
https://docs.google.com/document/d/1w5CXzODmv5Z9HoEpmORTi8-sh6vChkLX4KCajCoFX-s/edit?usp=sharing
...attendees please correct/extend...
Next meeting at the same time in one week. All are welcome to suggest agenda 
topics.
   Abid & Torre

On Tue, Aug 14, 2018 at 10:38 AM Malik,Abid 
<amalik AT bnl.gov<mailto:amalik AT bnl.gov><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov>><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov>>>>
 wrote:
I have changed the protections to editable. Kindly let me know if you still 
have the problem.

Thanks,

Abid M. Malik
Computational Science Initiative (CSI)
Bld-725, 2-131
Phone#6313444657

________________________________________
From: Torre Wenaus 
<wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>>
Sent: Tuesday, August 14, 2018 10:18 AM
To: Malik,Abid
Cc: Rob Gardner; 
usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov>><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov>>>
Subject: Re: [Usatlas-hllhc-computing-l] First meeting of the distributed 
training working group

Thanks Abid. Would you please change the protections to editable by all so 
that all can contribute to the live notes.
I made an agenda
https://indico.cern.ch/event/750428/
that should be accessible to all. The live notes are linked from it. Those 
not experienced with Vidyo please click 'join' a few minutes before the 
meeting and follow the instructions to download the VidyoConnect app. Once 
you have the app installed and click 'join' again you should be connected. Or 
just come to my BNL office 510A 1-220!
  Torre

On Mon, Aug 13, 2018 at 4:48 PM Malik,Abid 
<amalik AT bnl.gov<mailto:amalik AT bnl.gov><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov>><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov>>><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov>><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov><mailto:amalik AT bnl.gov<mailto:amalik AT bnl.gov>>>>>
 wrote:
Dear all,

I have started a google doc. for tomorrow's meeting.


https://docs.google.com/document/d/1w5CXzODmv5Z9HoEpmORTi8-sh6vChkLX4KCajCoFX-s/edit?usp=sharing

Please feel free to add items for discussion.

Regards,
Abid M. Malik
Computational Science Initiative (CSI)
Bld-725, 2-131
Phone#6313444657

________________________________________
From: Usatlas-hllhc-computing-l 
<usatlas-hllhc-computing-l-bounces AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov>><mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov>>><mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov>><mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l-bounces AT lists.bnl.gov>>>>>
 on behalf of Torre Wenaus 
<wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>>>
Sent: Friday, August 10, 2018 9:44 AM
To: Rob Gardner
Cc: 
usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov>><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov>>><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov>><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:usatlas-hllhc-computing-l AT lists.bnl.gov>>>>
Subject: Re: [Usatlas-hllhc-computing-l] First meeting of the distributed 
training working group

Oops, as Doug just noticed, that time conflicts with our new event 
service/HPC ops meeting.
So we will meet Tue Aug 14 at noon eastern. Sorry for the noise.
  Torre


On Fri, Aug 10, 2018 at 3:26 PM Torre Wenaus 
<wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>>>>
 wrote:
We’ll have the first meeting of the distributed training WG on Wed Aug 15 
10am eastern time. Thanks to all who responded to the poll. Agenda to follow, 
suggestions appreciated. We’ll use Vidyo (the ATLAS/CERN standard 
videoconferencing tool) unless that presents a problem for someone.
  Abid & Torre


On Fri, Aug 10, 2018 at 3:21 PM Torre Wenaus 
<wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>>>>
 wrote:
Hi Rob,
Very interesting, thanks! It does sound relevant. For a future meeting you're 
able to attend...
  Torre


On Fri, Aug 10, 2018 at 1:53 PM Robert Gardner 
<rwg AT uchicago.edu<mailto:rwg AT uchicago.edu><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu>><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu>>><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu>><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu>>>><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu>><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu>>><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu>><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu><mailto:rwg AT uchicago.edu<mailto:rwg AT uchicago.edu>>>>>>
 wrote:
Hi Torre

Unfortunately I’m on ‘staycation’ until Aug 22.  I keep missing these 
important meetings!  I did want to mention potentially relevant work we did 
supporting  this year’s CoDaS-HEP training event at Princeton 
(http://codas-hep.org).  We (Ilija, Benedikt, Lincoln) built a portal and 
backend to scale out to CHASE-CI — gpu resources on the Pacific Research 
Platform:  http://codas.slateci.net/.  There were some lessons learned there 
about sign-ups (using institutional identity management) and Kubernetes 
scheduling (we had 60 JupyterLabs running each attached to its own GPU).  We 
plan to create a version of this to support the new ATLAS analytics platform 
(for ML to Elasticsearch for ADC analytics), in time for the Oct S&C week.   
It looks like this might be a little outside the scope here but perhaps parts 
could be re-purposed for potential user frontends.

Cheers,
- Rob



On Aug 7, 2018, at 9:42 AM, Torre Wenaus 
<wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com><mailto:wenaus AT gmail.com<mailto:wenaus AT gmail.com>>>>>>
 wrote:

Hi,
We are planning a first meeting of the distributed training working group, 
one of the working groups defined at last month’s US ATLAS / CSI workshop at 
BNL. If you’re interested in attending please fill in the doodle:
https://doodle.com/poll/iayykxwd94isqdf4
The distributed training WG is to examine the scaling out of ML training 
across distributed/parallel resources in order to minimize the turnaround 
time on network tuning and ML studies. The technical approaches to be looked 
at include those discussed in the workshop; cf. the talks of Abid (Horovod), 
Alexei (PanDA), and Amir.  It was agreed at the workshop to define concrete 
objectives for the WG by the end of September, so that will be the main topic 
of this meeting.
  Abid & Torre

_______________________________________________
Usatlas-hllhc-computing-l mailing list
Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov>><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov>>><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov>><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov>>>><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov>><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov>>><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov>><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov><mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov<mailto:Usatlas-hllhc-computing-l AT lists.bnl.gov>>>>>
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-computing-l