usatlas-hllhc-computing-l AT lists.bnl.gov

Subject: US ATLAS HL-LHC computing discussion

List archive

Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group

From: Robert Gardner <rwg AT uchicago.edu>
To: Torre Wenaus <wenaus AT gmail.com>
Cc: "usatlas-hllhc-computing-l AT lists.bnl.gov" <usatlas-hllhc-computing-l AT lists.bnl.gov>
Subject: Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group
Date: Fri, 10 Aug 2018 11:53:28 +0000

Hi Torre

Unfortunately I’m on ‘staycation’ until Aug 22. I keep missing these important meetings! I did want to mention potentially relevant work we did supporting this year’s CoDaS-HEP training event at Princeton (http://codas-hep.org). We (Ilija, Benedikt, Lincoln) built a portal and backend to scale out to CHASE-CI — gpu resources on the Pacific Research Platform: http://codas.slateci.net/. There were some lessons learned there about sign-ups (using institutional identity management) and Kubernetes scheduling (we had 60 JupyterLabs running each attached to its own GPU). We plan to create a version of this to support the new ATLAS analytics platform (for ML to Elasticsearch for ADC analytics), in time for the Oct S&C week. It looks like this might be a little outside the scope here but perhaps parts could be re-purposed for potential user frontends.

Cheers,

- Rob

On Aug 7, 2018, at 9:42 AM, Torre Wenaus <wenaus AT gmail.com> wrote:

Hi,

We are planning a first meeting of the distributed training working group, one of the working groups defined at last month’s US ATLAS / CSI workshop at BNL. If you’re interested in attending please fill in the doodle:

https://doodle.com/poll/iayykxwd94isqdf4

The distributed training WG is to examine the scaling out of ML training across distributed/parallel resources in order to minimize the turnaround time on network tuning and ML studies. The technical approaches to be looked at include those discussed in the workshop; cf. the talks of Abid (Horovod), Alexei (PanDA), and Amir. It was agreed at the workshop to define concrete objectives for the WG by the end of September, so that will be the main topic of this meeting.

Abid & Torre

_______________________________________________
Usatlas-hllhc-computing-l mailing list
Usatlas-hllhc-computing-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-computing-l

[Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Torre Wenaus, 08/07/2018
- Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Robert Gardner, 08/10/2018
  - Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Torre Wenaus, 08/10/2018
    - Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Torre Wenaus, 08/10/2018
      - Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Torre Wenaus, 08/10/2018
        
        Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Malik,Abid, 08/13/2018
        
        Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Torre Wenaus, 08/14/2018
        
        Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Malik,Abid, 08/14/2018
        Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Torre Wenaus, 08/14/2018
        Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Malik,Abid, 08/21/2018
        Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Torre Wenaus, 08/21/2018
        Re: [Usatlas-hllhc-computing-l] First meeting of the distributed training working group, Malik,Abid, 08/21/2018