NPPS Leadership Team

[Brett Viren <bv AT bnl.gov>]

Hi Torre,

I'll beg off a little longer.  Here's why:

We now have LS4GAN translation between "low-fi" and "high-fi" LArTPC
detector simulations of cosmic muon events.  But, comparisons between
translated and target domains are so far in the form of L2-difference at
the "pixel" level.  That's rather dry and boring for physicists.

We are starting to work on comparisons involving reconstructed
ionization electron distributions (Wire-Cell signal processing) and
subsequent Wire-Cell 3D imaging.  Once those are done then we have the
content for our next paper and a better story to tell at a joint
meeting.

That would likely put us more toward the end of your list of meeting
dates.

-Brett.

Torre Wenaus <wenaus AT gmail.com> writes:

> Hi Brett,
> In the end I didn't give feedback but I really appreciated this. And I take 
> from it that you et al are ready for
> LS4GAN (and the reflections here) being included in an AI/ML joint meeting! 
> Our expected NPPS meeting schedule at
> this point is as follows. 
>
>   • Jan 10 - no meeting, ePIC collab meet (and respecting the subsequent 
> holiday)
>    
>   • Jan 17 - meeting
>    
>   • Jan 24 - meeting
>    
>   • Jan 31 - meeting
>    
>   • Feb 7 - no meeting, ATLAS S&C Week
>    
>   • Feb 14 - meeting
>    
>   • Feb 21 - no meeting respecting the holiday
>    
>   • Feb 28 - meeting
>
> Torre
>
> On Wed, Nov 15, 2023 at 11:48 AM Brett Viren <bv AT bnl.gov> wrote:
>
>     The "spin doctor'ish" connection is between "Extending our leadership in
>     Responsible AI" on slide 4 and the LS4GAN LDRD.  It's a bit awkward for
>     me to state well as it raises some vague dispersion on our field.  A
>     real spin doctor would say things in less objectionable terms.
>    
>     Anyways,
>    
>     The goal of LS4GAN is to develop a tool to minimize the systematic
>     difference between simulated event data and event data from real
>     detectors and to quantify and propagate the residual systematic
>     difference through arbitrary downstream processing to the final results.
>    
>     LS4GAN ties to "responsible AI" because it attacks a core misuse of
>     AI/ML in our field that I think we are all familiar with: train a
>     network on simulated events and apply the trained network to make
>     inference on real detector events.  Misuse arises because simulation, in
>     fact, does not (ever) perfectly replicate real detector output.  Thus
>     the network is not trained to properly infer about real data.  This is
>     sometimes called the "Domain Shift" problem.
>    
>     It is okay to have an imperfect AI/ML method if the systematic
>     differences between simulated and real data are estimated.  However, and
>     where the "irresponsible AI" comes in, some AI/ML researchers in our 
> field
>     will neglect this "domain shift" systematic.
>    
>     I suspect some reason for this is that systematics are the "hard part"
>     of experimental physics and also many of the researchers that are drawn
>     towards AI/ML techniques typically lack the expertise to properly handle
>     systematics.  Perhaps, myself included!
>    
>     So, LS4GAN is trying to provide tools to level up our ability to
>     properly handle "AI-induced systematics" so that we may use AI
>     "responsibly" and since BNL LDRD payed for it, we might use it as an
>     example of "Extending our leadership" in this area.
>
>     BTW, but perhaps pertinent, the LS4GAN R&D has uncovered some
>     fundamental issues in applying domain mapping techniques to scientific
>     data.  Ultimately, we require the mapping to be "meaningful" in some
>     sense and so far fail to see how to do this.  Specifically, I feel we
>     must construct some spacial metric in the latent feature space in which
>     the translation occurs so that we can know how "far" the translation has
>     gone from sim to real and how "far" that translation is to "fully real".
>    
>     In other words, we can confidently say "we transform the simulated
>     events to make them more realistic".  But we can not quantify how much
>     "realism" they accrued nor how much they still lack.  Furthermore,
>     transforming to increase realism is fundamentally counter to retaining
>     features of the original simulated event, though we necessarily want to
>     include both constraints.  This we've dubbed the "LS4GAN dilemma" and it
>     pits "consistency" against "realism".
>    
>     These issues need work to make the LS4GAN technique robust and
>     understood.  To get there requires more R&D beyond the LDRD (it ends
>     this summer) and that needs help from some people with more
>     mathematical, theory-minded expertise that the team has.  CSI had a good
>     math'y / theory type post-doc working with us for a while.  He was
>     making good progress but was called off to do other work before his work
>     with us could reach some fruition.
>
>     -Brett.
>
>     Torre Wenaus via Phys-npps-mgmt-l <phys-npps-mgmt-l AT lists.bnl.gov>
>     writes:
>    
>     > Interesting reading, though after reading it through I still don't 
> know what Human AI
>     > Facility Integration actually concretely is.
>     >
>     > My notables:
>     > slide 4: the vision is the targeting of funding agencies.
>     > slide later on: "JLAB and LBNL with their win of HPDF sit in the pole 
> position to direct
>     > where future integrated research infrastructure investments go and 
> what shape they will take,
>     > presenting a significant challenge to BNL Data Intensive Computing. 
> JLAB has been developing
>     > an excellent reputation in AI for NP data analysis."
>     > Quite a change from the routine previous comment about the little lab 
> not consequential for
>     > computing. 
>     >
>     > If the share doesn't work here is a copy
>     > 
> https://docs.google.com/presentation/d/1a5Fx-H1OnURGk3W-CWvaVBnFJJT5MDLoFbbM_q6XpNQ/edit?usp=sharing
>     >
>     > Happy to receive and convey comments. I was asked to add milestones, 
> I have no idea what I
>     > might add to this.
>     >
>     >   Torre
>     >
>     > ---------- Forwarded message ---------
>     > From: Kleese Van Dam, Kerstin <kleese AT bnl.gov>
>     > Date: Tue, Nov 7, 2023 at 2:11 PM
>     > Subject: HAI-FI Initiative pitch
>     > To: Hoisie, Adolfy <ahoisie AT bnl.gov>, Jha, Shantenu 
> <shantenu AT bnl.gov>, D'Imperio, Nicholas <
>     > dimperio AT bnl.gov>, Yoo, Shinjae <sjyoo AT bnl.gov>, Lin, Meifeng 
> <mlin AT bnl.gov>, Urban, Nathan <
>     > nurban AT bnl.gov>, Wilkins, Stuart <swilkins AT bnl.gov>, Torre Wenaus 
> <wenaus AT gmail.com>, Rogers,
>     > Alistair <arogers AT bnl.gov>
>     > Cc: Kleese Van Dam, Kerstin <kleese AT bnl.gov>
>     >
>     > Hi all,
>     >
>     > We are asked to present the pitch for a HAI-FI Initiative in the week 
> starting November 20.
>     > Please find here the link to the current incomplete slide deck. I 
> would very much appreciate
>     > your comments and improvements. Please feel free and suggest changes 
> in the slide deck itself
>     > or add comments.
>     > 
> https://docs.google.com/presentation/d/1-449_NeNcTtGnB1Iajkd2rAa8mLjRDBaRkGh48j0Jv8/edit?usp=sharing
>     > Attached here is the guidance for the slide deck that I received.
>     >
>     > Many thanks,
>     >
>     > Kerstin
>     >
>     >  
>     >
>     > Kerstin Kleese Van Dam
>     > Director
>     > Computational Science Initiative
>     >
>     > Building 725, room 2-127B
>     > Office: 631.344.6019
>     >
>     > Cell: 509 221 0758
>     > kleese AT bnl.gov
>     >                   
>     > signature_11374550
>     > Twitter | Facebook | Instagram | LinkedIn
>     >
>     >  
>     >
>     >  
>     >
>     > --
>     > -- Torre Wenaus, BNL NPPS Group, ATLAS Experiment
>     > -- BNL 510A 1-222 | 631-681-7892
>     >
>     > _______________________________________________
>     > Phys-npps-mgmt-l mailing list
>     > Phys-npps-mgmt-l AT lists.bnl.gov
>     > https://lists.bnl.gov/mailman/listinfo/phys-npps-mgmt-l
>
> --
> -- Torre Wenaus, BNL NPPS Group, ATLAS Experiment
> -- BNL 510A 1-222 | 631-681-7892

Attachment: signature.asc
Description: PGP signature