NPPS Leadership Team

[Brett Viren <bv AT bnl.gov>]

The "spin doctor'ish" connection is between "Extending our leadership in
Responsible AI" on slide 4 and the LS4GAN LDRD.  It's a bit awkward for
me to state well as it raises some vague dispersion on our field.  A
real spin doctor would say things in less objectionable terms.

Anyways,

The goal of LS4GAN is to develop a tool to minimize the systematic
difference between simulated event data and event data from real
detectors and to quantify and propagate the residual systematic
difference through arbitrary downstream processing to the final results.

LS4GAN ties to "responsible AI" because it attacks a core misuse of
AI/ML in our field that I think we are all familiar with: train a
network on simulated events and apply the trained network to make
inference on real detector events.  Misuse arises because simulation, in
fact, does not (ever) perfectly replicate real detector output.  Thus
the network is not trained to properly infer about real data.  This is
sometimes called the "Domain Shift" problem.

It is okay to have an imperfect AI/ML method if the systematic
differences between simulated and real data are estimated.  However, and
where the "irresponsible AI" comes in, some AI/ML researchers in our field
will neglect this "domain shift" systematic.

I suspect some reason for this is that systematics are the "hard part"
of experimental physics and also many of the researchers that are drawn
towards AI/ML techniques typically lack the expertise to properly handle
systematics.  Perhaps, myself included!

So, LS4GAN is trying to provide tools to level up our ability to
properly handle "AI-induced systematics" so that we may use AI
"responsibly" and since BNL LDRD payed for it, we might use it as an
example of "Extending our leadership" in this area.



BTW, but perhaps pertinent, the LS4GAN R&D has uncovered some
fundamental issues in applying domain mapping techniques to scientific
data.  Ultimately, we require the mapping to be "meaningful" in some
sense and so far fail to see how to do this.  Specifically, I feel we
must construct some spacial metric in the latent feature space in which
the translation occurs so that we can know how "far" the translation has
gone from sim to real and how "far" that translation is to "fully real".

In other words, we can confidently say "we transform the simulated
events to make them more realistic".  But we can not quantify how much
"realism" they accrued nor how much they still lack.  Furthermore,
transforming to increase realism is fundamentally counter to retaining
features of the original simulated event, though we necessarily want to
include both constraints.  This we've dubbed the "LS4GAN dilemma" and it
pits "consistency" against "realism".

These issues need work to make the LS4GAN technique robust and
understood.  To get there requires more R&D beyond the LDRD (it ends
this summer) and that needs help from some people with more
mathematical, theory-minded expertise that the team has.  CSI had a good
math'y / theory type post-doc working with us for a while.  He was
making good progress but was called off to do other work before his work
with us could reach some fruition.


-Brett.


Torre Wenaus via Phys-npps-mgmt-l <phys-npps-mgmt-l AT lists.bnl.gov>
writes:

> Interesting reading, though after reading it through I still don't know 
> what Human AI
> Facility Integration actually concretely is.
>
> My notables:
> slide 4: the vision is the targeting of funding agencies.
> slide later on: "JLAB and LBNL with their win of HPDF sit in the pole 
> position to direct
> where future integrated research infrastructure investments go and what 
> shape they will take,
> presenting a significant challenge to BNL Data Intensive Computing. JLAB 
> has been developing
> an excellent reputation in AI for NP data analysis."
> Quite a change from the routine previous comment about the little lab not 
> consequential for
> computing. 
>
> If the share doesn't work here is a copy
> https://docs.google.com/presentation/d/1a5Fx-H1OnURGk3W-CWvaVBnFJJT5MDLoFbbM_q6XpNQ/edit?usp=sharing
>
> Happy to receive and convey comments. I was asked to add milestones, I have 
> no idea what I
> might add to this.
>
>   Torre
>
> ---------- Forwarded message ---------
> From: Kleese Van Dam, Kerstin <kleese AT bnl.gov>
> Date: Tue, Nov 7, 2023 at 2:11 PM
> Subject: HAI-FI Initiative pitch
> To: Hoisie, Adolfy <ahoisie AT bnl.gov>, Jha, Shantenu <shantenu AT bnl.gov>, 
> D'Imperio, Nicholas <
> dimperio AT bnl.gov>, Yoo, Shinjae <sjyoo AT bnl.gov>, Lin, Meifeng 
> <mlin AT bnl.gov>, Urban, Nathan <
> nurban AT bnl.gov>, Wilkins, Stuart <swilkins AT bnl.gov>, Torre Wenaus 
> <wenaus AT gmail.com>, Rogers,
> Alistair <arogers AT bnl.gov>
> Cc: Kleese Van Dam, Kerstin <kleese AT bnl.gov>
>
> Hi all,
>
> We are asked to present the pitch for a HAI-FI Initiative in the week 
> starting November 20.
> Please find here the link to the current incomplete slide deck. I would 
> very much appreciate
> your comments and improvements. Please feel free and suggest changes in the 
> slide deck itself
> or add comments. 
> https://docs.google.com/presentation/d/1-449_NeNcTtGnB1Iajkd2rAa8mLjRDBaRkGh48j0Jv8/edit?usp=sharing
> Attached here is the guidance for the slide deck that I received.
>
> Many thanks,
>
> Kerstin
>
>  
>
> Kerstin Kleese Van Dam
> Director
> Computational Science Initiative
>
> Building 725, room 2-127B
> Office: 631.344.6019
>
> Cell: 509 221 0758
> kleese AT bnl.gov
>                   
> signature_11374550
> Twitter | Facebook | Instagram | LinkedIn
>
>  
>
>  
>
> --
> -- Torre Wenaus, BNL NPPS Group, ATLAS Experiment
> -- BNL 510A 1-222 | 631-681-7892
>
> _______________________________________________
> Phys-npps-mgmt-l mailing list
> Phys-npps-mgmt-l AT lists.bnl.gov
> https://lists.bnl.gov/mailman/listinfo/phys-npps-mgmt-l

Attachment: signature.asc
Description: PGP signature