Sphenix-calibration-l mailing list

["Benjamin, Douglas" <dbenjamin AT bnl.gov>]

Hello,

  Would it be possible for  the 'rev 3' of the flow chart include the  merge 
steps also listed. If all of the steps are listed in the flow chart then it 
more straight forward to understand the intermediate data products, their 
sizes, how often they are aggregated.  I would like to start to assemble the 
processing DAG (at least on paper). The flow chart will be the input to the 
DAG, there will be some steps that will need to be run in parallel to produce 
enough output for the aggregation steps.  I know I might be stating the 
obvious but I would like to write it down.  If there are a lot of small files 
produced by many processes across many nodes, then we need a storage device 
that can handle the load. Such things are not cheap. 

Regards,

Doug Benjamin


-----Original Message-----
From: sPHENIX-calibration-l <sphenix-calibration-l-bounces AT lists.bnl.gov> on 
behalf of Ross Corliss via sPHENIX-calibration-l 
<sphenix-calibration-l AT lists.bnl.gov>
Reply-To: Ross Corliss <rcorliss AT mit.edu>
Date: Monday, June 7, 2021 at 2:14 PM
To: Hugo Pereira Da Costa <hugo.pereira-da-costa AT cea.fr>
Cc: sphenix calibrations <sphenix-calibration-l AT lists.bnl.gov>
Subject: Re: [Sphenix-calibration-l] Input for Collaboration Meeting

    HI Hugo et al.,

    I think the details of how the distortions are derived and combined 
(hence the flow chart -- probably a 
    'rev 3', with the 'validate' step fleshed out) would be good to show, 
along with the status of the components.  This matches a lot of Hugo's 
comments. 
    Some of the details of the central membrane distortion monitoring (which 
we will discuss tomorrow, and hopefully have a pull request before the collab 
meeting) have also progressed since last time.

    I would be glad to assemble the slides and present the status.

    -Ross
    > On Jun 7, 2021, at 12:14 PM, Hugo Pereira Da Costa via 
sPHENIX-calibration-l <sphenix-calibration-l AT lists.bnl.gov> wrote:
    > 
    > Hello Christof, (repeating here what we discussed semi-privately)
    > 
    > On the track-based space charge distortion reconstruction, there really 
have not been much new things since last collaboration meeting, mainly 
because things where in a reasonable state by then already.
    > 
    > Obviously, there have been developments on making the TPOT happening 
for real (something which is critical to track-based SC corrections), but I 
expect this to be covered in a separate talk in the detector section
    > 
    > To me, the newest thing that should really be presented at the 
collaboration meeting is the effort, led by Ross, to put all the bits and 
pieces we have on the TPC SC distortion correction in a comprehensive flow 
chart, organized in a first-pass and second-pass reconstruction.
    > 
    > We should then emphasize which parts of this flowchart are already 
there (committed, ready to use), which are 'almost there', and which still 
need some significant work. I'll let Ross comment on this (and hopefully 
volunteer to present this).
    > 
    > On that topic, one of the missing parts of the flow-chart, namely the 
ability for the first-pass reconstruction to store the per-job space-charge 
correction matrices in independent, tiny files, together with a separate 
process to aggregate, invert these matrices and produce a full map, has been 
pushed to our repositories last week. So we are one step closer to having 
this flow chart turned into a real thing.
    > 
    > That is all the input I have.
    > 
    > Best,
    > 
    > Hugo
    > 
    > 
    > 
    > On 6/4/2021 8:58 AM, Christof Roland via sPHENIX-calibration-l wrote:
    >> Thanks to Klaus for pointing this out, obviously the mail should
    >> have started with "DeaR Calibration TaskFocre Members"
    >> 
    >> I sincerely apologize!
    >> 
    >> This will go onto my list of the more memorable typos or autocorrect 
failures.
    >> 
    >> Cheers
    >> 
    >>    Christof
    >> 
    >> 
    >>> On 4. Jun 2021, at 16:40, Christof Roland via sPHENIX-calibration-l 
<sphenix-calibration-l AT lists.bnl.gov> wrote:
    >>> 
    >>> Dead Calibration TaskForce Members,
    >>> 
    >>> we are supposed to provide feedback to Dave & Gunther so they
    >>> can set up the program for the upcoming collaboration meeting.
    >>> 
    >>> Could you please provide us with a few line response on the
    >>> topics you would want to highlight at the collaboration meeting?
    >>> 
    >>> Please also let us know if you would prefer the usual layout
    >>> if a separate report by each subcommittee or if we should
    >>> change for this particular meeting to a shorter executive style
    >>> report to keep the meeting time short and leaving more time for
    >>> discussion, as Gunther and Dave suggested last General Meeting
    >>> 
    >>> Cheers
    >>> 
    >>>   Takao & Christof
    >>> _______________________________________________
    >>> sPHENIX-calibration-l mailing list
    >>> sPHENIX-calibration-l AT lists.bnl.gov
    >>> https://lists.bnl.gov/mailman/listinfo/sphenix-calibration-l
    >> _______________________________________________
    >> sPHENIX-calibration-l mailing list
    >> sPHENIX-calibration-l AT lists.bnl.gov
    >> https://lists.bnl.gov/mailman/listinfo/sphenix-calibration-l
    > _______________________________________________
    > sPHENIX-calibration-l mailing list
    > sPHENIX-calibration-l AT lists.bnl.gov
    > https://lists.bnl.gov/mailman/listinfo/sphenix-calibration-l

    _______________________________________________
    sPHENIX-calibration-l mailing list
    sPHENIX-calibration-l AT lists.bnl.gov
    https://lists.bnl.gov/mailman/listinfo/sphenix-calibration-l