U.S. ATLAS HL-LHC Upgrade Level 2 and Deputies-NSF only Management Mailing List

[Thomas Schwarz <schwarzt AT umich.edu>]

Geez, we should stop working so we don’t fall farther behind!

On Sun, Feb 26, 2023 at 3:32 AM Gustaaf Brooijmans <gusbroo AT nevis.columbia.edu> wrote:

Hi Tom,

Good question!  The base cost is $2.5M.  Of that, you have only done $160k, so you would think it might be ~$2.3M.  However, you have a $285k cost variance, so you end up at $2.6M!

Best,

Gustaaf

On Feb 25, 2023, at 11:44 PM, Thomas Schwarz <schwarzt AT umich.edu> wrote:

Hi Chris,

Could you explain how the ‘remaining cost’ for the CSM is more than the base cost?  

Thanks,

Tom

On Sat, Feb 25, 2023 at 2:13 PM Chris Meyer <chris.meyer AT cern.ch> wrote:

Hi John, All,

I’ve re-run the simulation removing the proposed supply-chain risks. The float decreases slightly but still doesn’t dip below 400 days. I’ve attached the results (there is a new "LAr Maturity+Risk (few retired)” tab).

It looks like there are still quite a few supply-chain risks with 6+ month delays that keep the float large, even after retiring those you proposed:

RN-06-04-01-005add

RN-06-04-02-002add

RN-06-04-02-005add

RN-06-04-02-006add

RN-06-04-03-002add

RN-06-04-03-003add

(There are some DOE risks too, but I’m less sure about which of these might impact the NSF schedule.)

Cheers,

Chris

On Feb 25, 2023, at 15:29, John Parsons <parsons AT nevis.columbia.edu> wrote:

So, looking at the LAr maturity+risk tab, I see we are not reaching even 70% CL on the float, so it is not useful to add that info to my talk. As for your statement about "looking into options", the things you state are fine.  However, I think the imminent retirement of a number of the key Supply Chain Risks is more promising, at least to get a more achievable target to try reach.

Looking ahead, we could conceivably retire or at least greatly reduce, the following:

RN-06-04-01-003add
RN-06-04-01-004add
RN-06-04-01-006add
RN-06-04-02-001add
RN-06-04-02-003add
RN-06-04-03-001add
RD-06-04-05-001add
RD-06-04-05-002add

Would it be possible for Chris to quickly rerun LAr with these retired? That would give us a good idea of what we could try accomplish before freezing things for the April review.

John

On 2/25/23 2:33 PM, Gustaaf Brooijmans wrote:

Ok, so I added a statement
     o We are looking into options to address this: start parts
       procurement before ATLAS Final Design Review (“priming” in
       ATLAS-speak), accelerate production, optimize installation, etc.

On Feb 25, 2023, at 1:53 PM, Gustaaf Brooijmans <gusbroo AT nevis.columbia.edu <mailto:gusbroo AT nevis.columbia.edu>> wrote:


Hi John,

You mention "NSF guidance for MREFCs is that TPC should be between the 70% and 90% CL".  Does the same guidance apply to completion date?  In any case, do we have from the MC completion dates for less than 90% CL; it might be a natural question to ask what CL we are consistent with at the moment.

Yes, they’re in the sheet.  You’re looking for the LAr maturity+risk tab.

I would not say the supply chain risks for LAr are "wrong".  For the recent NSF risk review, we decided (as I believe did other subsystems) to keep unchanged the supply chain risks for items that are not yet completed; ie. we did not revisit the "standard" impacts assumed previously for the various types of things, such as PCB fab, PCB assy, ASICs, ...  The same was true for the DOE side, where we did not see a float problem coming from this on the MC for CD/2-3.  So I was not expecting such a stark problem to appear on the NSF side.

The simulation for the DOE review said 400 days, so this is not so far from what we expected, no?

As I mentioned previously, we can (very) soon retire some of these supply chain risks, at which point there is no need to tweak their potential impacts.  For some others, it might be time to revisit the "standard" impact assumptions,

I think I will have to add a slide to my L2 talk that proactively addresses these issues.  It will likely not deflect the questions that this is going to cause, but at least we will be seen to be tackling them head on.

Yes, I agree.  But I think we do want to decide what we would like the recommendation to say as that will change how we address this.

Best,

Gustaaf

John

On 2/25/23 12:13 PM, Gustaaf Brooijmans wrote:

Hi John,

We are using all the "standard" impacts proposed across the project for Supply Chain, eg. up to 4 months delay for PCB fab.  The LAr issue being revealed here, with the implication we could need an extra ~9 months more than we have before the needed-at-CERN dates, is maybe a reflection of the fact these are (by now) too conservative?  Do we really want to show this at DR??  It seems to me it could become the "dominant issue" of the entire review?  And how do we reply when asked "what happens if the MC is right and you deliver LAr boards 9 months too late"??

Do you think the supply chain risk registers for LAr are wrong?  We recently had an NSF risk workshop… Anyway, we don’t really have a choice I think in terms of what we show at the DR.  It’s too late to change our approach.
I put some text in my slides along the lines of “we might need extra funds to accelerate production,” I’m happy to hear more creative suggestions.  Of course it’s not a secret for anybody that pixels and strips for both ATLAS and CMS are in much worse shape.

Given we have not seen this problem before yesterday, we don't currently have anything prepared to address it.  (as requested, I do not even have the MC results in my L2 talk)  By April, I can see quite a few improvements, since the ADC preprod will be done and the lpGBT production should be done.  Also, we should have received the preprod. PA/S (from DOE scope) by then.   So we could "retire" those.  All that would help, but who knows whether it would be enough to get us to 90%CL float.  We could consider changing some of the assumptions.  For example, based on recent experience, I don't think a potential 4-month delay for PCB fab is a serious possibility anymore.  I would be comfortable reducing it to 2 months max.

That’s up to you.  I think PCB fab is probably less of an issue now that China has ended zero-covid.  Parts remain an issue, but maybe for FEB2 there are few things that can go wrong there.

So, bottom line, here are some questions:

- for next week, you are going to show these float results in your plenary talk?  And if so, what are we going to say?  Just say this is a current snapshot, discuss the points above, and argue it will be "better" by April?  That still leaves open for discussion whether it will be "good enough", and the answer is that today we don't know.  This seems problematic to me.

I don’t think we have a choice.  Please take a look at my slides.  I’m happy to adjust the text.

- Or, given that all these changes will happen by April, do we choose to NOT show the MC schedule result next week?  Once the DR is over, we could work with Chris to see where we expect things to stand at the time of the rebaseline review?

We have to show MC results.  Since this is the DR, I don’t think it’s a huge deal.  We just need to think what the recommendation is we’re fishing for.  Is it “revise the LAr risks” or is it “prepare a plan to accelerate production should this be needed”?
Best,
Gustaaf

John

On 2/24/23 5:34 PM, Gustaaf Brooijmans wrote:

Hi John,
Yes, but it turns out this is all due to supply chain.  In the working summary sheet you’ll see I made some small annotations.  Without supply chain the 438/461 becomes 251/288, so ~90% CL.  We didn’t have supply chain risks at FDR, or at the summer 2021 annula review.
Best,
Gustaaf

On Feb 24, 2023, at 4:48 PM, John Parsons <parsons AT nevis.columbia.edu <mailto:parsons AT nevis.columbia.edu>> wrote:


Hi Gustaaf,

I am surprised that @ 90%CL LAr seems to need about 200 "extra" days (almost a year) beyond the float we have.  What is driving that?  We have certainly made a lot of progress since FDR, and were not in such bad shape back then.  We also have more base float now than we did at FDR, and yet we are "under water"?

John

On 2/24/23 4:39 PM, Gustaaf Brooijmans wrote:

Hello all,
We think we’ve worked the kinks out of all the numbers.  In the attached you will find a for presentation tab, as well as a working tab with a bit more info on floats, and all the detailed info.  Let us know if you see something weird.  The for presentation tab calculates % contingency on the cost-to-go, which gives a better idea of where we are.
Best,
Chris and Gustaaf
_______________________________________________
Usatlas-hllhc-l2deputymgmt-nsf-l mailing list
Usatlas-hllhc-l2deputymgmt-nsf-l AT lists.bnl.gov <mailto:Usatlas-hllhc-l2deputymgmt-nsf-l AT lists.bnl.gov>
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-l2deputymgmt-nsf-l

--
______________________________________________________________________

John Parsons
Nevis Labs, Email: parsons AT nevis.columbia.edu <mailto:parsons AT nevis.columbia.edu>
Columbia UniversityPhone: (914) 591-2820
P.O. Box 137Fax: (914) 591-8120
Irvington, NY 10533WWW: https://urldefense.com/v3/__http://www.nevis.columbia.edu/*parsons/__;fg!!P4SdNyxKAPE!HBgbH9-tAB8XvzcQbxzWrhxnc1bZWKHl2E1TcTMED93qGUw6TyH54ShoWcgtknTf5tECy9ZeP3uWl9zU4hfEoQq6UzwYr5YO0EZrhjxZHhqj-MFSvQHhXg$

______________________________________________________________________

--
______________________________________________________________________

John Parsons
Nevis Labs, Email: parsons AT nevis.columbia.edu <mailto:parsons AT nevis.columbia.edu>
Columbia UniversityPhone: (914) 591-2820
P.O. Box 137Fax: (914) 591-8120
Irvington, NY 10533WWW: https://urldefense.com/v3/__http://www.nevis.columbia.edu/*parsons/__;fg!!P4SdNyxKAPE!HBgbH9-tAB8XvzcQbxzWrhxnc1bZWKHl2E1TcTMED93qGUw6TyH54ShoWcgtknTf5tECy9ZeP3uWl9zU4hfEoQq6UzwYr5YO0EZrhjxZHhqj-MFSvQHhXg$

______________________________________________________________________

--
______________________________________________________________________

John Parsons
Nevis Labs, Email: parsons AT nevis.columbia.edu <mailto:parsons AT nevis.columbia.edu>
Columbia UniversityPhone: (914) 591-2820
P.O. Box 137Fax: (914) 591-8120
Irvington, NY 10533WWW: https://urldefense.com/v3/__http://www.nevis.columbia.edu/*parsons/__;fg!!P4SdNyxKAPE!HBgbH9-tAB8XvzcQbxzWrhxnc1bZWKHl2E1TcTMED93qGUw6TyH54ShoWcgtknTf5tECy9ZeP3uWl9zU4hfEoQq6UzwYr5YO0EZrhjxZHhqj-MFSvQHhXg$

______________________________________________________________________

_______________________________________________
Usatlas-hllhc-l2deputymgmt-nsf-l mailing list
Usatlas-hllhc-l2deputymgmt-nsf-l AT lists.bnl.gov <mailto:Usatlas-hllhc-l2deputymgmt-nsf-l AT lists.bnl.gov>
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-l2deputymgmt-nsf-l

--
______________________________________________________________________

John Parsons
Nevis Labs, Email: parsons AT nevis.columbia.edu
Columbia University Phone: (914) 591-2820
P.O. Box 137 Fax: (914) 591-8120

Irvington, NY 10533 WWW: http://www.nevis.columbia.edu/~parsons/

______________________________________________________________________

_______________________________________________
Usatlas-hllhc-l2deputymgmt-nsf-l mailing list
Usatlas-hllhc-l2deputymgmt-nsf-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-l2deputymgmt-nsf-l

--

-------------------------------------
Tom Schwarz
Associate Professor
Department of Physics
University of Michigan
Office: 734-764-8941
Lab: 734-763-7805
schwarzt AT umich.edu
-------------------------------------

_______________________________________________
Usatlas-hllhc-l2deputymgmt-nsf-l mailing list
Usatlas-hllhc-l2deputymgmt-nsf-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/usatlas-hllhc-l2deputymgmt-nsf-l

--

-------------------------------------
Tom Schwarz
Associate Professor
Department of Physics
University of Michigan
Office: 734-764-8941
Lab: 734-763-7805
schwarzt AT umich.edu
-------------------------------------