NPPS Leadership Team

["Van Buren, Gene" <gene AT bnl.gov>]

Hi, all

Here is the final Recommendations document from this year's RHIC PAC:

https://www.bnl.gov/npp/docs/2022-npp-pac-recommendations-final.pdf

I have cut & paste the sections I found relevant to software & computing 
below for those who don't want to scan the document for these points 
themselves. The final "Offline Computing" section primarily emphasizes ("most 
profound concern") the upcoming computing hardware resource needs of both 
STAR and sPHENIX, not prioritizing either over the other. PHENIX analysis 
resource needs are also noted.

-Gene


_____________________________

1. Executive Summary
...
The PAC endorses STAR’s emphasis on the importance of maintaining sufficient 
expert personpower for software, computing and operations at BNL, which is 
critical for high-quality data collection, and timely calibration and 
production of data.
...
sPHENIX has met and surpassed its goals for designing, building and testing 
its software framework for compression, storage and analysis. It has also 
improved the memory footprint of the reconstruction software to optimize 
implementation at the SDCC. Software appears to be on schedule for 
simulation, reconstruction, calibration, compression and storage, for the 
inaugural run in February. The PAC is pleased to hear that the software is in 
place and has passed two Mock Data Challenges.

Detailed hardware requirements for off-line computing for sPHENIX have been 
provided to BNL management, including the use of off-site computing 
resources. The 15 kHz event-rate of sPHENIX requires an increase in CPU and 
storage usage of an order of magnitude. Concerns regarding computing 
resources were expressed in the 2021 S&C reports but have yet to be addressed 
adequately. sPHENIX estimates an increase by about a factor of two in 
resources is needed for initial processing of off-line data. It is imperative 
that sPHENIX and BNL management address these concerns in time for successful 
data-taking and analysis for Run 23. The PAC urges BNL management to assign 
highest priority to the procurement of the computational resources needed to 
analyze RHIC data in a timely manner.
...


_____________________________

3.1 STAR
...
The PAC is pleased to see the progress in the software developments for 
detector calibrations, alignment, offline data reconstruction, and the 
initial fast simulations. We look forward to seeing the completion of the 
full detector response simulation software, the optimization of the tracking 
software, and eventually – the data analysis and physics results.
...
The PAC endorses STAR’s emphasis on the importance of maintaining sufficient 
expert personpower for software, computing and operations at BNL, as is 
critical for high-quality data collection, and timely calibration and 
production of data.
...


_____________________________

3.2 PHENIX
...
PHENIX made significant progress in data and analysis preservation using 
international expert systems such as REANA, Zenodo and HEPdata.

The PAC supports these efforts in data preservation. The PAC supports 
PHENIX’s request to BNL management for resources necessary to complete 
critical analyses such as b,c→μ decays.


_____________________________

4 sPHEINX
...
4.2 Readiness for Data Taking including Computing & Commissioning Plan

The PAC was pleased to learn that sPHENIX has met and surpassed its goals for 
designing, building and testing its software framework for compression, 
storage and analysis. Using the current snapshot, sPHENIX can analyze a Au+Au 
event in 21 seconds, where the 2019 goal was set at 24 seconds. Further, 
sPHENIX has improved the memory footprint of their reconstruction software to 
optimize implementation at the SDCC. Software is on schedule for simulation, 
reconstruction, calibration, compression and storage, for the inaugural run 
in February.

Since 2019 sPHENIX has undergone annual S&C reviews. The PAC is pleased to 
hear that the software is in place and has passed two Mock Data Challenges. 
Reports from the 2021 S&C reviews were positive with respect to the software, 
but serious problems in procuring the resources were discussed. An S&C review 
was undertaken in May of 2022, but the report from that review was not yet 
available to the PAC.

Detailed hardware requirements for off-line computing have been provided to 
BNL management. This includes studies of using off-site computing resources, 
e.g. the OSG. Not surprisingly, the 15 kHz event-rate of sPHENIX (which was 
the principal motivation in building sPHENIX) requires an increase in CPU and 
storage usage of an order of magnitude.

The serious concerns about resources for off-line computing expressed in the 
2021 S&C report are not yet addressed. As these concerns affect both STAR and 
PHENIX, they are addressed below.


_____________________________

5. Off-Line Computing for Runs 23-25

The PAC expresses its most profound concern regarding the computing needs for 
both sPHENIX and STAR for runs 23-25. Despite the significant progress 
sPHENIX has made in optimizing their computational software, the sPHENIX 
computing needs remain an order of magnitude greater than previous 
experiments, and STAR is managing to double their event throughput while 
implementing several new detector subsystems. The current SDCC planned 
computing resources are more than 50% short of what is needed to meet the 
sPHENIX and STAR needs in 2023, and the shortfall is expected to increase in 
outer years. Without additional computing resources, the ability of sPHENIX 
to calibrate their analyses and plan for future runs will be severely 
compromised, and scientific results from both collaborations will be severely 
delayed. This shortfall needs to be addressed in the coming months to avoid a 
tragic underuse of the scientific effort being invested to collect this data.

The PAC urges BNL management to assign highest priority to the procurement of 
the computational resources needed to analyze this data in a timely manner. 
The PAC requests that BNL, STAR and sPHENIX provide an update on their plans 
for procuring the needed resources in October of this year.


_____________________________

7. PAC Reommendations
BNL Lab Management
...
The PAC urges BNL management to assign highest priority to the procurement of 
the computational resources needed to analyze RHIC data in a timely manner.
...