STAR QA Board

[李东升 <erl@mail.ustc.edu.cn>]

Hi all,

Thanks to the helpful discussion in the QA meeting, we have decided which badruns from AI diagnosis to be false positive. So I update the total badrun list in the following slides:

https://drupal.star.bnl.gov/STAR/system/files/dsli_Run21_3GeV_total_list_240926_v1.pdf

Best regards,

Dongsheng

-----原始邮件-----
发件人: "李东升 (via star-qaboard-l Mailing List)" <star-qaboard-l@lists.bnl.gov>
发送时间: 2024-09-26 13:51:46 (星期四)
收件人: "ASHIK IKBAL" <ashikhep@gmail.com>
抄送: "Tsang, Chun Yuen" <ctsang@bnl.gov>, "star-qaboard-l@lists.bnl.gov" <star-qaboard-l@lists.bnl.gov>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Ashik and all,

Please find my slides for discussion at 

https://drupal.star.bnl.gov/STAR/system/files/dsli_Run21_3GeV_total_list_240926.pdf

Best regards,

Dongsheng

-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-09-25 23:46:00 (星期三)
收件人: 李东升 <erl@mail.ustc.edu.cn>
抄送: "Tsang, Chun Yuen" <ctsang@bnl.gov>, "star-qaboard-l@lists.bnl.gov" <star-qaboard-l@lists.bnl.gov>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongsheng,

I think you did a good job with the run-by-run QA so far. Just one last step is left as explained by Tommy. Thanks to Tommy for explaining this.

Could you please do the checks (explained by Tommy) and present the combined badrun list on the QA board meeting.

Thanks,

Ashik 

On Wed, Sep 25, 2024 at 11:34 AM 李东升 <star-qaboard-l@lists.bnl.gov> wrote:

Hi Tommy,

I also missed the long mail on Sep.10th from you. So you plan to use AI+run-by-run QA, This answers part of my question.

Best regards

Dongsheng

-----原始邮件-----
发件人: "Tsang, Chun Yuen" <ctsang@bnl.gov>
发送时间: 2024-09-25 22:34:49 (星期三)
收件人: 李东升 <erl@mail.ustc.edu.cn>
抄送: "ASHIK IKBAL" <ashikhep@gmail.com>, "star-qaboard-l@lists.bnl.gov" <star-qaboard-l@lists.bnl.gov>
主题: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongsheng,

 

To add on to Ashik’s email, the following message is detached from the email chain, so I am sending it again just in case you missed that.

 

Best wishes,

Tommy

 

From: Tsang, Chun Yuen
Sent: Tuesday, September 10, 2024 5:26 PM
To: 李东升 <erl@mail.ustc.edu.cn>
Cc: ASHIK IKBAL <ashikhep@gmail.com>; star-qaboard-l@lists.bnl.gov
Subject: RE: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

 

Hi Dongsheng,

 

Sorry for the late reply. I missed your email.

 

I think the agreement is that QA worker probably needs to manually review those 44 bad runs. AI can only narrow down the run list so you don’t need to go through all ~500 runs, but people still need to do the final inspection.

 

If you think the shift log entries for some of those 44 runs are not that bad, I suggest that you take those runs out, make a list of runs that you “recovered” (hopefully there’s only a few of those), make a list, present it on our Friday meeting and have the list finalized.

 

I am not surprised that some ETOF runs are selected despite my explicit instructions not to. LLM can only approximately follow orders so that’s why manual review of their results is warranted. That’s especially the case for this analysis since I deliberately use a not-so-sophisticated version of LLM. I can’t use ChatGPT or any other commercial LLM both because of security and BNL won’t pay for those services. Since the LLM must run it on my desktop, it uses can’t be as smart as ChatGPT.  

 

If you don’t need ETOF, then I think it’s fine to keep those ETOF runs in. For those runs with no shift entries, the most correct way is to inspect the QA plots for all of them one-by-one. However, you have already ran your run-by-run QA code so perhaps if those runs are not flagged by the run-by-run QA, they’re fine? Since you are the QA worker for your run list, it’s your prerogative to decide what to do. Do you want to be safe and just reject all of them? Or do you think statistics is the most important consideration and you must keep them all? Or manually review all of them for a balanced run list? Any of those options work, but I think you need to decide and tell us on Friday. Afterall, you will work on those data, so you should be the one to make the call.

 

To summarize, you need to read the shift log for the 44 runs. If you believe there is a good enough justification for not including some runs in the bad run list, just take it out and present it on Friday. The remaining runs + what you got from run-by-run QA script will (hopefully) be the final official bad run list.

 

Best wishes,

Tommy

 

From: 李东升 <erl@mail.ustc.edu.cn>
Sent: Sunday, August 25, 2024 5:37 AM
To: Tsang, Chun Yuen <ctsang@bnl.gov>
Cc: ASHIK IKBAL <ashikhep@gmail.com>; star-qaboard-l@lists.bnl.gov
Subject: Re: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

 

Hi Tommy,

 

Thanks for your nice study. I never realized that LLM can come into the QA process, so it looks very interesting to me. 

 

I checked the 44 badruns listed in your slide 4. Actually I'm not familiar with the shift log and I don't fully understand what those texts really mean. Just based on my limited understanding, in general I agree most of these 44 badruns look suspicious, but a few of them maybe still look OK to me.

 

For example, if we look at run 22123007, the shift log says "BTOW: configuration failed -- STOP this run & restart", but they also say "Don't know if this is still what we are supposed to do". Then in run 22123012, they meet this problem again and they say "Was told not to stop the run for this error". So I guess this error message is not related to a major problem?

 

Another thing is, I find for some of these badruns, along with the error message, they say they mask out something. For instance, I notice run 22162015, 22168015 and 22166026 have this "mask" word. I don't quite understand this terminology, does that mean they hide the problem because it is not serious? 

 

Third thing is, although you mentioned these 44 badruns don't consider ETOF errors, I still find a couple of them related to ETOF, such as 22163028, 22163034, 22163035, 22163036, 22165026 and 22165035(I'm not sure whether they only have ETOF issues). Also I don't know whether ETOF is necessary or not for centrality purpose QA, actually I didn't include ETOF variables in the run-by-run analysis.

 

Best regards,

Dongsheng

 

 

-----原始邮件-----
发件人: "Tsang, Chun Yuen" <ctsang@bnl.gov>
发送时间: 2024-08-25 03:20:16 (星期日)
收件人: 李东升 <erl@mail.ustc.edu.cn>, "ASHIK IKBAL" <ashikhep@gmail.com>
抄送: "star-qaboard-l@lists.bnl.gov" <star-qaboard-l@lists.bnl.gov>
主题: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongsheng,

 

I tried to use AI to read the shift log. It’s a thing that we debuted at the collaboration meeting, and your data set is the first application of it so please double check my results.

 

The result shows that there are somewhere between 44-105 bad runs, depending on what you considered a bad run. Please see my full presentation here for details: https://drupal.star.bnl.gov/STAR/system/files/Using%20AI%20to%20read%20shift%20logs%20on%203GeV.pdf

 

The full list of shift log is also linked inside the presentation where you see that runs singled out by the algorithm indeed looks suspicious, However, are they bad enough that needs to be rejected? That is something we should discuss.

 

Best wishes,

Tommy

 

From: star-qaboard-l-request@lists.bnl.gov <star-qaboard-l-request@lists.bnl.gov> On Behalf Of ???
Sent: Friday, August 23, 2024 12:47 PM
To: ASHIK IKBAL <ashikhep@gmail.com>
Cc: star-qaboard-l@lists.bnl.gov; Tommy Tsang <tommytsang308@gmail.com>
Subject: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

 

Hi Ashik and Tommy,

 

Sorry I misunderstood. Please find the text file of all the runs in the attachment.

 

Best regards,

Dongsheng

-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-08-24 00:19:35 (星期六)
收件人: 李东升 <erl@mail.ustc.edu.cn>
抄送: star-qaboard-l@lists.bnl.gov, "Tommy Tsang" <tommytsang308@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongsheng,

 

Please send Tommy a list of ALL 3 GeV runs in text format. We have the bad run lists from your slides. We need a list of all the runs including good and bad runs. Can you send this in txt file format as I described before?

 

Best,

Ashik

 

On Fri, Aug 23, 2024 at 12:05 PM 李东升 <erl@mail.ustc.edu.cn> wrote:

Hi Ashik and Tommy,

 

Thanks for the message. Please find the badrun list file in the attachment, let me know if you have any further question.

 

Best regards,

Dongsheng

 

-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-08-23 23:15:22 (星期五)
收件人: 李东升 <erl@mail.ustc.edu.cn>
抄送: star-qaboard-l@lists.bnl.gov, "Tommy Tsang" <tommytsang308@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongsheng,

 

Thanks for your explanation. Now we need to do the shift log assessments of your found badrun lists for 3 GeV. 

 

Tommy (cc'ed above) is working on this. Could you please send Tommy a list of ALL 3 GeV runs in text format. The runs should be separated by line break, e.g. the file should look something like,

"""
20344007
20344008
20344009
....

 

Once the assessment is done, we can approve the badrun list.

 

Best,

Ashik

 

On Fri, Aug 9, 2024 at 12:47 PM 李东升 <star-qaboard-l@lists.bnl.gov> wrote:

Hi Ashik,

 

I checked the global rejection setting. I'm using the following command to run the package: 

 

python QA.py -i total.root -v tpname.list -o badrun.txt --element Au+Au --sNN 3 --batch --genPDF  -g 10 -gv tpname_glob.list

So it's 10 sigma rejection for the global case. And the global rejection is only done for Fxtmult.

 

Best regards,

Dongsheng

 

-----原始邮件-----
发件人: "李东升 (via star-qaboard-l Mailing List)" <star-qaboard-l@lists.bnl.gov>
发送时间: 2024-08-09 21:10:58 (星期五)
收件人: "Ray, Robert L" <ray@physics.utexas.edu>
抄送: star-qaboard-l <star-qaboard-l@lists.bnl.gov>, "ASHIK IKBAL" <ashikhep@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Ashik,

 

I would like to present QA results for the Run21 Au+Au collision at 3 GeV. The slides can be found at 

 

https://drupal.star.bnl.gov/STAR/system/files/QA_board_Run21_3GeV_dsli_240809.pdf

 

Thanks and best regards,

Dongsheng 

 

 

-----原始邮件-----
发件人: "Ray, Robert L" <ray@physics.utexas.edu>
发送时间: 2024-08-08 23:12:36 (星期四)
收件人: star-qaboard-l <star-qaboard-l@lists.bnl.gov>, "ASHIK IKBAL" <ashikhep@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT

Hi Ashik:

 

This week I will present the fast offline QA summary.

I will post my slides tomorrow morning prior to the meeting.

 

Thanks,

Lanny

---

From: star-qaboard-l-request@lists.bnl.gov <star-qaboard-l-request@lists.bnl.gov> on behalf of ASHIK IKBAL <ashikhep@gmail.com>
Sent: Thursday, August 8, 2024 10:04 AM
To: star-qaboard-l <star-qaboard-l@lists.bnl.gov>
Subject: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT

 

Dear All,

 

We will have our weekly QA board meeting this Friday
(09/Aug/2024) at noon 12 EDT.  Please let me know if you would like to present. 

 

The agenda can be found here: https://drupal.star.bnl.gov/STAR/blog/ashik/QA-Board

 

Zoom connection details:

Join ZoomGov Meeting

https://bnl.zoomgov.com/j/1618907709?pwd=YitQYzFUdWFYU1ppTjYxTjRYbDdsZz09

Meeting ID: 161 890 7709
Passcode: 775538

 

Thanks,

Ashik

 

 

--

Ashik Ikbal

Research Scientist

Kent State University

(stationed at Brookhaven National Laboratory)

Department Of Physics

Kent, OH 44242, USA

Contact no.s: +1 631-504-7647 (USA)

                      +91 8250216234 (India)

Alternate e-mails: asheikh2@kent.edu

                             ashik@rcf.rhic.bnl.gov

                             ashik.ikbal.sheikh@cern.ch

 

This message is from an external sender. Learn more about why this matters.

 

 

--

Ashik Ikbal

Research Scientist

Kent State University

(stationed at Brookhaven National Laboratory)

Department Of Physics

Kent, OH 44242, USA

Contact no.s: +1 631-504-7647 (USA)

                      +91 8250216234 (India)

Alternate e-mails: asheikh2@kent.edu

                             ashik@rcf.rhic.bnl.gov

                             ashik.ikbal.sheikh@cern.ch

 

--

Ashik Ikbal

Research Scientist

Kent State University

(stationed at Brookhaven National Laboratory)

Department Of Physics

Kent, OH 44242, USA

Contact no.s: +1 631-504-7647 (USA)

                      +91 8250216234 (India)

Alternate e-mails: asheikh2@kent.edu

                             ashik@rcf.rhic.bnl.gov

                             ashik.ikbal.sheikh@cern.ch

--

Ashik Ikbal

Research Scientist

Kent State University

(stationed at Brookhaven National Laboratory)

Department Of Physics

Kent, OH 44242, USA

Contact no.s: +1 631-504-7647 (USA)

                      +91 8250216234 (India)

Alternate e-mails: asheikh2@kent.edu

                             ashik@rcf.rhic.bnl.gov

                             ashik.ikbal.sheikh@cern.ch