STAR QA Board

["Tsang, Chun Yuen" <ctsang@bnl.gov>]

Dear Dongsheng,

 

I see your confusion. It’s a combination of both algorithms, as in, the bad run list is the sum of run-by-run QA and reviewing the online shift log, therefore the run-by-run QA part is not discarded. Here’s what should happen without this fancy AI method,

 

1. Use run-by-run QA algorithm to identify bad runs like you did.
2. Read the shift logs, supposedly for all runs. Identify runs with dubious shift logs, and then add those dubious runs to bad run list you got from step 1.

 

We don’t take bad run out of the list. We just add more runs. The only thing that AI does is that on step 2, you only need to manually review shift logs selected by the algorithm rather than reading them all. Of course it should be mentioned that I don’t think people really do read every single shift logs in previous run-by-run QA due to time constraint or other reasons, but ideally one really should really read shift logs of all runs and use their judgement to append more bad runs onto bad run list given by run-by-run QA algorithm.

 

You are correct that there absolutely are “some badruns not identified by AI”, hence it is used in conjunction with the run-by-run QA code. I hope that AI could cut down on time needed to manually review shift logs by cutting down the list of runs people need to manually review, but at the end of the day, either with or without AI, you still need to read the shift log and tell us what other runs look suspicious and report it to us for bad run list to be finalized.

 

I am sorry I did not make this clear earlier that run-by-run QA is a two-step process. Please compile a list of runs with dubious shift logs to finalize the bad run list. You are free to ignore AI’s result, but I hope it can be use as an assistant to guide to help you focus on some particular runs, rather than needing to read every single shift log entry.

 

Best wishes,

Tommy

 

From: 李东升 <erl@mail.ustc.edu.cn>
Sent: Wednesday, September 25, 2024 11:14 AM
To: ASHIK IKBAL <ashikhep@gmail.com>
Cc: Tsang, Chun Yuen <ctsang@bnl.gov>; star-qaboard-l@lists.bnl.gov
Subject: Re: Re: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

 

Hi Ashik and Tommy,

 

Sorry I missed Tommy's previous email. Actually when I got those 44 badruns, I immediately checked their shift log manually and I think you already see my email. I sent it out right on the next day I receive Tommy's 44 badruns. In the email, I already listed out some badruns that look not so suspicious to me. They could be OK, but you should be aware that I'm not sure I can understand the shift log information correctly so I suppose Tommy or someone else needs to do double check on it. 

 

I'm also not so clear about the strategy you plan to take for this badrun study. I thought it should be like this, first someone report run-by-run QA, then we check the shift log to see if there is really something wrong in those runs, after that we finalize the badrun list. I'm just a freshman, and maybe I mis-understood the procedure. Could you explain a little bit more about the strategy you plan to take? Now it seems you totally ignore the run-by-run QA part, and we only look at the shift log with AI. Ai is a good and funcy method, but I actually have some doubt about it. If there's a false positive in the AI method, we can identify them manually, which is good. However what if there were some badruns not identified by AI? At least, some runs even don't have any shift log, so AI is surely not so useful for them. Right now I don't see the position of run-by-run QA with this AI method, though it could serve as a tool to identify badruns reliably. I understand people's concern about this dataset. I also worked on flow analysis with it myself. But I'm a bit confused how I can do some help?

 

Best regards,

Dongsheng

-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-09-25 22:30:27 (星期三)
收件人: "Tsang, Chun Yuen" <ctsang@bnl.gov>
抄送: 李东升 <erl@mail.ustc.edu.cn>, "star-qaboard-l@lists.bnl.gov" <star-qaboard-l@lists.bnl.gov>
主题: Re: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongshen,

 

I assume that you got the list of the questionable 44 runs after shift log assessment by Tommy. Those 44 runs need to be checked manually as it was explained by Tommy in this email thread before.

 

This dataset is of interest to the centrality group and many of the PWGs for QM25. Could you please let us know how it is going?

 

Best,

Ashik

 

On Tue, Sep 10, 2024 at 5:31 PM Tsang, Chun Yuen <ctsang@bnl.gov> wrote:

Hi Dongsheng,

 

About “masking”, that’s when a sector gives you repeated errors and restarting the runs multiple times didn’t resolve the issue. Instead of rebooting the device, you tell the DAQ to ignore the device so there won’t be data from a particular sector if it’s masked out.

 

It’s possible that data is still usable despite some sectors being masked out, but if you want to be safe, you should remove those runs.

 

Best wishes,

Tommy

 

From: 李东升 <erl@mail.ustc.edu.cn>
Sent: Sunday, August 25, 2024 5:37 AM
To: Tsang, Chun Yuen <ctsang@bnl.gov>
Cc: ASHIK IKBAL <ashikhep@gmail.com>; star-qaboard-l@lists.bnl.gov
Subject: Re: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

 

Hi Tommy,

 

Thanks for your nice study. I never realized that LLM can come into the QA process, so it looks very interesting to me. 

 

I checked the 44 badruns listed in your slide 4. Actually I'm not familiar with the shift log and I don't fully understand what those texts really mean. Just based on my limited understanding, in general I agree most of these 44 badruns look suspicious, but a few of them maybe still look OK to me.

 

For example, if we look at run 22123007, the shift log says "BTOW: configuration failed -- STOP this run & restart", but they also say "Don't know if this is still what we are supposed to do". Then in run 22123012, they meet this problem again and they say "Was told not to stop the run for this error". So I guess this error message is not related to a major problem?

 

Another thing is, I find for some of these badruns, along with the error message, they say they mask out something. For instance, I notice run 22162015, 22168015 and 22166026 have this "mask" word. I don't quite understand this terminology, does that mean they hide the problem because it is not serious? 

 

Third thing is, although you mentioned these 44 badruns don't consider ETOF errors, I still find a couple of them related to ETOF, such as 22163028, 22163034, 22163035, 22163036, 22165026 and 22165035(I'm not sure whether they only have ETOF issues). Also I don't know whether ETOF is necessary or not for centrality purpose QA, actually I didn't include ETOF variables in the run-by-run analysis.

 

Best regards,

Dongsheng

 

 

-----原始邮件-----
发件人: "Tsang, Chun Yuen" <ctsang@bnl.gov>
发送时间: 2024-08-25 03:20:16 (星期日)
收件人: 李东升 <erl@mail.ustc.edu.cn>, "ASHIK IKBAL" <ashikhep@gmail.com>
抄送: "star-qaboard-l@lists.bnl.gov" <star-qaboard-l@lists.bnl.gov>
主题: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongsheng,

 

I tried to use AI to read the shift log. It’s a thing that we debuted at the collaboration meeting, and your data set is the first application of it so please double check my results.

 

The result shows that there are somewhere between 44-105 bad runs, depending on what you considered a bad run. Please see my full presentation here for details: https://drupal.star.bnl.gov/STAR/system/files/Using%20AI%20to%20read%20shift%20logs%20on%203GeV.pdf

 

The full list of shift log is also linked inside the presentation where you see that runs singled out by the algorithm indeed looks suspicious, However, are they bad enough that needs to be rejected? That is something we should discuss.

 

Best wishes,

Tommy

 

From: star-qaboard-l-request@lists.bnl.gov <star-qaboard-l-request@lists.bnl.gov> On Behalf Of ???
Sent: Friday, August 23, 2024 12:47 PM
To: ASHIK IKBAL <ashikhep@gmail.com>
Cc: star-qaboard-l@lists.bnl.gov; Tommy Tsang <tommytsang308@gmail.com>
Subject: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

 

Hi Ashik and Tommy,

 

Sorry I misunderstood. Please find the text file of all the runs in the attachment.

 

Best regards,

Dongsheng

-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-08-24 00:19:35 (星期六)
收件人: 李东升 <erl@mail.ustc.edu.cn>
抄送: star-qaboard-l@lists.bnl.gov, "Tommy Tsang" <tommytsang308@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongsheng,

 

Please send Tommy a list of ALL 3 GeV runs in text format. We have the bad run lists from your slides. We need a list of all the runs including good and bad runs. Can you send this in txt file format as I described before?

 

Best,

Ashik

 

On Fri, Aug 23, 2024 at 12:05 PM 李东升 <erl@mail.ustc.edu.cn> wrote:

Hi Ashik and Tommy,

 

Thanks for the message. Please find the badrun list file in the attachment, let me know if you have any further question.

 

Best regards,

Dongsheng

 

-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-08-23 23:15:22 (星期五)
收件人: 李东升 <erl@mail.ustc.edu.cn>
抄送: star-qaboard-l@lists.bnl.gov, "Tommy Tsang" <tommytsang308@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Dongsheng,

 

Thanks for your explanation. Now we need to do the shift log assessments of your found badrun lists for 3 GeV. 

 

Tommy (cc'ed above) is working on this. Could you please send Tommy a list of ALL 3 GeV runs in text format. The runs should be separated by line break, e.g. the file should look something like,

"""
20344007
20344008
20344009
....

 

Once the assessment is done, we can approve the badrun list.

 

Best,

Ashik

 

On Fri, Aug 9, 2024 at 12:47 PM 李东升 <star-qaboard-l@lists.bnl.gov> wrote:

Hi Ashik,

 

I checked the global rejection setting. I'm using the following command to run the package: 

 

python QA.py -i total.root -v tpname.list -o badrun.txt --element Au+Au --sNN 3 --batch --genPDF  -g 10 -gv tpname_glob.list

So it's 10 sigma rejection for the global case. And the global rejection is only done for Fxtmult.

 

Best regards,

Dongsheng

 

-----原始邮件-----
发件人: "李东升 (via star-qaboard-l Mailing List)" <star-qaboard-l@lists.bnl.gov>
发送时间: 2024-08-09 21:10:58 (星期五)
收件人: "Ray, Robert L" <ray@physics.utexas.edu>
抄送: star-qaboard-l <star-qaboard-l@lists.bnl.gov>, "ASHIK IKBAL" <ashikhep@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)

Hi Ashik,

 

I would like to present QA results for the Run21 Au+Au collision at 3 GeV. The slides can be found at 

 

https://drupal.star.bnl.gov/STAR/system/files/QA_board_Run21_3GeV_dsli_240809.pdf

 

Thanks and best regards,

Dongsheng 

 

 

-----原始邮件-----
发件人: "Ray, Robert L" <ray@physics.utexas.edu>
发送时间: 2024-08-08 23:12:36 (星期四)
收件人: star-qaboard-l <star-qaboard-l@lists.bnl.gov>, "ASHIK IKBAL" <ashikhep@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT

Hi Ashik:

 

This week I will present the fast offline QA summary.

I will post my slides tomorrow morning prior to the meeting.

 

Thanks,

Lanny

---

From: star-qaboard-l-request@lists.bnl.gov <star-qaboard-l-request@lists.bnl.gov> on behalf of ASHIK IKBAL <ashikhep@gmail.com>
Sent: Thursday, August 8, 2024 10:04 AM
To: star-qaboard-l <star-qaboard-l@lists.bnl.gov>
Subject: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT

 

Dear All,

 

We will have our weekly QA board meeting this Friday
(09/Aug/2024) at noon 12 EDT.  Please let me know if you would like to present. 

 

The agenda can be found here: https://drupal.star.bnl.gov/STAR/blog/ashik/QA-Board

 

Zoom connection details:

Join ZoomGov Meeting

https://bnl.zoomgov.com/j/1618907709?pwd=YitQYzFUdWFYU1ppTjYxTjRYbDdsZz09

Meeting ID: 161 890 7709
Passcode: 775538

 

Thanks,

Ashik

 

 

--

Ashik Ikbal

Research Scientist

Kent State University

(stationed at Brookhaven National Laboratory)

Department Of Physics

Kent, OH 44242, USA

Contact no.s: +1 631-504-7647 (USA)

                      +91 8250216234 (India)

Alternate e-mails: asheikh2@kent.edu

                             ashik@rcf.rhic.bnl.gov

                             ashik.ikbal.sheikh@cern.ch

 

This message is from an external sender. Learn more about why this matters.

 

 

--

Ashik Ikbal

Research Scientist

Kent State University

(stationed at Brookhaven National Laboratory)

Department Of Physics

Kent, OH 44242, USA

Contact no.s: +1 631-504-7647 (USA)

                      +91 8250216234 (India)

Alternate e-mails: asheikh2@kent.edu

                             ashik@rcf.rhic.bnl.gov

                             ashik.ikbal.sheikh@cern.ch

 

--

Ashik Ikbal

Research Scientist

Kent State University

(stationed at Brookhaven National Laboratory)

Department Of Physics

Kent, OH 44242, USA

Contact no.s: +1 631-504-7647 (USA)

                      +91 8250216234 (India)

Alternate e-mails: asheikh2@kent.edu

                             ashik@rcf.rhic.bnl.gov

                             ashik.ikbal.sheikh@cern.ch

 

--

Ashik Ikbal

Research Scientist

Kent State University

(stationed at Brookhaven National Laboratory)

Department Of Physics

Kent, OH 44242, USA

Contact no.s: +1 631-504-7647 (USA)

                      +91 8250216234 (India)

Alternate e-mails: asheikh2@kent.edu

                             ashik@rcf.rhic.bnl.gov

                             ashik.ikbal.sheikh@cern.ch