Hi Ashik,
I think slide 14 from Dongsheng’s presentation is the final bad run list for Run 21. Can you make that official please? I know that you need to upload the list somewhere. We have already discussed this in the QA meeting, and I don’t see
any show stopper.
Best wishes,
Tommy
From:
李东升 <erl@mail.ustc.edu.cn>
Sent: Friday, September 27, 2024 1:09 PM
To: 李东升 (via star-qaboard-l Mailing List) <star-qaboard-l@lists.bnl.gov>
Cc: ASHIK IKBAL <ashikhep@gmail.com>; Tsang, Chun Yuen <ctsang@bnl.gov>
Subject: Re: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)
Hi all,
Thanks to the helpful discussion in the QA meeting, we have decided which badruns from AI diagnosis to be false positive. So I update the total badrun list in the following slides:
https://drupal.star.bnl.gov/STAR/system/files/dsli_Run21_3GeV_total_list_240926_v1.pdf
Best regards,
Dongsheng
-----原始邮件-----
发件人: "李东升
(via star-qaboard-l Mailing List)" <star-qaboard-l@lists.bnl.gov>
发送时间: 2024-09-26 13:51:46 (星期四)
收件人: "ASHIK IKBAL" <ashikhep@gmail.com>
抄送: "Tsang, Chun Yuen" <ctsang@bnl.gov>, "star-qaboard-l@lists.bnl.gov"
<star-qaboard-l@lists.bnl.gov>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng
Li)
Hi Ashik and all,
Please find my slides for discussion at
https://drupal.star.bnl.gov/STAR/system/files/dsli_Run21_3GeV_total_list_240926.pdf
Best regards,
Dongsheng
-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-09-25 23:46:00 (星期三)
收件人:
李东升 <erl@mail.ustc.edu.cn>
抄送: "Tsang, Chun Yuen" <ctsang@bnl.gov>, "star-qaboard-l@lists.bnl.gov"
<star-qaboard-l@lists.bnl.gov>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng
Li)
Hi Dongsheng,
I think you did a good job with the run-by-run QA so far. Just one last step is left as explained by Tommy. Thanks to Tommy for explaining this.
Could you please do the checks (explained by Tommy) and present the combined badrun list on the QA board meeting.
Hi Tommy,
I also missed the long mail on Sep.10th from you. So you plan to use AI+run-by-run QA, This answers part of my question.
Best regards
Dongsheng
-----原始邮件-----
发件人: "Tsang, Chun Yuen" <ctsang@bnl.gov>
发送时间: 2024-09-25 22:34:49 (星期三)
收件人:
李东升 <erl@mail.ustc.edu.cn>
抄送: "ASHIK IKBAL" <ashikhep@gmail.com>,
"star-qaboard-l@lists.bnl.gov" <star-qaboard-l@lists.bnl.gov>
主题: RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng
Li)
Hi Dongsheng,
To add on to Ashik’s email, the following message is detached from the email chain, so I am sending it again just in case you missed that.
Best wishes,
Tommy
Hi Dongsheng,
Sorry for the late reply. I missed your email.
I think the agreement is that QA worker probably needs to manually review those 44 bad runs. AI can only narrow down the run list so you don’t need
to go through all ~500 runs, but people still need to do the final inspection.
If you think the shift log entries for some of those 44 runs are not that bad, I suggest that you take those runs out, make a list of runs that you
“recovered” (hopefully there’s only a few of those), make a list, present it on our Friday meeting and have the list finalized.
I am not surprised that some ETOF runs are selected despite my explicit instructions not to. LLM can only approximately follow orders so that’s why
manual review of their results is warranted. That’s especially the case for this analysis since I deliberately use a not-so-sophisticated version of LLM. I can’t use ChatGPT or any other commercial LLM both because of security and BNL won’t pay for those services.
Since the LLM must run it on my desktop, it uses can’t be as smart as ChatGPT.
If you don’t need ETOF, then I think it’s fine to keep those ETOF runs in. For those runs with no shift entries, the most correct way is to inspect
the QA plots for all of them one-by-one. However, you have already ran your run-by-run QA code so perhaps if those runs are not flagged by the run-by-run QA, they’re fine? Since you are the QA worker for your run list, it’s your prerogative to decide what
to do. Do you want to be safe and just reject all of them? Or do you think statistics is the most important consideration and you must keep them all? Or manually review all of them for a balanced run list? Any of those options work, but I think you need to
decide and tell us on Friday. Afterall, you will work on those data, so you should be the one to make the call.
To summarize, you need to read the shift log for the 44 runs. If you believe there is a good enough justification for not including some runs in the
bad run list, just take it out and present it on Friday. The remaining runs + what you got from run-by-run QA script will (hopefully) be the final official bad run list.
Best wishes,
Tommy
Hi Tommy,
Thanks for your nice study. I never realized that LLM can come into the QA process, so it looks very interesting to me.
I checked the 44 badruns listed in your slide 4. Actually I'm not familiar with the shift log and I don't fully understand what those texts really mean. Just based on my limited understanding, in general I agree most of these
44 badruns look suspicious, but a few of them maybe still look OK to me.
For example, if we look at run 22123007, the shift log says "BTOW: configuration failed -- STOP this run & restart", but they also say "Don't
know if this is still what we are supposed to do". Then in run 22123012, they meet this problem again and they say "Was told not to stop the run for this error". So I guess
this error message is not related to a major problem?
Another thing is, I find for some of these badruns, along with the error message, they say they mask out something. For instance, I notice run 22162015, 22168015 and 22166026 have this "mask"
word. I don't quite understand this terminology, does that mean they hide the problem because it is not serious?
Third thing is, although you mentioned these 44 badruns don't consider ETOF errors, I still find a couple of them related to ETOF, such as 22163028, 22163034, 22163035, 22163036, 22165026 and 22165035(I'm not sure whether they
only have ETOF issues). Also I don't know whether ETOF is necessary or not for centrality purpose QA, actually I didn't include ETOF variables in the run-by-run analysis.
-----原始邮件-----
发件人: "Tsang, Chun Yuen" <ctsang@bnl.gov>
发送时间: 2024-08-25 03:20:16 (星期日)
收件人:
李东升 <erl@mail.ustc.edu.cn>,
"ASHIK IKBAL" <ashikhep@gmail.com>
抄送: "star-qaboard-l@lists.bnl.gov"
<star-qaboard-l@lists.bnl.gov>
主题:
RE: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng Li)
Hi Dongsheng,
I tried to use AI to read the shift log. It’s a thing that we debuted at the collaboration meeting, and your data set is the first application of
it so please double check my results.
The result shows that there are somewhere between 44-105 bad runs, depending on what you considered a bad run. Please see my full presentation here
for details:
https://drupal.star.bnl.gov/STAR/system/files/Using%20AI%20to%20read%20shift%20logs%20on%203GeV.pdf
The full list of shift log is also linked inside the presentation where you see that runs singled out by the algorithm indeed looks suspicious, However,
are they bad enough that needs to be rejected? That is something we should discuss.
Best wishes,
Tommy
Hi Ashik and Tommy,
Sorry I misunderstood. Please find the text file of all the runs in the attachment.
Best regards,
Dongsheng
-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-08-24 00:19:35 (星期六)
收件人:
李东升 <erl@mail.ustc.edu.cn>
抄送:
star-qaboard-l@lists.bnl.gov, "Tommy Tsang" <tommytsang308@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng
Li)
Hi Dongsheng,
Please send Tommy a list of ALL 3 GeV runs in text format. We have the bad run lists from your slides. We need a list of all the runs including good
and bad runs. Can you send this in txt file format as I described before?
Hi Ashik and Tommy,
Thanks for the message. Please find the badrun list file in the attachment, let me know if you have any further question.
Best regards,
Dongsheng
-----原始邮件-----
发件人: "ASHIK IKBAL" <ashikhep@gmail.com>
发送时间: 2024-08-23 23:15:22 (星期五)
收件人:
李东升 <erl@mail.ustc.edu.cn>
抄送:
star-qaboard-l@lists.bnl.gov, "Tommy Tsang" <tommytsang308@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng
Li)
Hi Dongsheng,
Thanks for your explanation. Now we need to do the shift log assessments of your found badrun lists for 3 GeV.
Tommy (cc'ed above) is working on this. Could you please send Tommy a list of ALL 3 GeV runs in text format. The runs should be separated by line
break, e.g. the file should look something like,
"""
20344007
20344008
20344009
....
Once the assessment is done, we can approve the badrun list.
Hi Ashik,
I checked the global rejection setting. I'm using the following command to run the package:
python QA.py -i total.root -v tpname.list -o badrun.txt --element Au+Au --sNN 3 --batch --genPDF -g 10 -gv tpname_glob.list
So it's 10 sigma rejection for the global case. And the global rejection is only done for Fxtmult.
Best regards,
Dongsheng
-----原始邮件-----
发件人: "李东升
(via star-qaboard-l Mailing List)" <star-qaboard-l@lists.bnl.gov>
发送时间: 2024-08-09 21:10:58 (星期五)
收件人: "Ray, Robert L" <ray@physics.utexas.edu>
抄送: star-qaboard-l <star-qaboard-l@lists.bnl.gov>,
"ASHIK IKBAL" <ashikhep@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT (Dongsheng
Li)
Hi Ashik,
I would like to present QA results for the Run21 Au+Au collision at 3 GeV. The slides can be found at
https://drupal.star.bnl.gov/STAR/system/files/QA_board_Run21_3GeV_dsli_240809.pdf
Thanks and best regards,
Dongsheng
-----原始邮件-----
发件人: "Ray, Robert L" <ray@physics.utexas.edu>
发送时间: 2024-08-08 23:12:36 (星期四)
收件人: star-qaboard-l <star-qaboard-l@lists.bnl.gov>,
"ASHIK IKBAL" <ashikhep@gmail.com>
主题: Re: [[STAR-QAboard] ] Weekly QA Board Meeting - 09/Aug/2024 at noon 12 EDT
This week I will present the fast offline QA summary.
I will post my slides tomorrow morning prior to the meeting.
Dear All,
We will have our weekly QA board meeting this Friday
(09/Aug/2024) at noon 12 EDT. Please let me know if you would like to present.
--
(stationed at Brookhaven National Laboratory)
Contact no.s: +1 631-504-7647 (USA)
--
(stationed at Brookhaven National Laboratory)
Contact no.s: +1 631-504-7647 (USA)
--
(stationed at Brookhaven National Laboratory)
Contact no.s: +1 631-504-7647 (USA)
--
(stationed at Brookhaven National Laboratory)
Contact no.s: +1 631-504-7647 (USA)
|
