STAR HardProbes PWG

["Ma, Rongrong" <marr AT bnl.gov>]

Hello Nihar

This part of discussion between you and Isaac got me thinking. I also do not think it is fair to include ALL uncertainties present in data analysis in the closure test since some of them do not apply. For example, uncertainties in the response matrix (tracking efficiency and tower gain uncertainties) and prior should not be included since they are known precisely in the closure test. On the other hand, uncertainties due to different unfolding method (Bayesian vs. SVD), different number of iterations, and f_ME, are legitimate and should be included. If we do not make this distinction, the closure is likely inflated. 

By the way, the section of closure test in Au+Au includes only three lines of text and one case of jet kinematics. Maybe I missed the main discussion? If not, it will be helpful to expand this section. Also, it is not clear whether background fluctuation is included in the text. 

Thanks. 

Best

Rongrong

On Jul 13, 2024, at 1:49 PM, Nihar Sahoo <nihar AT rcf.rhic.bnl.gov> wrote:

Analysis Note, Fig. 14. Thanks for addressing this point.
Analysis Note, Fig. 23. I disagree. The approach I am familiar with is
to judge the closure test on its own terms, with statistical
uncertainties on the ratio. When the ratio is consistent within the
standard error, this is 'good' closure. Of course the edges will
always be fuzzy, but that is also true of your approach which uses the
systematics as the benchmark -- e.g. is 0.1% outside of the systematic
uncertainty 'bad'? 1%? ... The only thing that has changed is that the
benchmark is now the systematic rather than the statistical
uncertainty, so it is not true that it is the "only quantitative
standard". You say that "Given the finite statistics of the data,
which is mimicked in this test, some deviation from unity must occur",
but that is precisely what the statistical uncertainty should cover,
and in addition the non-closure here is in bins with good statistics.
I'm still not convinced that this is the proper approach. If you have
a reference which demonstrates the argument, I think that would be
very helpful for me.

This approach has been used by previous STAR and ALICE jet quenching papers, among others.

Unfolding by its nature entails systematic uncertainty. If the response matrix were known precisely, with arbitrarily high statistical precision, it could simply be inverted numerically to multiply the smeared data, resulting in the precise corrected distribution. But this doesn’t work in practice: we need to use a regularized (truncated) inversion, which is basically an assumption of smoothness, to avoid statistical noise that would generate arbitrarily large variance in the inverted matrix and render the result of the multiplication uninterpretable. Another name for this: regularized unfolding. So unfolding is never purely statistical.

Then to rephrase your proposal: compare the results of unfolding only to the magnitude of systematic uncertainty due to regularization in order to gauge its success. This we think is not a good basis for rejecting an analysis – why elevate this systematic uncertainty above all others?

The only reasonable standard we think is to ask whether unfolding results in deviations from the truth that are larger than the known uncertainties in the problem, i.e. the total systematic uncertainty. If closure is much better than that then there is no point of concern; if it is larger then the solution should not be considered to be valid.

We do not see a meaningful alternative to this procedure.