star-fst-l AT lists.bnl.gov

Subject: Star-fst-l mailing list

List archive

Re: [Star-fst-l] FST with 9 Time Bins Diagnostic

From: videbaek <videbaek AT bnl.gov>
To: "Visser, Gerard" <gvisser AT indiana.edu>
Cc: L Star-fst <star-fst-l AT lists.bnl.gov>, "Ogawa, Akio" <akio AT bnl.gov>, jml AT bnl.gov, ichakaberia AT bnl.gov
Subject: Re: [Star-fst-l] FST with 9 Time Bins Diagnostic
Date: Wed, 21 Jun 2023 11:04:32 -0400

Hi

Good plots, thanks
I believe some of the variation could be due to change within a fill with slight increase of background like de-bounched beam . Yuo see this in many APD at runid 58. Can you check if this corresponds to first run in a fill.
Also I do not know the mapping , but there could be different conditions for the inner vs outer
sensors.

best F.

On 2023-06-21 10:51, Visser, Gerard wrote:

hi Ziyue,
Thanks - these plots are really excellent! Just to confirm, I
think you said it is not realistic to do this on some of last year's
data? That would be great to do if feasible but I understand it may
not be.
A simpler request: Do you think you could make on separate plots
for each APV of one interesting case (I propose
RDO2_ARM1_GROUP(PORT)0), a plot showing not only the fraction of
counts (of being the max) in middle timebin but also in the first and
third? I.e. Each plot will have three curves, they sum to 1. (And when
we have some 9 timebin data we can repeat this showing all 9.) I am
not sure if this will show us anything new but perhaps.
I still don't have any theory (other than vague "maybe it is beam
conditions changing"). But maybe someone has some better ideas?

Gerard

-------------------------

From: Ziyue Zhang <zzhan70 AT uic.edu>
Sent: Wednesday, June 21, 2023 10:21 AM
To: Visser, Gerard <gvisser AT indiana.edu>
Cc: L Star-fst <star-fst-l AT lists.bnl.gov>; Ogawa, Akio <akio AT bnl.gov>;
jml AT bnl.gov <jml AT bnl.gov>; ichakaberia AT bnl.gov <ichakaberia AT bnl.gov>
Subject: Re: [Star-fst-l] FST with 9 Time Bins Diagnostic

Dear all,
Please check the plots attached.
1. Only PHYS runs with run time over 30 mins are selected;
2. Most recent (when the script ran last night) 100 runs that satisfy
1. were included.
3. Update of pedestals, laser runs, and zero-field data taking period
were marked.
These plots are not conclusive, but can at least have a more direct
view of the drift of timing.
Best,
Ziyue

On Tue, Jun 20, 2023 at 7:10 PM Ziyue Zhang <zzhan70 AT uic.edu> wrote:

Hello all,

Firstly I'd like to point out the fact that, at the end of Run22
data taking [1] (shift->FST->Max_Time_Bin), some of the time bin
distributions of "sectors" (sector = combination of 4 APV) were also
not ideal, aka the middle bin did not have the highest fraction and
was significantly lower than one of the other 2 bins, but not low
enough to be negligible. Of course, at the beginning of Run22, the
timing was properly adjusted/optimized (at least on "sector" level).
This means such a feature - whether it is a problem or not, existed
last year - I hate to say it, but I'd like to naively conclude that
it is not new.

Secondly, given the steak is high, I'm doing some checks ASAP before
we proceed:
I shall check the fraction vs runIdx for every APV. The run period
will be limited to June 16 - present, due to:
1. Avoid any cases of LAT adjustment. June 15 was the last time LAT
was changed.
2. I'm doing this on [evp] in order to get a reasonable processing
speed since restoring daq files from hpss is really not working out
for me recently. Older files on [evp] are (automatically?) removed,
so this is as many as I can get.

Once I have fraction vs runIdx, we'll be able to tell the following:

1. Are all APVs timing drifting?
2. Are those drifting totally random or, for a long enough time
concentrated in one direction ?
3. How long, if fewer than 5 days (June 16-20) does it take to
develop?
4. Is it (partially?) recoverable with down time? (Can it fix
itself?)

The answer to those 4 questions should tell us whether it is worth
it to go for 9 time bins.

Best,
Ziyue

On Tue, Jun 20, 2023 at 6:33 PM Visser, Gerard
<gvisser AT indiana.edu> wrote:

Hi all,
First, I collect here some facts of deadtime for FST:

300 RS (32 us) fixed deadtime is really optimized (for 3 timebins),
Tonko and I concluded in Dec 2021. 290 is known to fail.

The formula for fixed deadtime setting for various Ntimebin is 90 +
int( Ntimebin/3 + 0.99999 )*3*70.
(The APV always reads groups of 3 timebins, and it takes 70 RS to
read a timebin from all 128 channels, and there's 90 RS of overhead
overall (independent of Ntimebin).)
So for 9 timebins we need to set 720 RS (77 us, 15% at 2 kHz).

The data volume per fiber for FST is 32124 bytes for 3-tb, 92604
bytes for 9-tb. (In general 4*(6+3*(3+16*(2+ "nwords/apv" ))),
referring to the link in p.s. here)
The data transfer bottleneck is the fiber data rate 206.7 x 10^6
bytes/s.
There will be a ceiling on FST rate (i.e. above this it will
suddenly go 100% dead) at 6.43 kHz for 3-tb, 2.23 kHz for 9-tb. If
running less than 90% of this I think the extra deadtime incurred by
data transfer is nearly zero.

Hopefully this answers the questions about deadtime for FST.

Should we make a switch to 9 timebins during the maintenance
day tomorrow, and keep it that way until at least some 12x12 data is
caught on thursday and at least N =?? (Ziyue??) hours of good data
is accumulated for a decent study of the supposed timing
instability?
Or should the whole plan be shelved until the implementation
of lower rates on forward detectors with a new trigger setup, as
Tonko was suggesting? That makes a lot of sense -- if the rate has
to be >2 kHz or if 15% deadtime is too much to tolerate -- provided
that will be implemented fairly soon. We think there's enough cause
for concern about FST now to justify some cost of this test.
I should be ready to do this sometime tomorrow afternoon,
i.e. will have 9-timebin firmware for FST compiled. This should be
easy but it's been a while so I have to remind myself on some
details still. But the question of when to do this test, is not for
me to say.

Sincerely,

Gerard

p.s. FYI there is an old reference on data size and deadtime
settings here

https://drupal.star.bnl.gov/STAR/blog/gvisser/ist-and-gmt-fixed-busy-time-data-size-and-limiting-rate-function-ntimebins

. The deadtime settings therein are not fully optimized, and are
superseded by what I give above.

-------------------------

From: Ye, Zhenyu <yezhenyu AT uic.edu>
Sent: Tuesday, June 20, 2023 12:50 PM
To: Visser, Gerard <gvisser AT indiana.edu>
Cc: Tonko Ljubicic <tonko AT bnl.gov>; Star-fst L
<star-fst-l AT lists.bnl.gov>
Subject: Re: [Star-fst-l] FST with 9 Time Bins Diagnostic

Hi Gerard

In Run 22, we ran 9 time bin in the beginning, and switched to 3
time bin on 12/20/2021 after we tuned the latency
Below is the logbook entry left by Tonko

13:18

FST Gerard says:

We have downloaded arm_r108_fst_3tb.bin to all the RDO's of FST.
This sets 3 timebin readout (was 9), with 1 triple-timebin
triggering of the APV chip (was 3), and skips the APV's 8-11 and
20-23 which do not exist for FST.

On Jun 20, 2023, at 5:39 PM, Visser, Gerard <gvisser AT indiana.edu>
wrote:

And there's another complication, gadzooks! I do not see a firmware
file for FST 9-timebin, so I will have to prepare one today. That
shouldn't be a problem. But did we not at all run 9-timebin when we
first set up the FST?? Zhenyu do you recall? This is surprising me.
(It's possible we ran arm_r106_ist_9tb.bin but that will cause other
problems now so it's not the thing to do. I will have to prepare a
new file.)

Tonko thanks for reminder on the pedestals! Truthfully I might
have forgotten...

Gerard

-------------------------

From: Tonko Ljubicic <tonko AT bnl.gov>
Sent: Tuesday, June 20, 2023 11:29 AM
To: Visser, Gerard <gvisser AT indiana.edu>
Cc: Ziyue Zhang <zzhan70 AT uic.edu>; Star-fst L
<star-fst-l AT lists.bnl.gov>
Subject: Re: [Star-fst-l] FST with 9 Time Bins Diagnostic

Note another complication:

You _must_ take a pedestal run after you switched to 9 timebins
and then again once you switch back to 3.

On Tue, Jun 20, 2023 at 5:13 PM Visser, Gerard
<gvisser AT indiana.edu> wrote:

Hi Ziyue.

Roughly speaking the fixed deadtime will have to be increased by a

factor of three, if it is really optimized right now. For the record
Tonko says that the fixed deadtime setting right now is 300 (RHIC
strobe tics). I am trying to find my notes or recalculate the
required deadtime change for going from 3 timebin to 9 timebin APV
readout, I'll reply on that soon.

The higher data volume of 9-timebin will also impose a hard

ceiling on the rate. This was discussed a long time ago on FST
mailing list; likewise I'm going to have to do some digging to find
the number and will reply when I can.

Flemming (I think it was) asked for a brief summary of what the

apparent problem that we want to check on with this 9 timebin
running to be sent to starops (or triggerboard?). I agree that is a
very good idea. It would really be beneficial to have some statement
about any real problem observed in the data, beyond merely an
observation that the 'max timebin' plots seem unstable. It would
also be good to have some statement about if this is definitely
different than last year, or definitely different than IST.

I think that 9-timebin running, which is the FST way of "pre/post"

running, should give a confirmation that the timing is really
properly set, and it should give more insights into any instability
in the timing if there is an instability during the time period that
we run 9-tb. Since there will be a cost to such running we will have
to be careful to run only as long as really needed.

I can't think of any mechanism which would cause the timing of APV

sampling to drift. A particular broken FEE or broken ARM module
perhaps, but the observation is more widespread than that, if I
understand right. I still mainly suspect that the 3-timebin 'max
timebin' plot is influenced by something else than a drift of the
sampling time - background or some difference between triggers or
the pulse shapes are just not constant enough or something. None of
which (I think) would really directly be a problem for FST data
quality. Of course I may be completely wrong here. It would be
excellent to have some kind of tracking result from current FST data
to try to judge if it is working correctly or not, but I appreciate
that is a huge ask.

Gerard

________________________________
From: Star-fst-l <star-fst-l-bounces AT lists.bnl.gov> on behalf of

Ziyue Zhang <zzhan70 AT uic.edu>

Sent: Tuesday, June 20, 2023 10:44 AM
To: Star-fst L <star-fst-l AT lists.bnl.gov>
Subject: [Star-fst-l] FST with 9 Time Bins Diagnostic

Hello all,

We discussed the FST timing drift issue in this morning's (June

20) meeting and proposed to switch to 9 time-bin temporarily for
diagnostic purposes. In the daily ops meeting, it was brought up
that this will influence the deadtime of many triggers, and
therefore it does not come for free.

Gerard, I only have a general idea on the trigger dead time topic.

Do you have any comments on this?

Therefore we should estimate how long this diagnostic is going to

take and discuss what exactly we need to look at, then we can switch
to 9 time-bin and do the diagnostic as efficiently as possible.

Best,
Ziyue

_______________________________________________
Star-fst-l mailing list
Star-fst-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/star-fst-l

_______________________________________________
Star-fst-l mailing list
Star-fst-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/star-fst-l

_______________________________________________
Star-fst-l mailing list
Star-fst-l AT lists.bnl.gov

https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.bnl.gov%2Fmailman%2Flistinfo%2Fstar-fst-l&data=05%7C01%7Czzhan70%40groute.uic.edu%7Cc87b15e9ba024e5c3bde08db71de600b%7Ce202cd477a564baa99e3e3b71a7c77dd%7C0%7C0%7C638228972171481027%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4KXoAnDlHMrCTkwR6Npl38gOAgqRmoD7IYpLm88HQv4%3D&reserved=0

[2]

Links:
------
[1] https://online.star.bnl.gov/RTS/JEVP/protected/PHP/jevpViewer.php?run=23107048
[2] https://lists.bnl.gov/mailman/listinfo/star-fst-l
_______________________________________________
Star-fst-l mailing list
Star-fst-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/star-fst-l

--
Flemming Videbaek
senior scientist, emeritus
videbaek @ bnl.gov
Brookhaven National Lab
Physics Department
Bldg 510D
Upton, NY 11973

phone: 631-344-4106
cell : 631-681-1596

Re: [Star-fst-l] FST with 9 Time Bins Diagnostic , (continued)