Star-fst-l mailing list

[Ziyue Zhang <zzhan70 AT uic.edu>]

Dear all,

Please check the plots attached.

1. Only PHYS runs with run time over 30 mins are selected;

2. Most recent (when the script ran last night) 100 runs that satisfy 1. were included.

3. Update of pedestals, laser runs, and zero-field data taking period were marked.

These plots are not conclusive, but can at least have a more direct view of the drift of timing.

Best,

Ziyue

On Tue, Jun 20, 2023 at 7:10 PM Ziyue Zhang <zzhan70 AT uic.edu> wrote:

Hello all,

Firstly I'd like to point out the fact that, at the end of Run22 data taking (shift->FST->Max_Time_Bin), some of the time bin distributions of "sectors" (sector = combination of 4 APV) were also not ideal, aka the middle bin did not have the highest fraction and was significantly lower than one of the other 2 bins, but not low enough to be negligible. Of course, at the beginning of Run22, the timing was properly adjusted/optimized (at least on "sector" level). This means such a feature - whether it is a problem or not, existed last year - I hate to say it, but I'd like to naively conclude that it is not new.

Secondly, given the steak is high, I'm doing some checks ASAP before we proceed:

I shall check the fraction vs runIdx for every APV. The run period will be limited to June 16 - present, due to:

1. Avoid any cases of LAT adjustment. June 15 was the last time LAT was changed.

2. I'm doing this on [evp] in order to get a reasonable processing speed since restoring daq files from hpss is really not working out for me recently. Older files on [evp] are (automatically?) removed, so this is as many as I can get.

Once I have fraction vs runIdx, we'll be able to tell the following:

1. Are all APVs timing drifting?

2. Are those drifting totally random or, for a long enough time concentrated in one direction ?

3. How long, if fewer than 5 days (June 16-20) does it take to develop?

4. Is it (partially?) recoverable with down time? (Can it fix itself?)

The answer to those 4 questions should tell us whether it is worth it to go for 9 time bins.

Best,

Ziyue

On Tue, Jun 20, 2023 at 6:33 PM Visser, Gerard <gvisser AT indiana.edu> wrote:

Hi all,

      First, I collect here some facts of deadtime for FST:

300 RS (32 us) fixed deadtime is really optimized (for 3 timebins), Tonko and I concluded in Dec 2021. 290 is known to fail.

The formula for fixed deadtime setting for various Ntimebin is 90 + int( Ntimebin/3 + 0.99999 )*3*70.

(The APV always reads groups of 3 timebins, and it takes 70 RS to read a timebin from all 128 channels, and there's 90 RS of overhead overall (independent of Ntimebin).)

So for 9 timebins we need to set 720 RS (77 us, 15% at 2 kHz).

The data volume per fiber for FST is 32124 bytes for 3-tb, 92604 bytes for 9-tb. (In general 4*(6+3*(3+16*(2+ "nwords/apv" ))), referring to the link in p.s. here)

The data transfer bottleneck is the fiber data rate 206.7 x 10^6 bytes/s.

There will be a ceiling on FST rate (i.e. above this it will suddenly go 100% dead) at 6.43 kHz for 3-tb, 2.23 kHz for 9-tb. If running less than 90% of this I think the extra deadtime incurred by data transfer is nearly zero.

      Hopefully this answers the questions about deadtime for FST.

      Should we make a switch to 9 timebins during the maintenance day tomorrow, and keep it that way until at least some 12x12 data is caught on thursday and at least N =?? (Ziyue??) hours of good data is accumulated for a decent study of the supposed timing instability?

      Or should the whole plan be shelved until the implementation of lower rates on forward detectors with a new trigger setup, as Tonko was suggesting? That makes a lot of sense -- if the rate has to be >2 kHz or if 15% deadtime is too much to tolerate -- provided that will be implemented fairly soon. We think there's enough cause for concern about FST now to justify some cost of this test.

      I should be ready to do this sometime tomorrow afternoon, i.e. will have 9-timebin firmware for FST compiled. This should be easy but it's been a while so I have to remind myself on some details still. But the question of when to do this test, is not for me to say.

     Sincerely,

         Gerard

p.s. FYI there is an old reference on data size and deadtime settings here https://drupal.star.bnl.gov/STAR/blog/gvisser/ist-and-gmt-fixed-busy-time-data-size-and-limiting-rate-function-ntimebins . The deadtime settings therein are not fully optimized, and are superseded by what I give above.

---

From: Ye, Zhenyu <yezhenyu AT uic.edu>
Sent: Tuesday, June 20, 2023 12:50 PM
To: Visser, Gerard <gvisser AT indiana.edu>
Cc: Tonko Ljubicic <tonko AT bnl.gov>; Star-fst L <star-fst-l AT lists.bnl.gov>
Subject: Re: [Star-fst-l] FST with 9 Time Bins Diagnostic

 

Hi Gerard

In Run 22, we ran 9 time bin in the beginning, and switched to 3 time bin on 12/20/2021 after we tuned the latency 

Below is the logbook entry left by Tonko 

13:18 FST

Gerard says: We have downloaded arm_r108_fst_3tb.bin to all the RDO's of FST. This sets 3 timebin readout (was 9), with 1 triple-timebin triggering of the APV chip (was 3), and skips the APV's 8-11 and 20-23 which do not exist for FST.

On Jun 20, 2023, at 5:39 PM, Visser, Gerard <gvisser AT indiana.edu> wrote:

And there's another complication, gadzooks! I do not see a firmware file for FST 9-timebin, so I will have to prepare one today. That shouldn't be a problem. But did we not at all run 9-timebin when we first set up the FST?? Zhenyu do you recall? This is surprising me. (It's possible we ran arm_r106_ist_9tb.bin but that will cause other problems now so it's not the thing to do. I will have to prepare a new file.)

 Tonko thanks for reminder on the pedestals! Truthfully I might have forgotten...

    Gerard

---

From: Tonko Ljubicic <tonko AT bnl.gov>
Sent: Tuesday, June 20, 2023 11:29 AM
To: Visser, Gerard <gvisser AT indiana.edu>
Cc: Ziyue Zhang <zzhan70 AT uic.edu>; Star-fst L <star-fst-l AT lists.bnl.gov>
Subject: Re: [Star-fst-l] FST with 9 Time Bins Diagnostic

 

Note another complication:

You _must_ take a pedestal run after you switched to 9 timebins
and then again once you switch back to 3.


On Tue, Jun 20, 2023 at 5:13 PM Visser, Gerard <gvisser AT indiana.edu> wrote:
>
> Hi Ziyue.
>
> Roughly speaking the fixed deadtime will have to be increased by a factor of three, if it is really optimized right now. For the record Tonko says that the fixed deadtime setting right now is 300 (RHIC strobe tics). I am trying to find my notes or recalculate the required deadtime change for going from 3 timebin to 9 timebin APV readout, I'll reply on that soon.
>
> The higher data volume of 9-timebin will also impose a hard ceiling on the rate. This was discussed a long time ago on FST mailing list; likewise I'm going to have to do some digging to find the number and will reply when I can.
>
> Flemming (I think it was) asked for a brief summary of what the apparent problem that we want to check on with this 9 timebin running to be sent to starops (or triggerboard?). I agree that is a very good idea. It would really be beneficial to have some statement about any real problem observed in the data, beyond merely an observation that the 'max timebin' plots seem unstable. It would also be good to have some statement about if this is definitely different than last year, or definitely different than IST.
>
> I think that 9-timebin running, which is the FST way of "pre/post" running, should give a confirmation that the timing is really properly set, and it should give more insights into any instability in the timing if there is an instability during the time period that we run 9-tb. Since there will be a cost to such running we will have to be careful to run only as long as really needed.
>
> I can't think of any mechanism which would cause the timing of APV sampling to drift. A particular broken FEE or broken ARM module perhaps, but the observation is more widespread than that, if I understand right. I still mainly suspect that the 3-timebin 'max timebin' plot is influenced by something else than a drift of the sampling time - background or some difference between triggers or the pulse shapes are just not constant enough or something. None of which (I think) would really directly be a problem for FST data quality. Of course I may be completely wrong here. It would be excellent to have some kind of tracking result from current FST data to try to judge if it is working correctly or not, but I appreciate that is a huge ask.
>
>       Gerard
>
> ________________________________
> From: Star-fst-l <star-fst-l-bounces AT lists.bnl.gov> on behalf of Ziyue Zhang <zzhan70 AT uic.edu>
> Sent: Tuesday, June 20, 2023 10:44 AM
> To: Star-fst L <star-fst-l AT lists.bnl.gov>
> Subject: [Star-fst-l] FST with 9 Time Bins Diagnostic
>
> Hello all,
>
> We discussed the FST timing drift issue in this morning's (June 20) meeting and proposed to switch to 9 time-bin temporarily for diagnostic purposes. In the daily ops meeting, it was brought up that this will influence the deadtime of many triggers, and therefore it does not come for free.
>
> Gerard, I only have a general idea on the trigger dead time topic. Do you have any comments on this?
>
> Therefore we should estimate how long this diagnostic is going to take and discuss what exactly we need to look at, then we can switch to 9 time-bin and do the diagnostic as efficiently as possible.
>
> Best,
> Ziyue
>
>
>
> _______________________________________________
> Star-fst-l mailing list
> Star-fst-l AT lists.bnl.gov
> https://lists.bnl.gov/mailman/listinfo/star-fst-l

_______________________________________________
Star-fst-l mailing list
Star-fst-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/star-fst-l

_______________________________________________
Star-fst-l mailing list
Star-fst-l AT lists.bnl.gov
https://nam04.safelinks.protection.outlook.com/?url="https%3A%2F%2Flists.bnl.gov%2Fmailman%2Flistinfo%2Fstar-fst-l&data=05%7C01%7Czzhan70%40groute.uic.edu%7Cc87b15e9ba024e5c3bde08db71de600b%7Ce202cd477a564baa99e3e3b71a7c77dd%7C0%7C0%7C638228972171481027%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4KXoAnDlHMrCTkwR6Npl38gOAgqRmoD7IYpLm88HQv4%3D&reserved=0

Attachment: plots.pdf
Description: Adobe PDF document