Star-tpc-l mailing list

["Van Buren, Gene" <gene AT bnl.gov>]

Correction.... those 3 had no events processed, which I was falsely 
triggering my counts on. Doing it properly, I found...

- 1463 files st_physics(_adc) files processed by FastOffline in the past 36 
hours.
- 3 had no events processed (I haven't looked into this)...2 st_physics_adc, 
1 st_physics
- 669 st_physics files, no iTPC issue that I can tell
- 765 st_physics_adc files, of which 60 had the iTPC issue

So it appears that it may only be in st_physics_adc files, for which we run a 
different chain to run the offline cluster finder: TpxRaw, TpxClu. And of 
those, 60/765 = ~8% had the iTPC issue.

More investigating to do...

-Gene

> On May 31, 2024, at 12:20 PM, Van Buren, Gene <gene AT bnl.gov> wrote:
> 
> I just checked the log files of 1436 st_physics files that completed 
> FastOffline over the past 36 hours, and of those, 3 had the missing iTPC 
> issue....
> 
> 1 st_physics and 1 st_physics_adc file (2 total files) from run 25151063 
> had the issue
> 1 st_physics_adc file from run 25151058 had the issue
> 
> -Gene
> 
> 
>> On May 30, 2024, at 4:22 PM, Van Buren, Gene via Star-tpc-l 
>> <star-tpc-l AT lists.bnl.gov> wrote:
>> 
>> Thanks for the brainstorming, Flemming...
>> 
>>> On May 30, 2024, at 3:58 PM, videbaek <videbaek AT bnl.gov> wrote:
>>> 
>>> Thanks again,
>>> 
>>> One question to think about is
>>> - does this ever happen if files are processed later? If that does not 
>>> happen it must be due to environment of running.
>> 
>> I have only on a couple occasions re-run a job myself. Neither time was 
>> the problematic job problematic when I processed it. To make things more 
>> confusing, jobs that ran later in FastOffline for the same run also had 
>> issues. See this list of jobs for run 25149018 sorted by their job start 
>> time (EDT) on Tuesday, where the jobs with an asterisk * showed the issue:
>> 
>> 08:52:02 st_physics_adc_25149018_raw_1400010
>> 08:53:30 st_physics_25149018_raw_0900002
>> 08:53:34 st_physics_25149018_raw_0700003
>> 08:53:48 st_physics_25149018_raw_2200003
>> * 09:12:59 st_physics_adc_25149018_raw_1700009
>> 09:13:51 st_physics_adc_25149018_raw_2100010
>> 09:14:01 st_physics_25149018_raw_2100002
>> 09:14:08 st_physics_25149018_raw_0200002
>> 09:14:28 st_physics_25149018_raw_1000002
>> 09:15:12 st_physics_adc_25149018_raw_0500007
>> 09:15:21 st_physics_25149018_raw_1600002
>> 09:15:27 st_physics_25149018_raw_1900002
>> 09:15:43 st_physics_25149018_raw_0300003
>> 09:34:02 st_physics_adc_25149018_raw_1600005
>> 09:34:07 st_physics_adc_25149018_raw_2000006
>> 09:35:18 st_physics_adc_25149018_raw_2200006
>> 12:15:00 st_physics_adc_25149018_raw_1700009  <==== ME RE-PROCESSING THE 
>> EARLIER PROBLEMATIC JOB
>> * 20:17:20 st_physics_adc_25149018_raw_0400008
>> 20:31:24 st_physics_adc_25149018_raw_0900010
>> 20:31:35 st_physics_adc_25149018_raw_1500009
>> 20:33:16 st_physics_adc_25149018_raw_1100005
>> 20:35:34 st_physics_adc_25149018_raw_1300007
>> 20:35:46 st_physics_adc_25149018_raw_1000010
>> 20:57:01 st_physics_adc_25149018_raw_1900007
>> 21:11:17 st_physics_adc_25149018_raw_2300009
>> * 21:55:38 st_physics_adc_25149018_raw_1200008
>> 22:14:20 st_physics_adc_25149018_raw_2400009
>> 22:41:31 st_physics_adc_25149018_raw_0700009
>> 23:44:11 st_physics_adc_25149018_raw_0800009
>> 
>>> - What triggers do fast offline use?
>> 
>> No selection of triggers. Everything in the DAQ files, but only for 
>> certain DAQ file streams.
>> 
>>> - fastoffline do analyze both physics and adc_physics , yes?
>> 
>> Yes, though st_physics_adc has offline TPC cluster-finding turned on. I 
>> can't remember whether I fully determined that you no longer need this for 
>> the TPC QA you and Isaac set up, but at first I found we did need it.
>> 
>> Thanks,
>> -Gene
>> 
>>> 
>>> Flemming
>>> 
>>> 
>>> On 2024-05-30 15:48, Van Buren, Gene wrote:
>>>> There were many points ;-)
>>>>> On May 30, 2024, at 3:45 PM, videbaek <videbaek AT bnl.gov> wrote:
>>>>> Could it be dependent on what node it was running?
>>>>> On 2024-05-30 15:38, Van Buren, Gene wrote:
>>>>>> - No correlation with farm node on which the jobs ran
>>>> There are many farm nodes, but in fact in the run I studied, a single
>>>> node happened to process good job and one problematic job.
>>>> -Gene
>>> 
>>> -- 
>>> Flemming Videbaek
>>> senior scientist, emeritus
>>> videbaek @ bnl.gov
>>> Brookhaven National Lab
>>> Physics Department
>>> Bldg 510D
>>> Upton, NY 11973
>>> 
>>> phone: 631-344-4106
>>> cell     :  631-681-1596
>> 
>> _______________________________________________
>> Star-tpc-l mailing list
>> Star-tpc-l AT lists.bnl.gov
>> https://lists.bnl.gov/mailman/listinfo/star-tpc-l
>