Sphenix-production-l mailing list

[Alex Lebedev <lebedev AT iastate.edu>]

Hi all,

I've attached a list of recently failed jobs.

The TRKR_SEED failures are essentially gone after Joe's fix. 

I believe a few failed jobs were run before the fix 

was implemented.

But we have a real problem with calorimeter event building and 

reconstruction.

There are over 300 failed calo event building jobs, 

and the second step in the new two-step calo DST production 

fails almost completely. There are > 6000 failed jobs.

The first step looks ok, there are only two failed jobs.

In the failed jobs list I show only a few examples of the failed 

calo reco second step jobs, there are too many to list.

The log files are:

TRIGGERED_EVENT (calo event building): 

   /sphenix/data/data02/sphnxpro/calologs/ana430_2024p007/

CALOFITTING (first calo waveform fitting step): 

   /sphenix/data/data02/sphnxpro/caloy2fitting/ana430_2024p007/

CALOCALIB: (second calo production step, the one that fails): 

   /sphenix/data/data02/sphnxpro/caloy2calib/ana430_2024p007/

Sasha.

   id    |                           dstfile                          |  
status   | flags |  condor   |       started       
---------+------------------------------------------------------------+-----------+-------+-----------+---------------------
 4367659 | DST_TRKR_TRACKS_run2pp_new_2024p005-00049718-00004      | failed   
  |       | 30360.1583  | 2024-08-02 02:23:19
 4367658 | DST_TRKR_TRACKS_run2pp_new_2024p005-00049717-00004      | failed   
  |       | 30360.1582  | 2024-08-02 02:23:19
 4367657 | DST_TRKR_TRACKS_run2pp_new_2024p005-00049716-00004      | failed   
  |       | 30360.1581  | 2024-08-02 02:23:19
 4367656 | DST_TRKR_TRACKS_run2pp_new_2024p005-00049709-00004      | failed   
  |       | 30360.1580  | 2024-08-02 02:23:19
 4367655 | DST_TRKR_TRACKS_run2pp_new_2024p005-00049707-00004      | failed   
  |       | 30360.1579  | 2024-08-02 02:23:18
 4367654 | DST_TRKR_TRACKS_run2pp_new_2024p005-00049690-00005      | failed   
  |       | 30360.1578  | 2024-08-02 02:23:19
 4367653 | DST_TRKR_TRACKS_run2pp_new_2024p005-00049688-00004      | failed   
  |       | 30360.1577  | 2024-08-02 02:23:18
 4352639 | DST_TRKR_SEED_run2pp_new_2024p005-00049716-00004        | failed   
  |       | 30339.816   | 2024-08-02 01:09:03
 4352638 | DST_TRKR_SEED_run2pp_new_2024p005-00049711-00004        | failed   
  |       | 30339.815   | 2024-08-02 01:09:01
 4352637 | DST_TRKR_SEED_run2pp_new_2024p005-00049684-00002        | failed   
  |       | 30339.814   | 2024-08-02 01:09:01
 4338857 | DST_TRKR_SEED_run2pp_new_2024p005-00049718-00004        | failed   
  |       | 30316.768   | 2024-08-02 00:06:24
 4338856 | DST_TRKR_SEED_run2pp_new_2024p005-00049717-00004        | failed   
  |       | 30316.767   | 2024-08-02 00:06:23
 4338855 | DST_TRKR_SEED_run2pp_new_2024p005-00049709-00004        | failed   
  |       | 30316.766   | 2024-08-02 00:06:22
 4338854 | DST_TRKR_SEED_run2pp_new_2024p005-00049707-00004        | failed   
  |       | 30316.765   | 2024-08-02 00:06:19
 4338853 | DST_TRKR_SEED_run2pp_new_2024p005-00049690-00005        | failed   
  |       | 30316.764   | 2024-08-02 00:06:19
 4338852 | DST_TRKR_SEED_run2pp_new_2024p005-00049688-00004        | failed   
  |       | 30316.763   | 2024-08-02 00:06:18

 4428120 | DST_CALOFITTING_run2pp_ana430_2024p007-00045048-00029 | failed   | 
    5 | 30434.91   | 2024-08-02 06:57:40
 4425250 | DST_CALOFITTING_run2pp_ana430_2024p007-00047448-00036 | failed   | 
    5 | 30431.779  | 2024-08-02 06:45:05

 4392222 | DST_CALO_run2pp_ana430_2024p007-00044501-00006        | failed     
|       | 30393.102   | 2024-08-02 04:27:48
 4389563 | DST_CALO_run2pp_ana430_2024p007-00044501-00004        | failed     
|       | 30390.677   | 2024-08-02 04:27:05
 4389232 | DST_CALO_run2pp_ana430_2024p007-00044501-00005        | failed     
|       | 30390.346   | 2024-08-02 04:26:43
 4383623 | DST_CALO_run2pp_ana430_2024p007-00044502-00000        | failed     
|       | 30381.768   | 2024-08-02 04:24:40
 4380228 | DST_CALO_run2pp_ana430_2024p007-00044502-00001        | failed     
|       | 30377.396   | 2024-08-02 04:23:08
.....
 4511728 | DST_CALO_run2pp_ana430_2024p007-00043268-00000        | failed     
|       | 30527.289   | 2024-08-02 12:57:37
 4511666 | DST_CALO_run2pp_ana430_2024p007-00043535-00000        | failed     
|       | 30527.227   | 2024-08-02 12:57:34
 4511581 | DST_CALO_run2pp_ana430_2024p007-00043528-00000        | failed     
|       | 30527.142   | 2024-08-02 12:57:28
 4511523 | DST_CALO_run2pp_ana430_2024p007-00043265-00004        | failed     
|       | 30527.84    | 2024-08-02 12:57:05

 4325062 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045941-00000  | 
failed    |       | 30289.363  | 2024-08-02 18:21:47
 4325060 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045939-00000  | 
failed    |       | 30289.361  | 2024-08-02 18:24:01
 4325052 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045886-00000  | 
failed    |       | 30289.353  | 2024-08-02 17:54:19
 4325048 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045882-00000  | 
failed    |       | 30289.349  | 2024-08-02 17:49:15
 4325047 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045881-00000  | 
failed    |       | 30289.348  | 2024-08-02 17:45:46
 4325043 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045877-00000  | 
failed    |       | 30289.344  | 2024-08-02 17:34:10
 4325037 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045870-00000  | 
failed    |       | 30289.338  | 2024-08-02 17:28:52
 4325036 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045869-00000  | 
failed    |       | 30289.337  | 2024-08-02 17:26:35
 4325035 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045868-00000  | 
failed    |       | 30289.336  | 2024-08-02 17:23:01
 4325033 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045860-00000  | 
failed    |       | 30289.334  | 2024-08-02 17:18:15
 4325029 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045856-00000  | 
failed    |       | 30289.330  | 2024-08-02 17:16:47
 4325028 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045855-00000  | 
failed    |       | 30289.329  | 2024-08-02 17:16:47
 4325023 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045850-00000  | 
failed    |       | 30289.324  | 2024-08-02 17:13:18
 4325022 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045849-00000  | 
failed    |       | 30289.323  | 2024-08-02 17:11:09
 4325014 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045835-00000  | 
failed    |       | 30289.315  | 2024-08-02 16:50:36
 4325010 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045831-00000  | 
failed    |       | 30289.311  | 2024-08-02 16:41:34
 4325007 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045815-00000  | 
failed    |       | 30289.308  | 2024-08-02 16:35:38
 4325006 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045812-00000  | 
failed    |       | 30289.307  | 2024-08-02 16:35:19
 4325003 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045728-00000  | 
failed    |       | 30289.304  | 2024-08-02 16:30:33
 4325002 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045727-00000  | 
failed    |       | 30289.303  | 2024-08-02 16:30:27
 4325001 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045726-00000  | 
failed    |       | 30289.302  | 2024-08-02 16:30:27
 4324998 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045721-00000  | 
failed    |       | 30289.299  | 2024-08-02 16:27:14
 4324985 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045644-00000  | 
failed    |       | 30289.286  | 2024-08-02 16:04:10
 4324984 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045643-00000  | 
failed    |       | 30289.285  | 2024-08-02 16:03:59
 4324975 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045632-00000  | 
failed    |       | 30289.276  | 2024-08-02 15:43:17
 4324974 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045631-00000  | 
failed    |       | 30289.275  | 2024-08-02 15:42:12
 4324972 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045629-00000  | 
failed    |       | 30289.273  | 2024-08-02 15:38:01
 4324964 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045621-00000  | 
failed    |       | 30289.265  | 2024-08-02 15:28:11
 4324962 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045619-00000  | 
failed    |       | 30289.263  | 2024-08-02 15:26:34
 4324961 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045618-00000  | 
failed    |       | 30289.262  | 2024-08-02 15:25:10
 4324958 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045559-00000  | 
failed    |       | 30289.259  | 2024-08-02 15:21:24
 4324957 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045558-00000  | 
failed    |       | 30289.258  | 2024-08-02 15:20:58
 4324945 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045542-00000  | 
failed    |       | 30289.246  | 2024-08-02 15:06:39
 4324941 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045538-00000  | 
failed    |       | 30289.242  | 2024-08-02 15:01:50
 4324940 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045537-00000  | 
failed    |       | 30289.241  | 2024-08-02 15:00:49
 4324938 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045534-00000  | 
failed    |       | 30289.239  | 2024-08-02 15:00:21
 4324934 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045529-00000  | 
failed    |       | 30289.235  | 2024-08-02 14:57:45
 4324933 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045527-00000  | 
failed    |       | 30289.234  | 2024-08-02 14:54:25
 4324930 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043999-00000  | 
failed    |       | 30289.231  | 2024-08-02 14:49:33
 4324929 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043998-00000  | 
failed    |       | 30289.230  | 2024-08-02 14:48:58
 4324928 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043997-00000  | 
failed    |       | 30289.229  | 2024-08-02 14:48:40
 4324916 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043980-00000  | 
failed    |       | 30289.217  | 2024-08-02 14:41:40
 4324915 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043979-00000  | 
failed    |       | 30289.216  | 2024-08-02 14:40:30
 4324910 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043974-00000  | 
failed    |       | 30289.211  | 2024-08-02 14:36:50
 4324904 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043961-00000  | 
failed    |       | 30289.205  | 2024-08-02 14:27:20
 4324884 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043894-00000  | 
failed    |       | 30289.185  | 2024-08-02 13:52:05
 4324871 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043878-00000  | 
failed    |       | 30289.172  | 2024-08-02 13:46:18
 4324866 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043853-00000  | 
failed    |       | 30289.167  | 2024-08-02 13:45:13
 4324865 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043852-00000  | 
failed    |       | 30289.166  | 2024-08-02 13:44:40
 4324864 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043842-00000  | 
failed    |       | 30289.165  | 2024-08-02 13:44:23
 4324863 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043841-00000  | 
failed    |       | 30289.164  | 2024-08-02 13:44:14
 4324859 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043818-00000  | 
failed    |       | 30289.160  | 2024-08-02 13:43:28
 4324858 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043817-00000  | 
failed    |       | 30289.159  | 2024-08-02 13:43:27
 4324857 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043816-00000  | 
failed    |       | 30289.158  | 2024-08-02 13:43:25
 4324856 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043815-00000  | 
failed    |       | 30289.157  | 2024-08-02 13:43:21
 4324855 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043814-00000  | 
failed    |       | 30289.156  | 2024-08-02 13:42:35
 4324854 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043813-00000  | 
failed    |       | 30289.155  | 2024-08-02 13:42:22
 4324853 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043811-00000  | 
failed    |       | 30289.154  | 2024-08-02 13:42:12
 4324852 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043810-00000  | 
failed    |       | 30289.153  | 2024-08-02 13:41:42
 4324851 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043809-00000  | 
failed    |       | 30289.152  | 2024-08-02 13:41:34
 4324850 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043806-00000  | 
failed    |       | 30289.151  | 2024-08-02 13:41:04
 4324849 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043805-00000  | 
failed    |       | 30289.150  | 2024-08-02 13:40:59
 4324848 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043804-00000  | 
failed    |       | 30289.149  | 2024-08-02 13:40:45
 4324847 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043803-00000  | 
failed    |       | 30289.148  | 2024-08-02 13:40:19
 4324846 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043802-00000  | 
failed    |       | 30289.147  | 2024-08-02 13:39:35
 4324845 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043801-00000  | 
failed    |       | 30289.146  | 2024-08-02 13:38:07
 4324844 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043800-00000  | 
failed    |       | 30289.145  | 2024-08-02 13:37:12
 4324843 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043799-00000  | 
failed    |       | 30289.144  | 2024-08-02 13:36:33
 4324842 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043798-00000  | 
failed    |       | 30289.143  | 2024-08-02 13:36:21
 4324841 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043797-00000  | 
failed    |       | 30289.142  | 2024-08-02 13:36:02
 4324840 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043796-00000  | 
failed    |       | 30289.141  | 2024-08-02 13:35:59
 4324839 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043795-00000  | 
failed    |       | 30289.140  | 2024-08-02 13:35:02
 4324838 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043794-00000  | 
failed    |       | 30289.139  | 2024-08-02 13:34:50
 4324837 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043793-00000  | 
failed    |       | 30289.138  | 2024-08-02 13:34:46
 4324836 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043792-00000  | 
failed    |       | 30289.137  | 2024-08-02 13:34:18
 4324835 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043791-00000  | 
failed    |       | 30289.136  | 2024-08-02 13:33:55
 4324834 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043790-00000  | 
failed    |       | 30289.135  | 2024-08-02 13:33:51
 4324833 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043789-00000  | 
failed    |       | 30289.134  | 2024-08-02 13:33:47
 4324832 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043788-00000  | 
failed    |       | 30289.133  | 2024-08-02 13:33:43
 4324831 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043787-00000  | 
failed    |       | 30289.132  | 2024-08-02 13:33:56
 4324830 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043778-00000  | 
failed    |       | 30289.131  | 2024-08-02 13:33:12
 4324829 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043769-00000  | 
failed    |       | 30289.130  | 2024-08-02 13:31:26
 4324828 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043758-00000  | 
failed    |       | 30289.129  | 2024-08-02 13:30:53
 4324827 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043757-00000  | 
failed    |       | 30289.128  | 2024-08-02 13:30:46
 4324826 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043753-00000  | 
failed    |       | 30289.127  | 2024-08-02 13:30:17
 4324825 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043752-00000  | 
failed    |       | 30289.126  | 2024-08-02 13:29:00
 4324824 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043749-00000  | 
failed    |       | 30289.125  | 2024-08-02 13:28:43
 4324823 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043744-00000  | 
failed    |       | 30289.124  | 2024-08-02 13:27:05
 4324822 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043742-00000  | 
failed    |       | 30289.123  | 2024-08-02 13:26:04
 4324821 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043741-00000  | 
failed    |       | 30289.122  | 2024-08-02 13:25:55
 4324820 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043740-00000  | 
failed    |       | 30289.121  | 2024-08-02 13:25:05
 4324819 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043739-00000  | 
failed    |       | 30289.120  | 2024-08-02 13:24:38
 4324818 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043733-00000  | 
failed    |       | 30289.119  | 2024-08-02 13:22:41
 4324817 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043718-00000  | 
failed    |       | 30289.118  | 2024-08-02 13:24:58
 4324816 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043716-00000  | 
failed    |       | 30289.117  | 2024-08-02 13:20:43
 4324815 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043715-00000  | 
failed    |       | 30289.116  | 2024-08-02 13:19:37
 4324814 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043687-00000  | 
failed    |       | 30289.115  | 2024-08-02 13:18:40
 4324812 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043685-00000  | 
failed    |       | 30289.113  | 2024-08-02 13:10:54
 4324810 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043683-00000  | 
failed    |       | 30289.111  | 2024-08-02 13:05:23
 4324807 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043680-00000  | 
failed    |       | 30289.108  | 2024-08-02 13:03:13
 4324806 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043679-00000  | 
failed    |       | 30289.107  | 2024-08-02 13:02:36
 4324803 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043676-00000  | 
failed    |       | 30289.104  | 2024-08-02 13:01:28
 4324801 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043674-00000  | 
failed    |       | 30289.102  | 2024-08-02 12:52:07
 4324800 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043673-00000  | 
failed    |       | 30289.101  | 2024-08-02 12:51:12
 4324799 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043672-00000  | 
failed    |       | 30289.100  | 2024-08-02 12:51:08
 4324798 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043671-00000  | 
failed    |       | 30289.99   | 2024-08-02 12:50:56
 4324796 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043669-00000  | 
failed    |       | 30289.97   | 2024-08-02 12:46:57
 4324795 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043668-00000  | 
failed    |       | 30289.96   | 2024-08-02 12:45:52
 4324792 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043665-00000  | 
failed    |       | 30289.93   | 2024-08-02 12:43:57
 4324791 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043664-00000  | 
failed    |       | 30289.92   | 2024-08-02 12:42:55
 4324789 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043662-00000  | 
failed    |       | 30289.90   | 2024-08-02 12:40:05
 4324788 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043661-00000  | 
failed    |       | 30289.89   | 2024-08-02 12:37:18
 4324786 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043659-00000  | 
failed    |       | 30289.87   | 2024-08-02 12:36:32
 4324785 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043658-00000  | 
failed    |       | 30289.86   | 2024-08-02 12:36:23
 4324784 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043657-00000  | 
failed    |       | 30289.85   | 2024-08-02 12:36:07
 4324783 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043656-00000  | 
failed    |       | 30289.84   | 2024-08-02 12:35:25
 4324782 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043655-00000  | 
failed    |       | 30289.83   | 2024-08-02 12:35:56
 4324781 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043654-00000  | 
failed    |       | 30289.82   | 2024-08-02 12:33:35
 4324779 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043652-00000  | 
failed    |       | 30289.80   | 2024-08-02 12:33:05
 4324777 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043644-00000  | 
failed    |       | 30289.78   | 2024-08-02 12:31:56
 4324776 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043643-00000  | 
failed    |       | 30289.77   | 2024-08-02 12:31:48
 4324775 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043642-00000  | 
failed    |       | 30289.76   | 2024-08-02 12:31:09
 4324774 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043641-00000  | 
failed    |       | 30289.75   | 2024-08-02 12:31:00
 4324773 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043640-00000  | 
failed    |       | 30289.74   | 2024-08-02 12:30:31
 4324772 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043639-00000  | 
failed    |       | 30289.73   | 2024-08-02 12:30:19
 4324771 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043631-00000  | 
failed    |       | 30289.72   | 2024-08-02 12:30:04
 4324770 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043630-00000  | 
failed    |       | 30289.71   | 2024-08-02 12:30:00
 4324769 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043628-00000  | 
failed    |       | 30289.70   | 2024-08-02 12:29:56
 4324768 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043627-00000  | 
failed    |       | 30289.69   | 2024-08-02 12:29:49
 4324767 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043626-00000  | 
failed    |       | 30289.68   | 2024-08-02 12:29:23
 4324766 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043625-00000  | 
failed    |       | 30289.67   | 2024-08-02 12:29:16
 4324765 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043617-00000  | 
failed    |       | 30289.66   | 2024-08-02 12:29:15
 4324764 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043616-00000  | 
failed    |       | 30289.65   | 2024-08-02 12:29:07
 4324763 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043615-00000  | 
failed    |       | 30289.64   | 2024-08-02 12:28:46
 4324762 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043580-00000  | 
failed    |       | 30289.63   | 2024-08-02 12:28:31
 4324761 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043579-00000  | 
failed    |       | 30289.62   | 2024-08-02 12:27:51
 4324760 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043578-00000  | 
failed    |       | 30289.61   | 2024-08-02 12:27:22
 4324759 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043574-00000  | 
failed    |       | 30289.60   | 2024-08-02 12:27:16
 4324758 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043573-00000  | 
failed    |       | 30289.59   | 2024-08-02 12:26:57
 4324757 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043572-00000  | 
failed    |       | 30289.58   | 2024-08-02 12:26:31
 4324756 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043571-00000  | 
failed    |       | 30289.57   | 2024-08-02 12:25:02
 4324755 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043570-00000  | 
failed    |       | 30289.56   | 2024-08-02 12:26:31
 4324754 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043569-00000  | 
failed    |       | 30289.55   | 2024-08-02 12:26:30
 4324753 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043568-00000  | 
failed    |       | 30289.54   | 2024-08-02 12:22:18
 4324752 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043567-00000  | 
failed    |       | 30289.53   | 2024-08-02 12:26:27
 4324751 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043566-00000  | 
failed    |       | 30289.52   | 2024-08-02 12:26:23
 4324750 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043565-00000  | 
failed    |       | 30289.51   | 2024-08-02 12:21:36
 4324749 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043564-00000  | 
failed    |       | 30289.50   | 2024-08-02 12:20:36
 4324748 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043563-00000  | 
failed    |       | 30289.49   | 2024-08-02 12:20:26
 4324747 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043562-00000  | 
failed    |       | 30289.48   | 2024-08-02 12:18:34
 4324746 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043561-00000  | 
failed    |       | 30289.47   | 2024-08-02 12:18:18
 4324745 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043560-00000  | 
failed    |       | 30289.46   | 2024-08-02 12:17:49
 4324744 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043559-00000  | 
failed    |       | 30289.45   | 2024-08-02 12:17:23
 4324743 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043558-00000  | 
failed    |       | 30289.44   | 2024-08-02 12:17:20
 4324742 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043557-00000  | 
failed    |       | 30289.43   | 2024-08-02 12:17:19
 4324741 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043556-00000  | 
failed    |       | 30289.42   | 2024-08-02 12:16:51
 4324740 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043555-00000  | 
failed    |       | 30289.41   | 2024-08-02 12:16:50
 4324739 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043554-00000  | 
failed    |       | 30289.40   | 2024-08-02 12:15:21
 4324738 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043553-00000  | 
failed    |       | 30289.39   | 2024-08-02 12:15:19
 4324737 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043552-00000  | 
failed    |       | 30289.38   | 2024-08-02 12:15:10
 4324736 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043551-00000  | 
failed    |       | 30289.37   | 2024-08-02 12:15:08
 4324735 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043550-00000  | 
failed    |       | 30289.36   | 2024-08-02 12:14:55
 4324734 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043549-00000  | 
failed    |       | 30289.35   | 2024-08-02 12:12:23
 4324733 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043548-00000  | 
failed    |       | 30289.34   | 2024-08-02 12:12:19
 4324732 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043547-00000  | 
failed    |       | 30289.33   | 2024-08-02 12:12:08
 4324731 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043546-00000  | 
failed    |       | 30289.32   | 2024-08-02 12:12:07
 4324730 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043545-00000  | 
failed    |       | 30289.31   | 2024-08-02 12:12:06
 4324729 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043544-00000  | 
failed    |       | 30289.30   | 2024-08-02 12:15:51
 4324728 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043543-00000  | 
failed    |       | 30289.29   | 2024-08-02 12:12:08
 4324727 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043542-00000  | 
failed    |       | 30289.28   | 2024-08-02 12:10:27
 4324726 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043541-00000  | 
failed    |       | 30289.27   | 2024-08-02 12:10:20
 4324725 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043540-00000  | 
failed    |       | 30289.26   | 2024-08-02 12:09:51
 4324724 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043539-00000  | 
failed    |       | 30289.25   | 2024-08-02 12:08:37
 4324723 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043538-00000  | 
failed    |       | 30289.24   | 2024-08-02 12:07:29
 4324722 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043537-00000  | 
failed    |       | 30289.23   | 2024-08-02 12:06:51
 4324721 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043536-00000  | 
failed    |       | 30289.22   | 2024-08-02 12:04:40
 4324720 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043535-00000  | 
failed    |       | 30289.21   | 2024-08-02 12:03:48
 4324719 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043534-00000  | 
failed    |       | 30289.20   | 2024-08-02 12:03:12
 4324718 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043533-00000  | 
failed    |       | 30289.19   | 2024-08-02 12:03:07
 4324717 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043532-00000  | 
failed    |       | 30289.18   | 2024-08-02 12:02:59
 4324716 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043531-00000  | 
failed    |       | 30289.17   | 2024-08-02 12:02:19
 4324715 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043530-00000  | 
failed    |       | 30289.16   | 2024-08-02 12:01:05
 4324714 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043529-00000  | 
failed    |       | 30289.15   | 2024-08-02 11:59:22
 4324713 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043528-00000  | 
failed    |       | 30289.14   | 2024-08-02 11:59:17
 4324712 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043527-00000  | 
failed    |       | 30289.13   | 2024-08-02 11:59:10
 4324711 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043526-00000  | 
failed    |       | 30289.12   | 2024-08-02 11:59:02
 4324708 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043523-00000  | 
failed    |       | 30289.9    | 2024-08-02 11:58:48
 4324707 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043522-00000  | 
failed    |       | 30289.8    | 2024-08-02 11:58:47
 4324706 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043521-00000  | 
failed    |       | 30289.7    | 2024-08-02 11:58:39
 4324705 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043520-00000  | 
failed    |       | 30289.6    | 2024-08-02 11:58:34
 4324704 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043519-00000  | 
failed    |       | 30289.5    | 2024-08-02 11:58:32
 4324703 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043518-00000  | 
failed    |       | 30289.4    | 2024-08-02 11:58:20
 4324702 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043517-00000  | 
failed    |       | 30289.3    | 2024-08-02 11:58:11
 4324701 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043516-00000  | 
failed    |       | 30289.2    | 2024-08-02 11:57:56
 4324700 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043514-00000  | 
failed    |       | 30289.1    | 2024-08-02 11:57:47
 4324699 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043513-00000  | 
failed    |       | 30289.0    | 2024-08-02 11:57:44
 4312027 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043451-00000  | 
failed    |       | 30260.45   | 2024-08-02 11:57:42
 4312026 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043449-00000  | 
failed    |       | 30260.44   | 2024-08-02 11:57:38
 4312025 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043435-00000  | 
failed    |       | 30260.43   | 2024-08-02 11:57:05
 4312024 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043433-00000  | 
failed    |       | 30260.42   | 2024-08-02 11:56:52
 4312023 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043432-00000  | 
failed    |       | 30260.41   | 2024-08-02 11:56:31
 4312022 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043430-00000  | 
failed    |       | 30260.40   | 2024-08-02 11:56:26
 4312021 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043429-00000  | 
failed    |       | 30260.39   | 2024-08-02 11:56:08
 4312020 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043425-00000  | 
failed    |       | 30260.38   | 2024-08-02 11:56:02
 4312019 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043424-00000  | 
failed    |       | 30260.37   | 2024-08-02 11:55:59
 4312018 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043423-00000  | 
failed    |       | 30260.36   | 2024-08-02 11:55:50
 4312017 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043422-00000  | 
failed    |       | 30260.35   | 2024-08-02 11:55:49
 4312016 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043419-00000  | 
failed    |       | 30260.34   | 2024-08-02 11:55:18
 4312015 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043418-00000  | 
failed    |       | 30260.33   | 2024-08-02 11:55:10
 4312014 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043416-00000  | 
failed    |       | 30260.32   | 2024-08-02 11:55:03
 4312013 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043409-00000  | 
failed    |       | 30260.31   | 2024-08-02 11:54:49
 4312012 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043406-00000  | 
failed    |       | 30260.30   | 2024-08-02 11:54:36
 4312011 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043405-00000  | 
failed    |       | 30260.29   | 2024-08-02 11:54:17
 4312010 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043404-00000  | 
failed    |       | 30260.28   | 2024-08-02 11:53:57
 4312009 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043403-00000  | 
failed    |       | 30260.27   | 2024-08-02 11:53:45
 4312008 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043399-00000  | 
failed    |       | 30260.26   | 2024-08-02 11:53:42
 4312007 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043397-00000  | 
failed    |       | 30260.25   | 2024-08-02 11:52:33
 4312006 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043395-00000  | 
failed    |       | 30260.24   | 2024-08-02 11:51:59
 4312005 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043392-00000  | 
failed    |       | 30260.23   | 2024-08-02 11:51:52
 4312004 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043390-00000  | 
failed    |       | 30260.22   | 2024-08-02 11:51:30
 4312003 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043312-00000  | 
failed    |       | 30260.21   | 2024-08-02 11:51:16
 4312002 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043306-00000  | 
failed    |       | 30260.20   | 2024-08-02 11:51:13
 4312001 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043302-00000  | 
failed    |       | 30260.19   | 2024-08-02 11:51:05
 4312000 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043300-00000  | 
failed    |       | 30260.18   | 2024-08-02 11:50:51
 4311999 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043295-00000  | 
failed    |       | 30260.17   | 2024-08-02 11:49:03
 4311998 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043289-00000  | 
failed    |       | 30260.16   | 2024-08-02 11:48:27
 4311997 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043287-00000  | 
failed    |       | 30260.15   | 2024-08-02 11:47:53
 4311996 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043286-00000  | 
failed    |       | 30260.14   | 2024-08-02 11:47:46
 4311995 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043284-00000  | 
failed    |       | 30260.13   | 2024-08-02 11:47:19
 4311994 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043279-00000  | 
failed    |       | 30260.12   | 2024-08-02 11:47:06
 4311993 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043275-00000  | 
failed    |       | 30260.11   | 2024-08-02 11:47:06
 4311992 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043274-00000  | 
failed    |       | 30260.10   | 2024-08-02 11:45:05
 4311991 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043273-00000  | 
failed    |       | 30260.9    | 2024-08-02 11:45:18
 4311989 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043271-00000  | 
failed    |       | 30260.7    | 2024-08-02 11:41:05
 4311988 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043270-00000  | 
failed    |       | 30260.6    | 2024-08-02 11:39:33
 4311987 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043269-00000  | 
failed    |       | 30260.5    | 2024-08-02 11:37:04
 4311986 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043268-00000  | 
failed    |       | 30260.4    | 2024-08-02 11:35:54
 4311984 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043266-00000  | 
failed    |       | 30260.2    | 2024-08-02 11:35:41
 4311983 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043265-00000  | 
failed    |       | 30260.1    | 2024-08-02 11:35:36
 4311982 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00043263-00000  | 
failed    |       | 30260.0    | 2024-08-02 11:35:31
 4310902 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044451-00000  | 
failed    |       | 30256.158  | 2024-08-02 11:28:36
 4310901 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044450-00000  | 
failed    |       | 30256.157  | 2024-08-02 11:28:26
 4310900 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044448-00000  | 
failed    |       | 30256.156  | 2024-08-02 11:28:10
 4310899 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044447-00000  | 
failed    |       | 30256.155  | 2024-08-02 11:27:54
 4310898 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044446-00000  | 
failed    |       | 30256.154  | 2024-08-02 11:26:53
 4310897 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044433-00000  | 
failed    |       | 30256.153  | 2024-08-02 11:23:53
 4310896 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044432-00000  | 
failed    |       | 30256.152  | 2024-08-02 11:23:40
 4310895 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044428-00000  | 
failed    |       | 30256.151  | 2024-08-02 11:22:32
 4310894 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044426-00000  | 
failed    |       | 30256.150  | 2024-08-02 11:22:26
 4310893 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044422-00000  | 
failed    |       | 30256.149  | 2024-08-02 11:22:21
 4310892 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044421-00000  | 
failed    |       | 30256.148  | 2024-08-02 11:22:17
 4310891 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044420-00000  | 
failed    |       | 30256.147  | 2024-08-02 11:21:20
 4310890 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044417-00000  | 
failed    |       | 30256.146  | 2024-08-02 11:20:46
 4310889 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044413-00000  | 
failed    |       | 30256.145  | 2024-08-02 11:19:43
 4310888 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044412-00000  | 
failed    |       | 30256.144  | 2024-08-02 11:18:45
 4310887 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044411-00000  | 
failed    |       | 30256.143  | 2024-08-02 11:18:45
 4310886 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044410-00000  | 
failed    |       | 30256.142  | 2024-08-02 11:18:42
 4310885 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044409-00000  | 
failed    |       | 30256.141  | 2024-08-02 11:17:09
 4310882 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044406-00000  | 
failed    |       | 30256.138  | 2024-08-02 11:16:21
 4310881 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044405-00000  | 
failed    |       | 30256.137  | 2024-08-02 11:14:54
 4310880 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044404-00000  | 
failed    |       | 30256.136  | 2024-08-02 11:16:20
 4310879 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044403-00000  | 
failed    |       | 30256.135  | 2024-08-02 11:13:50
 4310878 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044402-00000  | 
failed    |       | 30256.134  | 2024-08-02 11:16:13
 4310877 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044401-00000  | 
failed    |       | 30256.133  | 2024-08-02 11:16:16
 4310876 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044400-00000  | 
failed    |       | 30256.132  | 2024-08-02 11:13:39
 4310875 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044399-00000  | 
failed    |       | 30256.131  | 2024-08-02 11:13:36
 4310874 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044398-00000  | 
failed    |       | 30256.130  | 2024-08-02 11:13:00
 4310873 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044397-00000  | 
failed    |       | 30256.129  | 2024-08-02 11:11:32
 4310872 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044396-00000  | 
failed    |       | 30256.128  | 2024-08-02 11:10:12
 4310871 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044395-00000  | 
failed    |       | 30256.127  | 2024-08-02 11:10:06
 4310870 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044394-00000  | 
failed    |       | 30256.126  | 2024-08-02 11:09:09
 4310869 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044382-00000  | 
failed    |       | 30256.125  | 2024-08-02 11:08:45
 4310868 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044381-00000  | 
failed    |       | 30256.124  | 2024-08-02 11:08:23
 4310867 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044380-00000  | 
failed    |       | 30256.123  | 2024-08-02 11:07:27
 4310866 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044379-00000  | 
failed    |       | 30256.122  | 2024-08-02 11:06:50
 4310865 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044378-00000  | 
failed    |       | 30256.121  | 2024-08-02 11:05:51
 4310864 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044377-00000  | 
failed    |       | 30256.120  | 2024-08-02 11:05:24
 4310863 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044373-00000  | 
failed    |       | 30256.119  | 2024-08-02 11:03:52
 4310862 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044372-00000  | 
failed    |       | 30256.118  | 2024-08-02 11:02:01
 4310861 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044371-00000  | 
failed    |       | 30256.117  | 2024-08-02 11:01:31
 4310860 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044370-00000  | 
failed    |       | 30256.116  | 2024-08-02 11:01:19
 4310859 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044369-00000  | 
failed    |       | 30256.115  | 2024-08-02 11:01:01
 4310858 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044368-00000  | 
failed    |       | 30256.114  | 2024-08-02 11:00:42
 4310857 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044367-00000  | 
failed    |       | 30256.113  | 2024-08-02 11:00:11
 4310856 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044366-00000  | 
failed    |       | 30256.112  | 2024-08-02 11:00:04
 4310855 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044365-00000  | 
failed    |       | 30256.111  | 2024-08-02 11:00:11
 4310854 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044364-00000  | 
failed    |       | 30256.110  | 2024-08-02 11:00:10
 4310853 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044363-00000  | 
failed    |       | 30256.109  | 2024-08-02 11:00:07
 4310852 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044362-00000  | 
failed    |       | 30256.108  | 2024-08-02 10:58:58
 4310851 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044361-00000  | 
failed    |       | 30256.107  | 2024-08-02 10:58:56
 4310850 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044360-00000  | 
failed    |       | 30256.106  | 2024-08-02 10:58:53
 4310849 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044359-00000  | 
failed    |       | 30256.105  | 2024-08-02 10:58:00
 4310848 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044300-00000  | 
failed    |       | 30256.104  | 2024-08-02 10:57:44
 4310846 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044298-00000  | 
failed    |       | 30256.102  | 2024-08-02 10:57:11
 4310840 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044291-00000  | 
failed    |       | 30256.96   | 2024-08-02 10:49:39
 4310839 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044290-00000  | 
failed    |       | 30256.95   | 2024-08-02 10:48:58
 4310838 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044289-00000  | 
failed    |       | 30256.94   | 2024-08-02 10:45:19
 4310835 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044286-00000  | 
failed    |       | 30256.91   | 2024-08-02 10:43:54
 4310833 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044284-00000  | 
failed    |       | 30256.89   | 2024-08-02 10:43:03
 4310826 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044269-00000  | 
failed    |       | 30256.82   | 2024-08-02 10:38:14
 4310824 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044237-00000  | 
failed    |       | 30256.80   | 2024-08-02 10:37:22
 4310810 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044221-00000  | 
failed    |       | 30256.66   | 2024-08-02 10:17:43
 4310805 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044209-00000  | 
failed    |       | 30256.61   | 2024-08-02 10:02:19
 4310804 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044208-00000  | 
failed    |       | 30256.60   | 2024-08-02 10:00:44
 4310803 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044207-00000  | 
failed    |       | 30256.59   | 2024-08-02 09:59:56
 4310802 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044206-00000  | 
failed    |       | 30256.58   | 2024-08-02 09:58:34
 4310799 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044138-00000  | 
failed    |       | 30256.55   | 2024-08-02 09:49:15
 4310754 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044038-00000  | 
failed    |       | 30256.10   | 2024-08-02 07:09:18
 4310753 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044037-00000  | 
failed    |       | 30256.9    | 2024-08-02 07:08:53
 4310752 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044036-00000  | 
failed    |       | 30256.8    | 2024-08-02 07:06:47
 4310749 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044008-00000  | 
failed    |       | 30256.5    | 2024-08-02 07:03:45
 4310748 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044007-00000  | 
failed    |       | 30256.4    | 2024-08-02 07:03:15
 4310747 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044006-00000  | 
failed    |       | 30256.3    | 2024-08-02 07:03:04
 4310746 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044002-00000  | 
failed    |       | 30256.2    | 2024-08-02 07:01:37
 4310745 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044001-00000  | 
failed    |       | 30256.1    | 2024-08-02 06:59:43
 4310744 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044000-00000  | 
failed    |       | 30256.0    | 2024-08-02 06:57:07
 4310117 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044713-00000  | 
failed    |       | 30253.93   | 2024-08-02 06:44:02
 4310101 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044696-00000  | 
failed    |       | 30253.77   | 2024-08-02 05:23:20
 4310096 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044688-00000  | 
failed    |       | 30253.72   | 2024-08-02 05:11:44
 4310085 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044641-00000  | 
failed    |       | 30253.61   | 2024-08-02 04:39:42
 4310082 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044637-00000  | 
failed    |       | 30253.58   | 2024-08-02 04:30:19
 4310077 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044628-00000  | 
failed    |       | 30253.53   | 2024-08-02 04:27:23
 4310071 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044620-00000  | 
failed    |       | 30253.47   | 2024-08-02 04:18:28
 4310068 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044617-00000  | 
failed    |       | 30253.44   | 2024-08-02 04:20:01
 4310066 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044615-00000  | 
failed    |       | 30253.42   | 2024-08-02 04:17:13
 4310058 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044607-00000  | 
failed    |       | 30253.34   | 2024-08-02 04:18:01
 4310057 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044606-00000  | 
failed    |       | 30253.33   | 2024-08-02 04:16:44
 4310053 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044589-00000  | 
failed    |       | 30253.29   | 2024-08-02 04:16:44
 4310052 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044582-00000  | 
failed    |       | 30253.28   | 2024-08-02 04:16:44
 4310051 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044581-00000  | 
failed    |       | 30253.27   | 2024-08-02 04:16:44
 4310049 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044543-00000  | 
failed    |       | 30253.25   | 2024-08-02 04:16:44
 4310042 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044526-00000  | 
failed    |       | 30253.18   | 2024-08-02 04:15:32
 4310026 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044502-00000  | 
failed    |       | 30253.2    | 2024-08-02 02:06:30
 4310025 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044501-00000  | 
failed    |     5 | 30253.1    | 2024-08-02 02:05:56
 4310024 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00044500-00000  | 
failed    |       | 30253.0    | 2024-08-02 02:04:19
 4309595 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045492-00000  | 
failed    |       | 30250.174  | 2024-08-02 02:00:26
 4309587 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045484-00000  | 
failed    |       | 30250.166  | 2024-08-02 01:49:32
 4309586 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045477-00000  | 
failed    |       | 30250.165  | 2024-08-02 01:48:59
 4309584 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045458-00000  | 
failed    |       | 30250.163  | 2024-08-02 01:45:10
 4309582 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045445-00000  | 
failed    |       | 30250.161  | 2024-08-02 01:40:45
 4309579 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045441-00000  | 
failed    |       | 30250.158  | 2024-08-02 01:39:27
 4309578 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045440-00000  | 
failed    |       | 30250.157  | 2024-08-02 01:37:47
 4309574 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045413-00000  | 
failed    |       | 30250.153  | 2024-08-02 01:28:37
 4309573 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045412-00000  | 
failed    |       | 30250.152  | 2024-08-02 01:21:43
 4309556 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045324-00000  | 
failed    |       | 30250.135  | 2024-08-02 00:56:43
 4309554 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045322-00000  | 
failed    |       | 30250.133  | 2024-08-02 00:54:38
 4309553 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045321-00000  | 
failed    |       | 30250.132  | 2024-08-02 00:54:38
 4309547 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045314-00000  | 
failed    |       | 30250.126  | 2024-08-02 00:50:54
 4309546 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045313-00000  | 
failed    |       | 30250.125  | 2024-08-02 00:45:27
 4309542 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045289-00000  | 
failed    |       | 30250.121  | 2024-08-02 00:44:28
 4309537 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045273-00000  | 
failed    |       | 30250.116  | 2024-08-02 00:29:26
 4309535 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045259-00000  | 
failed    |       | 30250.114  | 2024-08-02 00:29:25
 4309529 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045251-00000  | 
failed    |       | 30250.108  | 2024-08-02 00:20:01
 4309519 | DST_TRIGGERED_EVENT_run2pp_ana430_2024p007-00045241-00000  | 
failed    |       | 30250.98   | 2024-08-02 00:09:37