[15:50:21.017] D [ sys ] loaded plugin #1 "WireCellPgraph" from library "libWireCellPgraph.so": 0x1f61070 [15:50:21.017] D [ sys ] loaded plugin #2 "WireCellGen" from library "libWireCellGen.so": 0x1f78560 [15:50:21.020] D [ sys ] loaded plugin #3 "WireCellSio" from library "libWireCellSio.so": 0x292e0e0 [15:50:21.189] D [ sys ] loaded plugin #4 "WireCellRoot" from library "libWireCellRoot.so": 0x21ffeb0 [15:50:21.250] D [ sys ] loaded plugin #5 "WireCellLarsoft" from library "libWireCellLarsoft.so": 0x1edb950 [15:50:21.269] D [ sys ] loaded plugin #6 "WireCellHio" from library "libWireCellHio.so": 0x26e58e0 [15:50:21.282] D [ sys ] loaded plugin #7 "WireCellTbb" from library "libWireCellTbb.so": 0x26e6270 [15:50:21.291] D [ sys ] loaded plugin #8 "WireCellImg" from library "libWireCellImg.so": 0x26e6370 [15:50:21.668] D [ sys ] loaded plugin #9 "WireCellPytorch" from library "libWireCellPytorch.so": 0x3370f40 [15:50:21.694] D [ sim ] Drifter: time offset: 0.0 ms, drift speed: 1.6 mm/us [15:50:21.695] I [ wct ] loading compressed json file: /wcdo/share/wirecell/data/protodune-wires-larsoft-v4.json.bz2 [15:50:23.394] D [ geom ] AnodePlane: X planes: "cathode"@ -0.025400000000000093m, "response"@-3.4941375m, "anode"@-3.57985m [15:50:23.396] D [ geom ] AnodePlane: face:0, plane:0, origin:true mm [15:50:23.399] D [ geom ] AnodePlane: face:0, plane:1, origin:true mm [15:50:23.399] D [ geom ] AnodePlane: face:0, plane:2, origin:true mm [15:50:23.400] D [ geom ] AnodePlane: face:0 with 3 planes and sensvol: [(-3579.85 76.1 3.35) --> (-25.4 6060 2303.37)] [15:50:23.400] D [ geom ] anode 0 face 1 is not sensitive [15:50:23.400] D [ geom ] AnodePlane: X planes: "cathode"@ 0.0m, "response"@0.0m, "anode"@0.0m [15:50:23.401] D [ geom ] AnodePlane: face:1, plane:0, origin:true mm [15:50:23.403] D [ geom ] AnodePlane: face:1, plane:1, origin:true mm [15:50:23.404] D [ geom ] AnodePlane: face:1, plane:2, origin:true mm [15:50:23.404] D [ geom ] AnodePlane: face:1 with 3 planes and sensvol: [(0 0 0) --> (0 0 0)] [15:50:23.405] I [ wct ] loading compressed json file: /wcdo/share/wirecell/data/dune-garfield-1d565.json.bz2 [15:50:24.294] D [ geom ] PIR: short response ColdElecResponse has different number of samples (6125) than expected (200) [15:50:24.296] D [ geom ] PIR: long response RCResponse has different number of samples (6125) than expected (6174) [15:50:24.300] D [ geom ] PIR: short response ColdElecResponse has different number of samples (6125) than expected (200) [15:50:24.300] D [ geom ] PIR: long response RCResponse has different number of samples (6125) than expected (6174) [15:50:24.302] D [ geom ] PIR: short response ColdElecResponse has different number of samples (6125) than expected (200) [15:50:24.302] D [ geom ] PIR: long response RCResponse has different number of samples (6125) than expected (6174) [15:50:24.304] D [ sim ] AnodePlane: AnodePlane:apa0, mode: fixed, fluctuate: on, time start: -0.3125 ms, readout time: 3.0625 ms, frame start: 100 [15:50:24.305] D [ sim ] Gen::Reframer: using anode: "AnodePlane:apa0" [15:50:24.305] D [ sim ] EmpiricalNoiseModel: using anode AnodePlane:apa0, chanstat [15:50:24.305] I [ wct ] loading compressed json file: /wcdo/share/wirecell/data/protodune-noise-spectra-v1.json.bz2 [15:50:24.310] D [ sim ] AddNoise: using IRandom: "Random", IChannelSpectrum: "EmpiricalNoiseModel:empericalnoise-apa0" [15:50:24.310] D [ sim ] Gen::Digitizer: tag="orig0", resolution=12 bits, maxvalue=4096 counts, gain=1, fullscale=[200,1600] mV, baselines=[1003,1003,508] mV [15:50:24.333] D [sigproc ] OmniChannelNoiseDB: setting 134:[1,13363] bad channels [15:50:24.397] D [sigproc ] OmnibusNoiseFilter: adding channel filter: 0 "pdStickyCodeMitig:nf0" [15:50:24.397] D [sigproc ] OmnibusNoiseFilter: adding channel filter: 1 "pdOneChannelNoise:nf0" [15:50:24.397] D [sigproc ] OmnibusNoiseFilter: adding channel filter: 2 "pdRelGainCalib:nf0" [15:50:24.397] D [sigproc ] OmnibusNoiseFilter: adding grouped filter: 0 "mbCoherentNoiseSub:nf0" [15:50:24.397] D [sigproc ] OmnibusNoiseFilter: using channel noise DB object: "OmniChannelNoiseDB:ocndbperfect0" [15:50:24.399] D [sigproc ] OmnibusSigproc: internal channel map for tags: gauss:"gauss0", wiener:"wiener0", frame:"sigproc" pind0 aid0 fid0 pid0 cid400 -> cid799, cind0 -> cind399, (n=400) pind1 aid0 fid0 pid1 cid1599 -> cid1200, cind0 -> cind399, (n=400) pind2 aid0 fid0 pid2 cid2080 -> cid2559, cind0 -> cind479, (n=480) pind0 aid0 fid1 pid0 cid0 -> cid399, cind0 -> cind399, (n=800) pind1 aid0 fid1 pid1 cid1199 -> cid800, cind0 -> cind399, (n=800) pind2 aid0 fid1 pid2 cid2079 -> cid1600, cind0 -> cind479, (n=960) [15:50:24.549] I [pytorch ] Model: ts-model/unet-l23-cosmic500-e50.ts loaded [15:50:24.549] D [ pgraph ] connecting: 17 edges [15:50:26.288] D [ pgraph ] executing with 18 nodes [15:50:59.243] D [ sim ] xregion: anode: -3579.85 mm, response: -3494.1375 mm, cathode: -25.40000000000009 mm, flushing 20239 [15:50:59.247] D [ sim ] xregion: anode: 0.0 mm, response: 0.0 mm, cathode: 0.0 mm, flushing 0 [15:50:59.247] D [ sim ] xregion: anode: 3579.85 mm, response: 3494.1375 mm, cathode: 25.40000000000009 mm, flushing 10384 [15:50:59.257] D [ sim ] at EOS, dropped 2178 / 508515 depos from stream, outside of all 3 drift regions [15:51:00.646] D [ sim ] anode: 0, face: 0, processing 113655 depos spanning t:[-3.6981772395868826,6.106999022895617]ms, bb:[true-->true]cm [15:51:00.646] D [ sim ] anode: 0, face: 0, dropped 392682 depos spanning t:[-3.7151678266569417,6.106667616715686]ms, outside bb:[true-->true]cm [15:51:01.124] D [ sim ] anode: 0 face: 1 is marked insensitive, skipping [15:51:01.124] D [ sim ] made frame: 100 with 2017 traces @ -0.3125ms [15:51:01.194] D [ io ] AnodePlane:apa0: t: 0 - 5999; c: 0 - 2559 [15:51:01.195] D [ io ] HDF5FrameTap: chunking ncols=0 nrows=0 [15:51:01.195] D [ io ] HDF5FrameTap: gzip_level 2 [15:51:01.195] D [ io ] HDF5FrameTap: see frame #100 with 2017 traces with frame tags: "ductor" and trace tags: "ductor0" looking for tags: "ductor0" [15:51:01.208] D [ io ] HDF5FrameTap: save 2017 tagged as ductor0 [15:51:01.208] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:51:01.208] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:51:01.500] D [ io ] HDF5FrameTap: saved /100/frame_ductor0 with 2560 channels 6000 ticks @t=-0.3125 ms qtot=823010560.0 [15:51:01.502] D [ glue ] sink frame: #100 @-0.3125 with 2017 traces, frame tags:[ductor], trace tags:[ductor0] [15:51:01.505] D [ io ] HDF5FrameTap: EOS [15:51:01.505] D [ glue ] frame sink sees EOS [15:51:01.536] D [ sim ] anode: 0, face: 0, processing 68201 depos spanning t:[-0.31241389092766286,2.7499750240984193]ms, bb:[true-->true]cm [15:51:01.536] D [ sim ] anode: 0, face: 0, dropped 212507 depos spanning t:[-0.31249781867049836,2.74981670532275]ms, outside bb:[true-->true]cm [15:51:11.243] D [ sim ] ImpactTransform: # of channels: 1120 # of ticks: 6655 [15:51:20.930] D [ sim ] ImpactTransform: # of channels: 1280 # of ticks: 6655 [15:51:29.101] D [ sim ] ImpactTransform: # of channels: 512 # of ticks: 6655 [15:51:29.306] D [ sim ] anode 0 face 1 is marked insensitive, skipping [15:51:29.417] D [ sim ] Gen::Reframer: frame 100 2560 traces, 6000 ticks [15:51:29.958] D [ io ] AnodePlane:apa0: t: 0 - 5999; c: 0 - 2559 [15:51:29.958] D [ io ] HDF5FrameTap: chunking ncols=0 nrows=0 [15:51:29.958] D [ io ] HDF5FrameTap: gzip_level 2 [15:51:29.958] D [ io ] HDF5FrameTap: see frame #100 with 2560 traces with frame tags: "orig0" and trace tags: looking for tags: "orig0" [15:51:29.958] D [ io ] HDF5FrameTap: save 2560 tagged as orig0 [15:51:29.958] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:51:29.958] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:51:31.319] D [ io ] HDF5FrameTap: saved /100/frame_orig0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=27586873000.0 [15:51:31.321] D [sigproc ] OmnibusNoiseFilter: nticks based on first waveform: 6000 [15:51:34.266] D [sigproc ] OmnibusSigProc: receive frame 100 with 2560 traces [15:51:34.267] D [sigproc ] OmnibusSigProc: nticks=6000 tbinmin=0 tbinmax=6000 [15:51:34.294] D [sigproc ] OmnibusSigProc: plane index: 0 input data identifies 10 bad regions [15:51:35.218] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 24042730.449247487 traces to total 1580 indices:[1580,0] [15:51:35.435] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 126914610.42210872 traces to total 1578 indices:[1578,1580] [15:51:35.910] D [sigproc ] OmnibusSigProc: plane index: 1 input data identifies 9 bad regions [15:51:36.821] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 34588698.888319686 traces to total 4739 indices:[3161,0] [15:51:37.038] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 120673122.58296 traces to total 4741 indices:[3160,1580] [15:51:37.513] D [sigproc ] OmnibusSigProc: plane index: 2 input data identifies 18 bad regions [15:51:38.412] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 249645721.2067405 traces to total 8232 indices:[5072,0] [15:51:38.456] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 249645721.2067405 traces to total 8563 indices:[5071,1580] [15:51:38.496] I [sigproc ] ROI_refinement::MultiPlaneProtection: [15:51:38.643] I [sigproc ] ROI_refinement::MultiPlaneROI: [15:51:45.492] I [sigproc ] ROI_refinement::MultiPlaneProtection: [15:51:45.649] I [sigproc ] ROI_refinement::MultiPlaneROI: [15:52:07.249] I [sigproc ] ROI_refinement::MultiPlaneProtection: [15:52:07.305] I [sigproc ] ROI_refinement::MultiPlaneROI: [15:52:33.533] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 242694249.0 traces to total 2872 indices:[2872,57656] [15:52:33.683] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 0.03173968009650707 traces to total 790 indices:[790,60528] [15:52:33.728] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 236743675.0 traces to total 2881 indices:[2881,61318] [15:52:34.097] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 240820725.0 traces to total 3229 indices:[3229,76890] [15:52:34.237] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added -0.04707450792193413 traces to total 20382 indices:[1581,60528] [15:52:34.292] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 234199608.0 traces to total 22819 indices:[6108,61318] [15:52:34.617] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 265206140.0 traces to total 2722 indices:[2722,95587] [15:52:34.870] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 249469236.83842817 traces to total 39692 indices:[3492,60528] [15:52:34.930] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 265250743.0 traces to total 41617 indices:[8823,61318] [15:52:34.932] D [sigproc ] OmnibusSigProc: produce 102935 traces: 8823 wiener0, 3492 decon_charge0, 8823 gauss0, frame tag: sigproc [15:52:35.044] I [pytorch ] frame2eigen: 0.04 [15:52:35.063] I [pytorch ] eigen2tensor: 0.0 [15:52:36.766] I [pytorch ] thread_wait_time: 0.0 sec [15:52:36.766] I [pytorch ] forward: 19.15 [15:52:36.777] I [pytorch ] tensor2eigen: 0.01 [15:52:36.827] I [pytorch ] DNNROIFinding: produce 800 traces: 800 [15:52:36.827] I [pytorch ] eigen2frame: 0.0 [15:52:36.843] D [ io ] AnodePlane:apa0: t: 0 - 5999; c: 0 - 2559 [15:52:36.843] D [ io ] HDF5FrameTap: chunking ncols=0 nrows=0 [15:52:36.843] D [ io ] HDF5FrameTap: gzip_level 2 [15:52:36.843] D [ io ] HDF5FrameTap: see frame #100 with 800 traces with frame tags: "DNNROIFinding" and trace tags: "dnn_sp0" looking for tags: "loose_lf0" "tight_lf0" "cleanup_roi0" "break_roi_1st0" "break_roi_2nd0" "shrink_roi0" "extend_roi0" "mp3_roi0" "mp2_roi0" "decon_charge0" "gauss0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as loose_lf0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "loose_lf0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as tight_lf0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "tight_lf0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as cleanup_roi0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "cleanup_roi0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as break_roi_1st0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "break_roi_1st0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as break_roi_2nd0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "break_roi_2nd0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as shrink_roi0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "shrink_roi0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as extend_roi0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "extend_roi0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as mp3_roi0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "mp3_roi0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as mp2_roi0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "mp2_roi0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as decon_charge0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "decon_charge0" [15:52:36.843] D [ io ] HDF5FrameTap: save 0 tagged as gauss0 [15:52:36.843] W [ io ] HDF5FrameTap: no traces for tag: "gauss0" [15:52:36.843] D [ glue ] sink frame: #100 @-0.25 with 800 traces, frame tags:[DNNROIFinding], trace tags:[dnn_sp0] [15:52:36.843] D [ sim ] Gen::Digitizer: EOS [15:52:36.843] D [ io ] HDF5FrameTap: EOS [15:52:36.843] D [sigproc ] OmnibusSigProc: see EOS [15:52:36.843] D [pytorch ] DNNROIFinding: EOS [15:52:36.843] D [ io ] HDF5FrameTap: EOS [15:52:36.843] D [ glue ] frame sink sees EOS [15:52:36.843] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 61.4 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::DepoTransform : 28.529999 sec [15:52:36.843] I [ timer ] Timer: WireCell::Pytorch::DNNROIFinding : 19.869999 sec [15:52:36.843] I [ timer ] Timer: WireCell::SigProc::OmnibusNoiseFilter : 3.5299997 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::DepoSplat : 2.4199986 sec [15:52:36.843] I [ timer ] Timer: WireCell::Hio::HDF5FrameTap : 2.0799997 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::Drifter : 1.9099985 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::DepoFanout : 1.6999987 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::DepoBagger : 1.5899988 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::DepoFanout : 1.439999 sec [15:52:36.843] I [ timer ] Timer: WireCell::Hio::HDF5FrameTap : 1.1199995 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.98999935 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.9399997 sec [15:52:36.843] I [ timer ] Timer: wcls::SimDepoSource : 0.79999954 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::AddNoise : 0.7899998 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::Reframer : 0.7199996 sec [15:52:36.843] I [ timer ] Timer: WireCell::Hio::HDF5FrameTap : 0.5999997 sec [15:52:36.843] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.58999974 sec [15:52:36.843] I [ timer ] Timer: Total node execution : 131.01998937129974 sec [15:56:23.906] D [ sys ] loaded plugin #1 "WireCellPgraph" from library "libWireCellPgraph.so": 0x127a070 [15:56:23.906] D [ sys ] loaded plugin #2 "WireCellGen" from library "libWireCellGen.so": 0x1291560 [15:56:23.909] D [ sys ] loaded plugin #3 "WireCellSio" from library "libWireCellSio.so": 0x1c48940 [15:56:24.074] D [ sys ] loaded plugin #4 "WireCellRoot" from library "libWireCellRoot.so": 0x133b0a0 [15:56:24.135] D [ sys ] loaded plugin #5 "WireCellLarsoft" from library "libWireCellLarsoft.so": 0x11f4ba0 [15:56:24.151] D [ sys ] loaded plugin #6 "WireCellHio" from library "libWireCellHio.so": 0x1a02c20 [15:56:24.163] D [ sys ] loaded plugin #7 "WireCellTbb" from library "libWireCellTbb.so": 0x1a035b0 [15:56:24.173] D [ sys ] loaded plugin #8 "WireCellImg" from library "libWireCellImg.so": 0x1a036b0 [15:56:24.551] D [ sys ] loaded plugin #9 "WireCellPytorch" from library "libWireCellPytorch.so": 0x268abd0 [15:56:24.582] D [ sim ] Drifter: time offset: 0.0 ms, drift speed: 1.6 mm/us [15:56:24.582] I [ wct ] loading compressed json file: /wcdo/share/wirecell/data/protodune-wires-larsoft-v4.json.bz2 [15:56:26.272] D [ geom ] AnodePlane: X planes: "cathode"@ -0.025400000000000093m, "response"@-3.4941375m, "anode"@-3.57985m [15:56:26.275] D [ geom ] AnodePlane: face:0, plane:0, origin:true mm [15:56:26.277] D [ geom ] AnodePlane: face:0, plane:1, origin:true mm [15:56:26.278] D [ geom ] AnodePlane: face:0, plane:2, origin:true mm [15:56:26.278] D [ geom ] AnodePlane: face:0 with 3 planes and sensvol: [(-3579.85 76.1 3.35) --> (-25.4 6060 2303.37)] [15:56:26.278] D [ geom ] anode 0 face 1 is not sensitive [15:56:26.278] D [ geom ] AnodePlane: X planes: "cathode"@ 0.0m, "response"@0.0m, "anode"@0.0m [15:56:26.280] D [ geom ] AnodePlane: face:1, plane:0, origin:true mm [15:56:26.283] D [ geom ] AnodePlane: face:1, plane:1, origin:true mm [15:56:26.283] D [ geom ] AnodePlane: face:1, plane:2, origin:true mm [15:56:26.284] D [ geom ] AnodePlane: face:1 with 3 planes and sensvol: [(0 0 0) --> (0 0 0)] [15:56:26.284] I [ wct ] loading compressed json file: /wcdo/share/wirecell/data/dune-garfield-1d565.json.bz2 [15:56:27.173] D [ geom ] PIR: short response ColdElecResponse has different number of samples (6125) than expected (200) [15:56:27.175] D [ geom ] PIR: long response RCResponse has different number of samples (6125) than expected (6174) [15:56:27.178] D [ geom ] PIR: short response ColdElecResponse has different number of samples (6125) than expected (200) [15:56:27.179] D [ geom ] PIR: long response RCResponse has different number of samples (6125) than expected (6174) [15:56:27.180] D [ geom ] PIR: short response ColdElecResponse has different number of samples (6125) than expected (200) [15:56:27.180] D [ geom ] PIR: long response RCResponse has different number of samples (6125) than expected (6174) [15:56:27.183] D [ sim ] AnodePlane: AnodePlane:apa0, mode: fixed, fluctuate: on, time start: -0.3125 ms, readout time: 3.0625 ms, frame start: 100 [15:56:27.191] D [ sim ] Gen::Reframer: using anode: "AnodePlane:apa0" [15:56:27.192] D [ sim ] EmpiricalNoiseModel: using anode AnodePlane:apa0, chanstat [15:56:27.192] I [ wct ] loading compressed json file: /wcdo/share/wirecell/data/protodune-noise-spectra-v1.json.bz2 [15:56:27.200] D [ sim ] AddNoise: using IRandom: "Random", IChannelSpectrum: "EmpiricalNoiseModel:empericalnoise-apa0" [15:56:27.200] D [ sim ] Gen::Digitizer: tag="orig0", resolution=12 bits, maxvalue=4096 counts, gain=1, fullscale=[200,1600] mV, baselines=[1003,1003,508] mV [15:56:27.224] D [sigproc ] OmniChannelNoiseDB: setting 134:[1,13363] bad channels [15:56:27.284] D [sigproc ] OmnibusNoiseFilter: adding channel filter: 0 "pdStickyCodeMitig:nf0" [15:56:27.284] D [sigproc ] OmnibusNoiseFilter: adding channel filter: 1 "pdOneChannelNoise:nf0" [15:56:27.284] D [sigproc ] OmnibusNoiseFilter: adding channel filter: 2 "pdRelGainCalib:nf0" [15:56:27.284] D [sigproc ] OmnibusNoiseFilter: adding grouped filter: 0 "mbCoherentNoiseSub:nf0" [15:56:27.284] D [sigproc ] OmnibusNoiseFilter: using channel noise DB object: "OmniChannelNoiseDB:ocndbperfect0" [15:56:27.287] D [sigproc ] OmnibusSigproc: internal channel map for tags: gauss:"gauss0", wiener:"wiener0", frame:"sigproc" pind0 aid0 fid0 pid0 cid400 -> cid799, cind0 -> cind399, (n=400) pind1 aid0 fid0 pid1 cid1599 -> cid1200, cind0 -> cind399, (n=400) pind2 aid0 fid0 pid2 cid2080 -> cid2559, cind0 -> cind479, (n=480) pind0 aid0 fid1 pid0 cid0 -> cid399, cind0 -> cind399, (n=800) pind1 aid0 fid1 pid1 cid1199 -> cid800, cind0 -> cind399, (n=800) pind2 aid0 fid1 pid2 cid2079 -> cid1600, cind0 -> cind479, (n=960) [15:56:27.436] I [pytorch ] Model: ts-model/unet-l23-cosmic500-e50.ts loaded [15:56:27.437] D [ pgraph ] connecting: 18 edges [15:56:29.183] D [ pgraph ] executing with 19 nodes [15:57:03.398] D [ sim ] xregion: anode: -3579.85 mm, response: -3494.1375 mm, cathode: -25.40000000000009 mm, flushing 20239 [15:57:03.402] D [ sim ] xregion: anode: 0.0 mm, response: 0.0 mm, cathode: 0.0 mm, flushing 0 [15:57:03.402] D [ sim ] xregion: anode: 3579.85 mm, response: 3494.1375 mm, cathode: 25.40000000000009 mm, flushing 10384 [15:57:03.412] D [ sim ] at EOS, dropped 2178 / 508515 depos from stream, outside of all 3 drift regions [15:57:04.748] D [ sim ] anode: 0, face: 0, processing 113655 depos spanning t:[-3.6981772395868826,6.106999022895617]ms, bb:[true-->true]cm [15:57:04.748] D [ sim ] anode: 0, face: 0, dropped 392682 depos spanning t:[-3.7151678266569417,6.106667616715686]ms, outside bb:[true-->true]cm [15:57:05.191] D [ sim ] anode: 0 face: 1 is marked insensitive, skipping [15:57:05.191] D [ sim ] made frame: 100 with 2017 traces @ -0.3125ms [15:57:05.257] D [ io ] AnodePlane:apa0: t: 0 - 5999; c: 0 - 2559 [15:57:05.257] D [ io ] HDF5FrameTap: chunking ncols=0 nrows=0 [15:57:05.257] D [ io ] HDF5FrameTap: gzip_level 2 [15:57:05.258] D [ io ] HDF5FrameTap: see frame #100 with 2017 traces with frame tags: "ductor" and trace tags: "ductor0" looking for tags: "ductor0" [15:57:05.265] D [ io ] HDF5FrameTap: save 2017 tagged as ductor0 [15:57:05.266] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:57:05.266] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:57:05.568] D [ io ] HDF5FrameTap: saved /100/frame_ductor0 with 2560 channels 6000 ticks @t=-0.3125 ms qtot=823010560.0 [15:57:05.570] D [ glue ] sink frame: #100 @-0.3125 with 2017 traces, frame tags:[ductor], trace tags:[ductor0] [15:57:05.574] D [ io ] HDF5FrameTap: EOS [15:57:05.574] D [ glue ] frame sink sees EOS [15:57:05.604] D [ sim ] anode: 0, face: 0, processing 68201 depos spanning t:[-0.31241389092766286,2.7499750240984193]ms, bb:[true-->true]cm [15:57:05.604] D [ sim ] anode: 0, face: 0, dropped 212507 depos spanning t:[-0.31249781867049836,2.74981670532275]ms, outside bb:[true-->true]cm [15:57:15.482] D [ sim ] ImpactTransform: # of channels: 1120 # of ticks: 6655 [15:57:25.326] D [ sim ] ImpactTransform: # of channels: 1280 # of ticks: 6655 [15:57:33.529] D [ sim ] ImpactTransform: # of channels: 512 # of ticks: 6655 [15:57:33.750] D [ sim ] anode 0 face 1 is marked insensitive, skipping [15:57:33.873] D [ sim ] Gen::Reframer: frame 100 2560 traces, 6000 ticks [15:57:34.409] D [ io ] AnodePlane:apa0: t: 0 - 5999; c: 0 - 2559 [15:57:34.409] D [ io ] HDF5FrameTap: chunking ncols=0 nrows=0 [15:57:34.409] D [ io ] HDF5FrameTap: gzip_level 2 [15:57:34.409] D [ io ] HDF5FrameTap: see frame #100 with 2560 traces with frame tags: "orig0" and trace tags: looking for tags: "orig0" [15:57:34.410] D [ io ] HDF5FrameTap: save 2560 tagged as orig0 [15:57:34.410] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:57:34.410] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:57:35.760] D [ io ] HDF5FrameTap: saved /100/frame_orig0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=27586908000.0 [15:57:35.763] D [sigproc ] OmnibusNoiseFilter: nticks based on first waveform: 6000 [15:57:38.800] D [sigproc ] OmnibusSigProc: receive frame 100 with 2560 traces [15:57:38.800] D [sigproc ] OmnibusSigProc: nticks=6000 tbinmin=0 tbinmax=6000 [15:57:38.826] D [sigproc ] OmnibusSigProc: plane index: 0 input data identifies 10 bad regions [15:57:39.754] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 24502438.204114124 traces to total 1579 indices:[1579,0] [15:57:39.969] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 130551491.45400155 traces to total 1574 indices:[1574,1579] [15:57:40.447] D [sigproc ] OmnibusSigProc: plane index: 1 input data identifies 9 bad regions [15:57:41.383] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 35539146.46949897 traces to total 4731 indices:[3157,0] [15:57:41.597] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 119189640.51615334 traces to total 4733 indices:[3155,1579] [15:57:42.072] D [sigproc ] OmnibusSigProc: plane index: 2 input data identifies 20 bad regions [15:57:42.978] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 248848108.443191 traces to total 8224 indices:[5069,0] [15:57:43.020] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 248848108.443191 traces to total 8557 indices:[5067,1579] [15:57:43.054] I [sigproc ] ROI_refinement::MultiPlaneProtection: [15:57:43.207] I [sigproc ] ROI_refinement::MultiPlaneROI: [15:57:51.008] I [sigproc ] ROI_refinement::MultiPlaneProtection: [15:57:51.166] I [sigproc ] ROI_refinement::MultiPlaneROI: [15:58:13.620] I [sigproc ] ROI_refinement::MultiPlaneProtection: [15:58:13.669] I [sigproc ] ROI_refinement::MultiPlaneROI: [15:58:41.353] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 244997443.0 traces to total 2864 indices:[2864,57203] [15:58:41.479] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added -0.10116347949951887 traces to total 790 indices:[790,60067] [15:58:41.534] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=0 added 238674287.0 traces to total 2873 indices:[2873,60857] [15:58:41.906] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 247767802.0 traces to total 3255 indices:[3255,76585] [15:58:42.046] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 0.02145703975111246 traces to total 20564 indices:[1581,60067] [15:58:42.102] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=1 added 238459278.0 traces to total 23034 indices:[6133,60857] [15:58:42.416] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 264702716.0 traces to total 2708 indices:[2708,95333] [15:58:42.672] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 248647524.68974727 traces to total 39887 indices:[3494,60067] [15:58:42.726] D [sigproc ] OmnibusSigProc::save_data plane index: Qtot=2 added 264756064.0 traces to total 41807 indices:[8843,60857] [15:58:42.728] D [sigproc ] OmnibusSigProc: produce 102664 traces: 8827 wiener0, 3494 decon_charge0, 8843 gauss0, frame tag: sigproc [15:58:42.806] D [ io ] AnodePlane:apa0: t: 0 - 5999; c: 0 - 2559 [15:58:42.806] D [ io ] HDF5FrameTap: chunking ncols=0 nrows=0 [15:58:42.806] D [ io ] HDF5FrameTap: gzip_level 2 [15:58:42.806] D [ io ] HDF5FrameTap: see frame #100 with 102664 traces with frame tags: "sigproc" and trace tags: "break_roi_1st0" "break_roi_2nd0" "cleanup_roi0" "decon_charge0" "extend_roi0" "gauss0" "loose_lf0" "mp2_roi0" "mp3_roi0" "shrink_roi0" "threshold0" "tight_lf0" "wiener0" looking for tags: "loose_lf0" "tight_lf0" "cleanup_roi0" "break_roi_1st0" "break_roi_2nd0" "shrink_roi0" "extend_roi0" "mp3_roi0" "mp2_roi0" "decon_charge0" "gauss0" [15:58:42.806] D [ io ] HDF5FrameTap: save 5067 tagged as loose_lf0 [15:58:42.806] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:42.806] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:44.626] D [ io ] HDF5FrameTap: saved /100/frame_loose_lf0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=512222530.0 [15:58:44.629] D [ io ] HDF5FrameTap: save 5069 tagged as tight_lf0 [15:58:44.629] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:44.629] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:46.487] D [ io ] HDF5FrameTap: saved /100/frame_tight_lf0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=312962370.0 [15:58:46.489] D [ io ] HDF5FrameTap: save 8184 tagged as cleanup_roi0 [15:58:46.489] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:46.489] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:46.676] D [ io ] HDF5FrameTap: saved /100/frame_cleanup_roi0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=4853550.0 [15:58:46.679] D [ io ] HDF5FrameTap: save 8914 tagged as break_roi_1st0 [15:58:46.679] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:46.679] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:46.862] D [ io ] HDF5FrameTap: saved /100/frame_break_roi_1st0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=4691860.0 [15:58:46.865] D [ io ] HDF5FrameTap: save 9346 tagged as break_roi_2nd0 [15:58:46.865] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:46.865] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:47.041] D [ io ] HDF5FrameTap: saved /100/frame_break_roi_2nd0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=4650160.0 [15:58:47.044] D [ io ] HDF5FrameTap: save 9324 tagged as shrink_roi0 [15:58:47.044] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:47.044] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:47.199] D [ io ] HDF5FrameTap: saved /100/frame_shrink_roi0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=3304050.0 [15:58:47.202] D [ io ] HDF5FrameTap: save 8667 tagged as extend_roi0 [15:58:47.202] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:47.202] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:47.383] D [ io ] HDF5FrameTap: saved /100/frame_extend_roi0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=3921510.0 [15:58:47.385] D [ io ] HDF5FrameTap: save 4989 tagged as mp3_roi0 [15:58:47.385] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:47.385] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:47.563] D [ io ] HDF5FrameTap: saved /100/frame_mp3_roi0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=583395968.0 [15:58:47.566] D [ io ] HDF5FrameTap: save 21940 tagged as mp2_roi0 [15:58:47.566] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:47.566] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:47.739] D [ io ] HDF5FrameTap: saved /100/frame_mp2_roi0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=3146639872.0 [15:58:47.742] D [ io ] HDF5FrameTap: save 3494 tagged as decon_charge0 [15:58:47.742] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:47.742] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:49.571] D [ io ] HDF5FrameTap: saved /100/frame_decon_charge0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=465162180.0 [15:58:49.575] D [ io ] HDF5FrameTap: save 8843 tagged as gauss0 [15:58:49.575] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:49.575] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:49.783] D [ io ] HDF5FrameTap: saved /100/frame_gauss0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=745763460.0 [15:58:49.819] I [pytorch ] frame2eigen: 0.04 [15:58:49.851] I [pytorch ] eigen2tensor: 0.08 [15:58:51.512] I [pytorch ] thread_wait_time: 0.0 sec [15:58:51.512] I [pytorch ] forward: 19.04 [15:58:51.523] I [pytorch ] tensor2eigen: 0.01 [15:58:51.572] I [pytorch ] DNNROIFinding: produce 800 traces: 800 [15:58:51.572] I [pytorch ] eigen2frame: 0.01 [15:58:51.589] D [ io ] AnodePlane:apa0: t: 0 - 5999; c: 0 - 2559 [15:58:51.589] D [ io ] HDF5FrameTap: chunking ncols=0 nrows=0 [15:58:51.589] D [ io ] HDF5FrameTap: gzip_level 2 [15:58:51.589] D [ io ] HDF5FrameTap: see frame #100 with 800 traces with frame tags: "DNNROIFinding" and trace tags: "dnn_sp0" looking for tags: "dnn_sp0" [15:58:51.589] D [ io ] HDF5FrameTap: save 800 tagged as dnn_sp0 [15:58:51.589] D [ io ] HDF5FrameTap: saving ncols=6000 nrows=2560 [15:58:51.589] D [ io ] HDF5FrameTap: chunking ncols=6000 nrows=2560 [15:58:51.785] D [ io ] HDF5FrameTap: saved /100/frame_dnn_sp0 with 2560 channels 6000 ticks @t=-0.25 ms qtot=216233220.0 [15:58:51.785] D [ glue ] sink frame: #100 @-0.25 with 800 traces, frame tags:[DNNROIFinding], trace tags:[dnn_sp0] [15:58:51.785] D [ sim ] Gen::Digitizer: EOS [15:58:51.785] D [ io ] HDF5FrameTap: EOS [15:58:51.785] D [sigproc ] OmnibusSigProc: see EOS [15:58:51.785] D [ io ] HDF5FrameTap: EOS [15:58:51.785] D [pytorch ] DNNROIFinding: EOS [15:58:51.785] D [ io ] HDF5FrameTap: EOS [15:58:51.785] D [ glue ] frame sink sees EOS [15:58:51.786] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 64.68 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::DepoTransform : 28.85 sec [15:58:51.786] I [ timer ] Timer: WireCell::Pytorch::DNNROIFinding : 19.77 sec [15:58:51.786] I [ timer ] Timer: WireCell::Hio::HDF5FrameTap : 7.6099997 sec [15:58:51.786] I [ timer ] Timer: WireCell::SigProc::OmnibusNoiseFilter : 3.7799995 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::DepoSplat : 2.3499987 sec [15:58:51.786] I [ timer ] Timer: WireCell::Hio::HDF5FrameTap : 2.0499997 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::Drifter : 1.8799986 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::DepoFanout : 1.7799987 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::DepoBagger : 1.7399987 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::DepoFanout : 1.5599989 sec [15:58:51.786] I [ timer ] Timer: WireCell::Hio::HDF5FrameTap : 1.1699995 sec [15:58:51.786] I [ timer ] Timer: WireCell::Hio::HDF5FrameTap : 1.0099995 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::AddNoise : 0.93999964 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::Digitizer : 0.9099997 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.88999945 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0.68999964 sec [15:58:51.786] I [ timer ] Timer: wcls::SimDepoSource : 0.66999966 sec [15:58:51.786] I [ timer ] Timer: WireCell::Gen::Reframer : 0.6199997 sec [15:58:51.786] I [ timer ] Timer: Total node execution : 142.94999033212662 sec