User: jbrewste@af.uchicago.edu Condor Job Id: 2588885.0
Walltime (from condor_q): 230:45:02 CPU time (from condor_q): 00:00:00 ServerTime: 1785077402 JobStartDate: 1784246700 RemoteUserCpu: 0.0 RemoteSysCpu: 0.0 ResidentSetSize: 175000 x509UserProxyVOName: RemoteHost: slot1_1@c051.af.uchicago.edu
ActivationDuration: 36 ActivationExecutionDuration: 35 ActivationSetupDuration: 1 ActivationTeardownDuration: 0 Arguments: AutoClusterAttrs: FirstUpdateUptimeGPUsSeconds,LastUpdateUptimeGPUsSeconds,MachineLastMatchTime,No,cache,information,for,host,workers-ads-8b88d7889-4bvng,No,cache,information,for,host,workers-ads-8b88d7889-4jxgt,No,cache,information,for,host,workers-ads-8b88d7889-4tsf4,No,cache,information,for,host,workers-ads-8b88d7889-4x72r,No,cache,information,for,host,workers-ads-8b88d7889-5f9kh,No,cache,information,for,host,workers-ads-8b88d7889-5kcll,No,cache,information,for,host,workers-ads-8b88d7889-6sqkx,No,cache,information,for,host,workers-ads-8b88d7889-76gth,No,cache,information,for,host,workers-ads-8b88d7889-87gj6,No,cache,information,for,host,workers-ads-8b88d7889-b4jwh,No,cache,information,for,host,workers-ads-8b88d7889-f4l48,No,cache,information,for,host,workers-ads-8b88d7889-hjfx6,No,cache,information,for,host,workers-ads-8b88d7889-j9vzx,No,cache,information,for,host,workers-ads-8b88d7889-jqgt7,No,cache,information,for,host,workers-ads-8b88d7889-k6mj5,No,cache,information,for,host,workers-ads-8b88d7889-l28b5,No,cache,information,for,host,workers-ads-8b88d7889-l7jbl,No,cache,information,for,host,workers-ads-8b88d7889-lrm6n,No,cache,information,for,host,workers-ads-8b88d7889-mbzkq,No,cache,information,for,host,workers-ads-8b88d7889-mmszb,No,cache,information,for,host,workers-ads-8b88d7889-nqlmv,No,cache,information,for,host,workers-ads-8b88d7889-nzw4f,No,cache,information,for,host,workers-ads-8b88d7889-p5qvf,No,cache,information,for,host,workers-ads-8b88d7889-pw8gz,No,cache,information,for,host,workers-ads-8b88d7889-qt7mj,No,cache,information,for,host,workers-ads-8b88d7889-r2rmw,No,cache,information,for,host,workers-ads-8b88d7889-r9rll,No,cache,information,for,host,workers-ads-8b88d7889-rlkbv,No,cache,information,for,host,workers-ads-8b88d7889-vh95k,No,cache,information,for,host,workers-ads-8b88d7889-vqkgz,No,cache,information,for,host,workers-ads-8b88d7889-wbqpk,No,cache,information,for,host,workers-ads-8b88d7889-wzpn8,No,cache,information,for,host,workers-ads-8b88d7889-zkldx,No,cache,information,for,host,workers-gpu-5d8bdf5f6f-nrh79,No,cache,information,for,host,workers-gpu-5d8bdf5f6f-r27sq,No,cache,information,for,host,workers-gpu-5d8bdf5f6f-vk89q,No,cache,information,for,host,workers-login-7686fc76f7-jhtzl,No,cache,information,for,host,workers-login-7686fc76f7-nrx98,No,cache,information,for,host,workers-login-7686fc76f7-xlfjx,queue,RemoteOwner,RequestCpus,RequestDisk,RequestGPUs,RequestMemory,StartOfJobUptimeGPUsSeconds,TotalJobRuntime,UptimeGPUsSeconds,ConcurrencyLimits,FlockTo,Rank,Requirements,workers-ads-8b88d7889-cfvrk,workers-ads-8b88d7889-5qnpd,workers-login-7686fc76f7-pmxkp,workers-ads-8b88d7889-4w7sv,workers-ads-8b88d7889-9rm8l,workers-ads-8b88d7889-ddl2x,workers-ads-8b88d7889-j9498,workers-ads-8b88d7889-z7xbg,workers-ads-8b88d7889-5llv2,workers-ads-8b88d7889-2xgw6,workers-ads-8b88d7889-9hzxw,workers-ads-8b88d7889-dg244,workers-ads-8b88d7889-j8hnn,workers-ads-8b88d7889-l4hrj,workers-ads-8b88d7889-lsxq6,workers-ads-8b88d7889-mfsnr,workers-ads-8b88d7889-wr68p,workers-ads-8b88d7889-r6mjx,workers-ads-8b88d7889-msdw8,workers-ads-8b88d7889-q2c7t,workers-ads-8b88d7889-r5wgh,workers-ads-8b88d7889-x4j5d,workers-ads-8b88d7889-lwtcd,workers-ads-8b88d7889-j5jxk,workers-ads-8b88d7889-dzwfk,workers-ads-8b88d7889-qr9rd,workers-ads-8b88d7889-wpm77,workers-ads-8b88d7889-f8tzx,workers-ads-8b88d7889-krg2r,workers-ads-8b88d7889-pcjrl,workers-ads-8b88d7889-rtlld,workers-ads-8b88d7889-drx5j,workers-ads-8b88d7889-8fn9n,workers-ads-8b88d7889-9bmvd,workers-ads-8b88d7889-c65f7,workers-ads-8b88d7889-ckkr5,workers-ads-8b88d7889-4cwrd,workers-ads-8b88d7889-6cc67,workers-ads-8b88d7889-9psrf,workers-ads-8b88d7889-7ssd7,workers-ads-8b88d7889-9m7tv,workers-ads-8b88d7889-pnxbj,workers-ads-8b88d7889-rbq9g,workers-gpu-5d8bdf5f6f-9klrm,workers-ads-8b88d7889-bjf7l,workers-ads-8b88d7889-cbdhz,workers-ads-8b88d7889-4cjr6,workers-ads-8b88d7889-qrmst,workers-ads-8b88d7889-vr9j7,workers-ads-8b88d7889-56tlc,workers-ads-8b88d7889-9kt79,workers-ads-8b88d7889-nm4mp,workers-ads-8b88d7889-p4xm6,workers-ads-8b88d7889-m26c9,workers-ads-8b88d7889-n8xbm,workers-ads-8b88d7889-qnrhj,workers-ads-8b88d7889-4d4dd,workers-ads-8b88d7889-7z88d,workers-ads-8b88d7889-4xn27,workers-ads-8b88d7889-76pw7,workers-ads-8b88d7889-lrvnt,workers-ads-8b88d7889-p4spm,workers-ads-8b88d7889-xn7vx,workers-ads-8b88d7889-ztlpv,workers-gpu-5d8bdf5f6f-b8s9m,workers-ads-8b88d7889-cxxf2,workers-ads-8b88d7889-jsrmb,workers-ads-8b88d7889-wbjmr,workers-ads-8b88d7889-bplmd,workers-ads-8b88d7889-xmdhz,workers-ads-8b88d7889-44dgp,workers-ads-8b88d7889-jf7rh,workers-ads-8b88d7889-q795f,workers-login-7686fc76f7-5zwdb,workers-ads-8b88d7889-fqsvk,workers-ads-8b88d7889-2mx8s,workers-ads-8b88d7889-dktsz,workers-ads-8b88d7889-h5l2s,workers-ads-8b88d7889-6hw2c,workers-ads-8b88d7889-b9pm9,workers-ads-8b88d7889-cw6p9,workers-ads-8b88d7889-dvbs6,workers-ads-8b88d7889-h8tpx,workers-ads-8b88d7889-t6vn4,workers-gpu-5d8bdf5f6f-9djs9,workers-login-7686fc76f7-bctnl,workers-ads-8b88d7889-2rrsc,workers-ads-8b88d7889-8c525,workers-login-7686fc76f7-xt578,workers-ads-8b88d7889-cbz7k,workers-ads-8b88d7889-mkw7v,workers-login-7686fc76f7-wzkwf,workers-ads-8b88d7889-hfz2c,workers-ads-8b88d7889-x5ctl,workers-ads-8b88d7889-hgvkk,workers-ads-8b88d7889-lt46h,workers-ads-8b88d7889-2tpql,workers-ads-8b88d7889-7mbf2,workers-ads-8b88d7889-wlxrx,workers-ads-8b88d7889-8wp8j,workers-ads-8b88d7889-9c9lf,ALLOW_MWT2,ERROR,only,0,GB,free,in,/,NodeOnline,benchmarking,cannot,access,/data:,Permission,denied,DiskUsage,FileSystemDomain,RequestRuntime AutoClusterId: 15 BlockReadKbytes: 0 BlockReads: 0 BlockWriteKbytes: 0 BlockWrites: 0 BytesRecvd: 4446360.0 BytesSent: 2264172338.0 ClusterId: 2588885 Cmd: /home/jbrewste/muon_aqp/run/TMP/2026-07-16/myProcessJob_Z2017_2/CONFIG/EnvScript_DupH9tcwbT2vO4MGIGDBNWi4GOYorBSIuL6Fnt9EaiOT4nkbkO.sh CommittedSlotTime: 0 CommittedSuspensionTime: 0 CommittedTime: 0 CondorPlatform: $CondorPlatform: x86_64_AlmaLinux9 $ CondorVersion: $CondorVersion: 25.0.12 2026-07-07 BuildID: 930047 PackageID: 25.0.12-1 $ CpusProvisioned: 1 CpusUsage: 0.0 CumulativeRemoteSysCpu: 68282.0 CumulativeRemoteUserCpu: 35457.0 CumulativeSlotTime: 456748.0 CumulativeSuspensionTime: 0 CurrentHosts: 1 DAGManJobId: 2585629 DAGManNodesLog: /home/jbrewste/muon_aqp/run/LOGS/2026-07-16/myProcessJob_Z2017_2/DAG/./myProcessJob_Z2017_2.dag.nodes.log DAGManNodesMask: 0,1,2,4,5,7,8,9,10,11,12,13,16,17,24,27,35,36 DAGNodeName: C2753 DAGParentNodeNames: B1,B2,B3,B4,B5,B6,B7,B8,B9,B10,B11,B12,B13,B14,B15,B16,B17,B18,B19,B20,B21,B22,B23,B24,B25,B26,B27,B28,B29,B30,B31,B32,B33,B34,B35,B36,B37,B38,B39,B40,B41,B42,B43,B44,B45,B46 DiskProvisioned: 2311 DiskUsage: 225 DiskUsage_RAW: 221 EnteredCurrentStatus: 1785077378 Environment: CONDOR_TASK_ID=2753 Err: /home/jbrewste/muon_aqp/run/LOGS/2026-07-16/myProcessJob_Z2017_2/myProcessJob_Z2017_2_2753.err ExecutableSize: 1 ExecutableSize_RAW: 1 ExecuteDirWasEncrypted: Value.Undefined ExitBySignal: False ExitCode: 1 ExitReason: died on signal 15 (Terminated) ExitSignal: 15 ExitStatus: 0 FileSystemDomain: nfs.af.uchicago.edu FirstJobMatchDate: 1784246699 GPUsProvisioned: 0 GlobalJobId: head01.af.uchicago.edu#2588885.0#1784246694 ImageSize: 175000 ImageSize_RAW: 150616 In: /dev/null InitialWaitDuration: 6 Iwd: /home/jbrewste/muon_aqp/run/LOGS/2026-07-16/myProcessJob_Z2017_2/DAG JobBatchId: 2585629.0 JobBatchName: myProcessJob_Z2017_2.dag+2585629 JobCurrentFinishTransferInputDate: 1785077318 JobCurrentFinishTransferOutputDate: 1785077353 JobCurrentReconnectAttempt: Value.Undefined JobCurrentStartDate: 1785077378 JobCurrentStartExecutingDate: 1785077379 JobCurrentStartTransferInputDate: 1785077317 JobCurrentStartTransferOutputDate: 1785077353 JobDisconnectedDate: 1784815487 JobLastStartDate: 1785077317 JobLeaseDuration: 2400 JobNotification: 3 JobPrio: 0 JobRunCount: 12352 JobStartDate: 1784246700 JobStatus: 2 JobSubmitFile: /home/jbrewste/muon_aqp/run/TMP/2026-07-16/myProcessJob_Z2017_2/CONFIG/Job_vzSvT2XhYcwzXrSlMf6eVXNTh.sub JobSubmitMethod: 1 JobUniverse: 5 KeepClaimIdle: 20 LastJobLeaseRenewal: 1785077379 LastJobStatus: 1 LastMatchTime: 1785077378 LastPublicClaimId: <192.170.240.201:9618?CCBID=192.170.240.14:9618%3faddrs%3d192.170.240.14-9618+[2605-9a00-10-200a-e63d-1aff-fe01-80e0]-9618%26alias%3dhead01.af.uchicago.edu%26noUDP%26sock%3dcollector#302104&PrivNet=af.uchicago.edu&addrs=192.170.240.201-9618+[2605-9a00-10-200a-266e-96ff-fe18-b5b8]-9618&alias=c036.af.uchicago.edu&noUDP&sock=startd_56685_c60b>#1784526126#51101#... LastRejMatchNegotiator: head01.af.uchicago.edu LastRejMatchReason: no match found LastRejMatchTime: 1785075574 LastRemoteHost: slot1_2@c036.af.uchicago.edu LastRemoteWallClockTime: 36.0 LastSuspensionTime: 0 LastVacateTime: 1784815487 LeaveJobInQueue: False MachineAttrCpus0: 1 MachineAttrSlotWeight0: 1 MaxHosts: 1 MaxRuntime: 86399 MemoryProvisioned: 3072 MemoryUsage: ((ResidentSetSize + 1023) / 1024) MinHosts: 1 MyType: Job NotifyUser: jbrewste@cern.ch NumCkpts: 0 NumCkpts_RAW: 0 NumInputTransferStarts: 12351 NumJobCompletions: 12345 NumJobMatches: 12352 NumJobStarts: 12349 NumOutputTransferStarts: 12331 NumRestarts: 0 NumShadowStarts: 12352 NumSystemHolds: 0 NumVacates: 6 NumVacatesByReason: [ StartdShutdown = 3; ReconnectFailed = 3 ] OnExitRemove: (ExitBySignal == false) && (ExitCode == 0) OrigMaxHosts: 1 Out: /home/jbrewste/muon_aqp/run/LOGS/2026-07-16/myProcessJob_Z2017_2/myProcessJob_Z2017_2_2753.out Owner: jbrewste PeriodicRelease: False PeriodicRemove: (JobStatus == 2) && !isUndefined(JobCurrentStartDate) && ((CurrentTime - JobCurrentStartDate) > 86399) ProcId: 0 PublicClaimId: <192.170.240.216:9618?CCBID=192.170.240.14:9618%3faddrs%3d192.170.240.14-9618+[2605-9a00-10-200a-e63d-1aff-fe01-80e0]-9618%26alias%3dhead01.af.uchicago.edu%26noUDP%26sock%3dcollector#302227&PrivNet=af.uchicago.edu&addrs=192.170.240.216-9618+[2605-9a00-10-200a-266e-96ff-fe19-cd58]-9618&alias=c051.af.uchicago.edu&noUDP&sock=startd_56685_c60b>#1784528462#83832#... QDate: 1784246694 Rank: 0.0 RecentBlockReadKbytes: 0 RecentBlockReads: 0 RecentBlockWriteKbytes: 0 RecentBlockWrites: 0 RecentStatsLifetimeStarter: 26 ReleaseConditionsEditedByAdmin: True RemoteHost: slot1_1@c051.af.uchicago.edu RemoteSlotID: 1 RemoteSysCpu: 0.0 RemoteUserCpu: 0.0 RemoteWallClockTime: 456748.0 RequestCpus: 1 RequestDisk: DiskUsage RequestMemory: 3000 RequestRuntime: 86399 RequeueReason: The job attribute OnExitRemove expression '(ExitBySignal == false) && (ExitCode == 0)' evaluated to FALSE Requirements: ((TARGET.Arch == "X86_64") && (TARGET.OpSys == "LINUX") && (TARGET.Memory >= RequestMemory) && (TARGET.Cpus >= RequestCpus) && (isUndefined(TARGET.Runtime) || TARGET.Runtime >= RequestRuntime)) && (TARGET.Disk >= RequestDisk) && (TARGET.Memory >= RequestMemory) && ((TARGET.FileSystemDomain == MY.FileSystemDomain) || (TARGET.HasFileTransfer)) ResidentSetSize: 175000 ResidentSetSize_RAW: 150996 ScratchDirFileCount: 18 ServerTime: 1785077402 ShadowBday: 1785077378 ShouldTransferFiles: IF_NEEDED SlotAccount: SlotOwner: jbrewste SpooledOutputFiles: Value.Undefined StartdIpAddr: <192.170.240.216:9618?CCBID=192.170.240.14:9618%3faddrs%3d192.170.240.14-9618+[2605-9a00-10-200a-e63d-1aff-fe01-80e0]-9618%26alias%3dhead01.af.uchicago.edu%26noUDP%26sock%3dcollector#302227&PrivNet=af.uchicago.edu&addrs=192.170.240.216-9618+[2605-9a00-10-200a-266e-96ff-fe19-cd58]-9618&alias=c051.af.uchicago.edu&noUDP&sock=startd_56685_c60b> StartdPrincipal: execute-side@matchsession/192.170.240.216 StatsLifetimeStarter: 35 StderrMtime: 1785077352 StdoutMtime: 1785077353 StreamErr: False StreamOut: False SubmitEventNotes: DAG Node: C2753 TargetType: Machine TotalJobReconnectAttempts: 16 TotalSubmitProcs: 1 TotalSuspensions: 0 TransferIn: False TransferInFinished: 1785077378 TransferInStarted: 1785077378 TransferInputFileCounts: [ CEDAR = 1 ] TransferInputSizeMB: 0 TransferInputStats: [ CedarSizeBytesTotal = 4446360; CedarFilesCountTotal = 12351; CedarSizeBytesLastRun = 360; CedarFilesCountLastRun = 1 ] TransferOutFinished: 1785077353 TransferOutStarted: 1785077353 TransferOutputStats: [ CedarSizeBytesTotal = 715472992; CedarFilesCountTotal = 7792; CedarSizeBytesLastRun = 183634; CedarFilesCountLastRun = 2 ] TransferQueued: False TransferringInput: False User: jbrewste@af.uchicago.edu UserLog: /home/jbrewste/muon_aqp/run/LOGS/2026-07-16/myProcessJob_Z2017_2/myProcessJob_Z2017_2_2753.log WhenToTransferOutput: ON_EXIT cputime: 0 csecs: 0.0 effcy: 0.0 job_id: 2588885.0 rss: 170.9M submitter: head01.af.uchicago.edu vm: 147.1M walltime: 830702 wsecs: 830702.0