Systems Engineering
Infrastructure implementation, operations and deployment, and incident recovery over services, configurations, and runtime state.
Mean Score
69.87
general suite
Score ≥ 0.80
56.66%
R0.8
Full Credit
32.66%
R1
Averaged over all 65 tasks in this category, each evaluated under the 70 general model-harness configurations (7 harnesses × 10 models).
Infrastructure Implementation
26 tasks
T119
Repair a selective backup restore plan without overwriting the wrong files
T139
Repair OCI image layer merging across whiteouts and type replacement
T147
Repair an infrastructure lifecycle planner with drift and replacement rules
T148
Repair Kubernetes-style server-side apply field ownership
T152
Repair PostgreSQL logical replication slot recovery planning
T154
Repair an online PostgreSQL expand-contract migration planner
T158
Repair Kafka transactional offset recovery after ambiguous commits
T161
Repair a Prometheus native histogram migration planner
T164
Repair S3 event-notification ordering and duplicate reconciliation
T165
Repair Kubernetes admission webhook mutation convergence
T169
Repair Redis Cluster redirection and retry planning
T173
Repair Kubernetes watch compaction and resume planning
T174
Repair DNS serve-stale and negative-cache response planning
T175
Repair a flapping service health budget
T180
Repair OCI multi-platform manifest selection
T181
Repair systemd restart-budget decisions
T183
Repair certificate path selection
T184
Repair rolling-deployment surge planning
T185
Repair a Unix permission transition plan
T186
Repair DMARC alignment and disposition decisions
T188
Repair DNS zone serial update ordering
T189
Repair database deadlock retry decisions
T191
Repair backup chunk garbage-collection reachability
T193
Repair Kubernetes finalizer recovery ordering
T196
Repair proxy and NO_PROXY selection
T198
Repair PostgreSQL vacuum wraparound planning
Operations and Deployment
22 tasks
T040
Repair checkout authentication after an issuer migration
T041
Capacity planning from traffic history and instance catalog
T068
Release Approval Gate with Blockers and Pending Actions
T071
Adjudicate SBOM and VEX Evidence for a Release Gate
T078
Reconcile a release artifact provenance gate
T094
Restore paging in an isolated Alertmanager control-plane rehearsal
T096
Restore a Kubernetes workload after secret rotation and port drift
T106
Repair and stage an offline WAF policy through request replay
T112
Remediate a vulnerable release image without breaking its runtime ABI
T200
Stage a Cross-System Least-Privilege Transfer Plan
T209
Admit a JSON-RPC partner batch through a conflicting policy overlay
T210
Rebuild an attestation evidence chain after a signing-key rotation
T212
Ingest a bounded log archive without evicting pinned objects
T214
Drain an RPC failure stream through a probe-budget circuit breaker
T215
Merge three placement snapshots under a running-over-waiting rank and limit
T216
Quarantine SSO audit events that carry dedicated token fields
T226
Apply shared-mailbox sent items only for sent-as identity
T233
Admit DNS cache rows only for in-zone positive answers with a live TTL
T240
Allocate mill CIDRs from a pool without overlap
T245
Apply unordered SQLite migrations with gaps and duplicate versions
T248
Flatten mill host aliases and report CNAME loops
T249
Reject Airflow sensor edges that close a wait-for cycle
Incident Recovery
17 tasks
T001
Recover a crashed DNSSEC rollover staging run
T007
Recover a PostgreSQL Incident to an Unambiguous Point in Time
T066
Reconcile an asynchronous operations update window
T070
Diagnose mixed CI failures and update the rerun plan
T095
Recover Interrupted Incident Evidence Acquisition and Containment State
T097
Recover an Interrupted Kubernetes CA Trust Rotation
T099
Recover a crashed credential-rotation projection
T101
Recover an mTLS Certificate Rotation Without Breaking Overlap Trust
T115
Recover an Interrupted CDC/SCD2 Materialization
T116
Recover an interrupted mailbox DSN and local-draft reconciliation
T117
Recover an interrupted recurring-calendar delta sync
T131
Recover a journal projection after a stale cursor crash
T135
Recover a file tailer across log rotation
T201
Quarantine a propagated synthetic credential and recover the release graph
T208
Recover a settlement idempotency ledger after a lost replay worker
T213
Reconcile an out-of-order CI job journal without promoting failed attempts
T251
Replay a torn JSONL write-ahead log with idempotent ops