Systems Engineering
Infrastructure implementation, operations and deployment, and incident recovery over services, configurations, and runtime state.
Mean Score
69.87
general suite
Score ≥ 0.80
56.66%
R0.8
Full Credit
32.66%
R1
Averaged over all 65 tasks in this category, each evaluated under the 70 general model-harness configurations (7 harnesses × 10 models). Open a task to inspect its instruction, package files, and the trajectory of every run.
Infrastructure Implementation
26 tasks
T119
Repair a selective backup restore plan without overwriting the wrong files
→
T139
Repair OCI image layer merging across whiteouts and type replacement
→
T147
Repair an infrastructure lifecycle planner with drift and replacement rules
→
T148
Repair Kubernetes-style server-side apply field ownership
→
T152
Repair PostgreSQL logical replication slot recovery planning
→
T154
Repair an online PostgreSQL expand-contract migration planner
→
T158
Repair Kafka transactional offset recovery after ambiguous commits
→
T161
Repair a Prometheus native histogram migration planner
→
T164
Repair S3 event-notification ordering and duplicate reconciliation
→
T165
Repair Kubernetes admission webhook mutation convergence
→
T169
Repair Redis Cluster redirection and retry planning
→
T173
Repair Kubernetes watch compaction and resume planning
→
T174
Repair DNS serve-stale and negative-cache response planning
→
T175
Repair a flapping service health budget
→
T180
Repair OCI multi-platform manifest selection
→
T181
Repair systemd restart-budget decisions
→
T183
Repair certificate path selection
→
T184
Repair rolling-deployment surge planning
→
T185
Repair a Unix permission transition plan
→
T186
Repair DMARC alignment and disposition decisions
→
T188
Repair DNS zone serial update ordering
→
T189
Repair database deadlock retry decisions
→
T191
Repair backup chunk garbage-collection reachability
→
T193
Repair Kubernetes finalizer recovery ordering
→
T196
Repair proxy and NO_PROXY selection
→
T198
Repair PostgreSQL vacuum wraparound planning
→
Operations and Deployment
22 tasks
T040
Repair checkout authentication after an issuer migration
→
T041
Capacity planning from traffic history and instance catalog
→
T068
Release Approval Gate with Blockers and Pending Actions
→
T071
Adjudicate SBOM and VEX Evidence for a Release Gate
→
T078
Reconcile a release artifact provenance gate
→
T094
Restore paging in an isolated Alertmanager control-plane rehearsal
→
T096
Restore a Kubernetes workload after secret rotation and port drift
→
T106
Repair and stage an offline WAF policy through request replay
→
T112
Remediate a vulnerable release image without breaking its runtime ABI
→
T200
Stage a Cross-System Least-Privilege Transfer Plan
→
T209
Admit a JSON-RPC partner batch through a conflicting policy overlay
→
T210
Rebuild an attestation evidence chain after a signing-key rotation
→
T212
Ingest a bounded log archive without evicting pinned objects
→
T214
Drain an RPC failure stream through a probe-budget circuit breaker
→
T215
Merge three placement snapshots under a running-over-waiting rank and limit
→
T216
Quarantine SSO audit events that carry dedicated token fields
→
T226
Apply shared-mailbox sent items only for sent-as identity
→
T233
Admit DNS cache rows only for in-zone positive answers with a live TTL
→
T240
Allocate mill CIDRs from a pool without overlap
→
T245
Apply unordered SQLite migrations with gaps and duplicate versions
→
T248
Flatten mill host aliases and report CNAME loops
→
T249
Reject Airflow sensor edges that close a wait-for cycle
→
Incident Recovery
17 tasks
T001
Recover a crashed DNSSEC rollover staging run
→
T007
Recover a PostgreSQL Incident to an Unambiguous Point in Time
→
T066
Reconcile an asynchronous operations update window
→
T070
Diagnose mixed CI failures and update the rerun plan
→
T095
Recover Interrupted Incident Evidence Acquisition and Containment State
→
T097
Recover an Interrupted Kubernetes CA Trust Rotation
→
T099
Recover a crashed credential-rotation projection
→
T101
Recover an mTLS Certificate Rotation Without Breaking Overlap Trust
→
T115
Recover an Interrupted CDC/SCD2 Materialization
→
T116
Recover an interrupted mailbox DSN and local-draft reconciliation
→
T117
Recover an interrupted recurring-calendar delta sync
→
T131
Recover a journal projection after a stale cursor crash
→
T135
Recover a file tailer across log rotation
→
T201
Quarantine a propagated synthetic credential and recover the release graph
→
T208
Recover a settlement idempotency ledger after a lost replay worker
→
T213
Reconcile an out-of-order CI job journal without promoting failed attempts
→
T251
Replay a torn JSONL write-ahead log with idempotent ops
→