Systems Engineering

Infrastructure implementation, operations and deployment, and incident recovery over services, configurations, and runtime state.

65 tasks3 subcategories65 general tasksArtifact-based oracle scoring
Mean Score
69.87
general suite
Score ≥ 0.80
56.66%
R0.8
Full Credit
32.66%
R1
Averaged over all 65 tasks in this category, each evaluated under the 70 general model-harness configurations (7 harnesses × 10 models). Open a task to inspect its instruction, package files, and the trajectory of every run.
Infrastructure Implementation
26 tasks
T119
Repair a selective backup restore plan without overwriting the wrong files
→
T139
Repair OCI image layer merging across whiteouts and type replacement
→
T147
Repair an infrastructure lifecycle planner with drift and replacement rules
→
T148
Repair Kubernetes-style server-side apply field ownership
→
T152
Repair PostgreSQL logical replication slot recovery planning
→
T154
Repair an online PostgreSQL expand-contract migration planner
→
T158
Repair Kafka transactional offset recovery after ambiguous commits
→
T161
Repair a Prometheus native histogram migration planner
→
T164
Repair S3 event-notification ordering and duplicate reconciliation
→
T165
Repair Kubernetes admission webhook mutation convergence
→
T169
Repair Redis Cluster redirection and retry planning
→
T173
Repair Kubernetes watch compaction and resume planning
→
T174
Repair DNS serve-stale and negative-cache response planning
→
T175
Repair a flapping service health budget
→
T180
Repair OCI multi-platform manifest selection
→
T181
Repair systemd restart-budget decisions
→
T183
Repair certificate path selection
→
T184
Repair rolling-deployment surge planning
→
T185
Repair a Unix permission transition plan
→
T186
Repair DMARC alignment and disposition decisions
→
T188
Repair DNS zone serial update ordering
→
T189
Repair database deadlock retry decisions
→
T191
Repair backup chunk garbage-collection reachability
→
T193
Repair Kubernetes finalizer recovery ordering
→
T196
Repair proxy and NO_PROXY selection
→
T198
Repair PostgreSQL vacuum wraparound planning
→
Operations and Deployment
22 tasks