Research and Analysis
Preparing data, grounding conclusions in evidence, and producing quantitative analyses from local datasets and documents.
Mean Score
64.55
general suite
Score ≥ 0.80
53.00%
R0.8
Full Credit
25.46%
R1
Averaged over all 40 tasks in this category, each evaluated under the 70 general model-harness configurations (7 harnesses × 10 models).
Data Preparation
20 tasks
T028
Excel-like sales table cleaning
T051
Small File Batch Normalize With Reject Ledger
T055
Prepare a Bank-Tied Month-End Close Package
T056
Build an auditable clickstream session dataset
T057
Metric definition migration before/after difference report
T083
Align cross-year Census geography and variable definitions
T084
Reconcile indicator units, pages, and data vintages
T085
Apply station-day quality flags before coverage analysis
T086
Update a labor-series report across preliminary and revised vintages
T088
Repair a rerunnable revenue close join without multiplying orders
T104
Repair a dataset DOI version graph and its citation relations
T121
Stage a Month-End Three-Way Invoice Review Batch
T122
Recover an Interrupted Purchase-Card Month Close
T129
Recover an interrupted royalty usage allocation
T141
Recover an interrupted DICOM de-identification export
T160
Repair a synthetic filing without semantic regression
T218
Record aerodynamic cases by canonical digest without silent overwrite
T219
Ingest climate datasets without moving a pinned manifest row
T228
Apply event-time records against watermark lateness, not processing time
T238
Reconcile GTFS realtime events against the static trip set
Evidence-Based Research
13 tasks
T014
Summarize Contract Terms And Extract Policy Risks
T018
Audit And Repair Cross-Document Citations
T021
Reconcile a Clinical-Trial Results QC Correction Packet
T048
Research reproducibility package completeness check
T059
Offline Knowledge QA with Explicit Unanswerable Detection
T060
Batch Research Claims Evidence Audit
T064
Prompt Injection Isolation in Internal Document Retrieval
T100
Reconcile heterogeneous reports and close a PRISMA evidence flow
T102
Repair a paginated research evidence graph without inferring missing entities
T250
Rank mill documents by conjunctive term presence
T258
policy document contradiction review
T266
evidence grounded travel plan
T267
eldercare research evidence report
Quantitative Analysis
7 tasks
T029
Customer metrics from joined order and refund tables
T030
Rebuild the quarterly net-revenue reporting query
T034
Product funnel dropoff analysis from JSONL events
T054
Prepare an anomaly and incident-attribution brief
T107
Recompute and stage a corrected meta-analysis update
T253
Compute mill-vessel CPA with a published haversine rule
T261
sales analysis formula workbook