Research and Analysis
Preparing data, grounding conclusions in evidence, and producing quantitative analyses from local datasets and documents.
Mean Score
64.55
general suite
Score ≥ 0.80
53.00%
R0.8
Full Credit
25.46%
R1
Averaged over all 40 tasks in this category, each evaluated under the 70 general model-harness configurations (7 harnesses × 10 models). Open a task to inspect its instruction, package files, and the trajectory of every run.
Data Preparation
20 tasks
T028
Excel-like sales table cleaning
→
T051
Small File Batch Normalize With Reject Ledger
→
T055
Prepare a Bank-Tied Month-End Close Package
→
T056
Build an auditable clickstream session dataset
→
T057
Metric definition migration before/after difference report
→
T083
Align cross-year Census geography and variable definitions
→
T084
Reconcile indicator units, pages, and data vintages
→
T085
Apply station-day quality flags before coverage analysis
→
T086
Update a labor-series report across preliminary and revised vintages
→
T088
Repair a rerunnable revenue close join without multiplying orders
→
T104
Repair a dataset DOI version graph and its citation relations
→
T121
Stage a Month-End Three-Way Invoice Review Batch
→
T122
Recover an Interrupted Purchase-Card Month Close
→
T129
Recover an interrupted royalty usage allocation
→
T141
Recover an interrupted DICOM de-identification export
→
T160
Repair a synthetic filing without semantic regression
→
T218
Record aerodynamic cases by canonical digest without silent overwrite
→
T219
Ingest climate datasets without moving a pinned manifest row
→
T228
Apply event-time records against watermark lateness, not processing time
→
T238
Reconcile GTFS realtime events against the static trip set
→
Evidence-Based Research
13 tasks
T014
Summarize Contract Terms And Extract Policy Risks
→
T018
Audit And Repair Cross-Document Citations
→
T021
Reconcile a Clinical-Trial Results QC Correction Packet
→
T048
Research reproducibility package completeness check
→
T059
Offline Knowledge QA with Explicit Unanswerable Detection
→
T060
Batch Research Claims Evidence Audit
→
T064
Prompt Injection Isolation in Internal Document Retrieval
→
T100
Reconcile heterogeneous reports and close a PRISMA evidence flow
→
T102
Repair a paginated research evidence graph without inferring missing entities
→
T250
Rank mill documents by conjunctive term presence
→
T258
policy document contradiction review
→
T266
evidence grounded travel plan
→
T267
eldercare research evidence report
→
Quantitative Analysis
7 tasks
T029
Customer metrics from joined order and refund tables
→
T030
Rebuild the quarterly net-revenue reporting query
→
T034
Product funnel dropoff analysis from JSONL events
→
T054
Prepare an anomaly and incident-attribution brief
→
T107
Recompute and stage a corrected meta-analysis update
→
T253
Compute mill-vessel CPA with a published haversine rule
→
T261
sales analysis formula workbook
→