NeuroShard / Research evidence / 13 September 2026
Can workers communicate less while preserving learning? Three predetermined training arms use the same 1.7B seed, global document schedule and 128 updates. Local windows exchange endpoint deltas every 16 updates and keep each worker’s Adam state. The model remains fixed in size.
This is one operated experiment on four generated task families and previously exposed public retention probes. It does not establish broad assistant quality, independent ownership, permissionless GPU verification, continual improvement or model growth. It issues no NEURO and does not promote a native serving checkpoint.
| Model | Exact answers | Valid JSON | Mean retention change |
|---|---|---|---|
| Seed | 105 / 1024 (10.25%) | 776 | — |
| 1 GPU · AdamW | 641 / 1024 (62.60%) | 1009 | -0.002428 |
| 4 GPUs · DDP | 613 / 1024 (59.86%) | 1024 | -0.002885 |
| 4 GPUs · local windows | 557 / 1024 (54.39%) | 965 | +0.083294 |
| Method | Active time | Loop with checkpoints | Whole-host bytes sent |
|---|---|---|---|
| 1 GPU · AdamW | 208.09 s | 266.12 s | 0.000 GB |
| 4 GPUs · DDP | 1979.26 s | 2048.37 s | 5330.663 GB |
| 4 GPUs · local windows | 250.08 s | 494.51 s | 331.778 GB |
Active time includes local updates and synchronization; startup, checkpoints and whole-instance allocation are separate. The 2.5 Gbit/s baseline and burst bandwidth of these AWS instances affected the timings. The methods ran in a fixed order, and checkpoint pauses can replenish network credits. This is an observed workload comparison, not an isolated estimate of algorithmic speedup.
After the rank-one process was killed following step 75, the group restarted at step 64. Exact final model, outer momentum, every rank’s Adam/RNG contents and the numerical trajectories agree with the uninterrupted run: True. This is coordinated recovery of the same operated membership.
A separate training-only probe completes eight dense warmup updates and four rank-32 PowerSGD updates on two GPUs, with both workers agreeing. The first attempt runs out of GPU memory; moving dense error feedback to CPU between completed bucket reductions fixes that failure. Peak allocated GPU memory is 36.89 GiB. Small-model CPU and CUDA checks preserve the resident control’s numerical state exactly.
The four compressed updates show 45.14× logical compression, excluding dense warmup and transport overhead. This is not a full-length quality or sustained-throughput result, and no complete compression-state recovery is implemented. Probe measurements · Failed resident attempt · PowerSGD paper.
| Providers | Completed | Requests/second | Median latency |
|---|---|---|---|
| single | 64 / 64 | 3.008 | 1.285 s |
| four | 64 / 64 | 9.025 | 0.321 s |
| failure | 64 / 64 | 6.713 | 0.450 s |
Each phase uses the same 64 development tasks and client concurrency four. The outage phase advertises four endpoints with one stopped. This private fixed-task fixture has no native billing or general chat API.
Methods and interpretation · Recomputed metrics · Frozen inputs · Candidate commitments · Checksums · Small evidence archive
Predetermined weights: single GPU · four-worker DDP · four-worker local windows. All outcomes are retained, including failed comparisons. Downloads contain model/tokenizer files and attribution; full optimizer/outer states are separately backed up by the operator.
Use only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"H142","city":"Perth","units":6},{"id":"C241","city":"Lima","units":17},{"id":"A538","city":"Bern","units":11},{"id":"D771","city":"Riga","units":3},{"id":"A655","city":"Riga","units":12}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Perth","units":6},{"city":"Lima","units":17},{"city":"Bern","units":11},{"city":"Riga","units":3},{"city":"Riga","units":12}]54 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"P980","status":"closed"},{"id":"N228","status":"pending"},{"id":"V371","status":"closed"},{"id":"H680","status":"ready"},{"id":"S551","status":"closed"}]Checked answer{"ids": ["H680"]}wrong_schema_or_value
{"ids": ["H680", "S551", "P980", "V371", "N228"]}35 tokensCorrect
{"ids":["H680"]}11 tokensCorrect
{"ids":["H680"]}11 tokensCorrect
{"ids":["H680"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"N657","units":7,"price":5},{"id":"P766","units":0,"price":10},{"id":"F259","units":19,"price":9},{"id":"V629","units":7,"price":3},{"id":"A709","units":14,"price":9}]Checked answer{"total": 353}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"D699","priority":8},{"id":"Q971","priority":4},{"id":"W300","priority":5},{"id":"F657","priority":7},{"id":"Q559","priority":1}]Checked answer{"ids": ["D699", "F657"]}wrong_schema_or_value
{"ids": ["Q559", "W300", "D699", "Q971", "F657"]}35 tokensCorrect
{"ids":["D699","F657"]}16 tokensCorrect
{"ids":["D699","F657"]}16 tokensCorrect
{"ids":["D699","F657"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id M852 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"M534","city":"Lima","units":9},{"id":"M852","city":"Riga","units":11},{"id":"G527","city":"Oslo","units":18},{"id":"C961","city":"Lima","units":7},{"id":"N447","city":"Riga","units":19}]Checked answer{"city": "Riga", "units": 11}wrong_schema_or_value
{"city":"Lima","units":7}11 tokensCorrect
{"city":"Riga","units":11}12 tokensCorrect
{"city":"Riga","units":11}12 tokensCorrect
{"city":"Riga","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"Y439","status":"pending"},{"id":"H889","status":"pending"},{"id":"S402","status":"closed"},{"id":"N641","status":"ready"},{"id":"N993","status":"pending"}]Checked answer{"ids": ["N641"]}wrong_schema_or_value
{
"ids": ["N641", "N993", "S402"]
}26 tokensCorrect
{"ids":["N641"]}11 tokensCorrect
{"ids":["N641"]}11 tokensCorrect
{"ids":["N641"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"X440","units":10,"price":17},{"id":"G152","units":1,"price":5},{"id":"Y376","units":3,"price":1},{"id":"T175","units":17,"price":10},{"id":"M868","units":13,"price":5}]Checked answer{"total": 413}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":468}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Q486","priority":1},{"id":"Q910","priority":6},{"id":"N223","priority":3},{"id":"E772","priority":9},{"id":"R407","priority":2}]Checked answer{"ids": ["E772", "Q910"]}invalid_json_object
[{"id":"Q486","priority":1},{"id":"Q910","priority":6},{"id":"N223","priority":3},{"id":"E772","priority":9},{"id":"R407","priority":2}]62 tokensCorrect
{"ids":["E772","Q910"]}16 tokenswrong_schema_or_value
{"ids":["E772","N223"]}16 tokensCorrect
{"ids":["E772","Q910"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to L421? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"X495","city":"Perth","units":17},{"id":"W559","city":"Suva","units":7},{"id":"L828","city":"Oslo","units":7},{"id":"L421","city":"Accra","units":9},{"id":"G770","city":"Suva","units":12}]Checked answer{"city": "Accra", "units": 9}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Accra","units":9}11 tokensCorrect
{"city":"Accra","units":9}11 tokensCorrect
{"city":"Accra","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"M856","status":"ready"},{"id":"V540","status":"ready"},{"id":"R519","status":"pending"},{"id":"G858","status":"closed"},{"id":"L747","status":"ready"}]Checked answer{"ids": ["L747", "M856", "V540"]}invalid_json_object
[{"id":"M856","status":"ready"},{"id":"V540","status":"ready"},{"id":"R519","status":"pending"},{"id":"G858","status":"closed"},{"id":"L747","status":"ready"}]63 tokenswrong_schema_or_value
{"ids":["M856","L747","V540"]}21 tokensCorrect
{"ids":["L747","M856","V540"]}21 tokensinvalid_json_object
["ids":["M856","L747","V540"]]21 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"V967","units":15,"price":10},{"id":"K750","units":12,"price":17},{"id":"G558","units":14,"price":4},{"id":"R162","units":0,"price":15},{"id":"C130","units":5,"price":18}]Checked answer{"total": 500}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"R250","priority":2},{"id":"I161","priority":4},{"id":"S243","priority":5},{"id":"K361","priority":6},{"id":"Z965","priority":2}]Checked answer{"ids": ["K361", "S243"]}wrong_schema_or_value
{"ids": ["R250", "I161", "S243", "K361", "Z965"]}35 tokensCorrect
{"ids":["K361","S243"]}16 tokensCorrect
{"ids":["K361","S243"]}16 tokensCorrect
{"ids":["K361","S243"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=P975. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"P975","city":"Riga","units":9},{"id":"A231","city":"Suva","units":2},{"id":"X246","city":"Oslo","units":1},{"id":"X349","city":"Bern","units":18},{"id":"Q313","city":"Perth","units":12}]Checked answer{"city": "Riga", "units": 9}Correct
{"city": "Riga", "units": 9}14 tokensCorrect
{"city":"Riga","units":9}11 tokensCorrect
{"city":"Riga","units":9}11 tokensCorrect
{"city":"Riga","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"T936","status":"ready"},{"id":"N656","status":"pending"},{"id":"Y463","status":"pending"},{"id":"N888","status":"ready"},{"id":"Q934","status":"closed"}]Checked answer{"ids": ["N888", "T936"]}wrong_schema_or_value
{"ids": ["N656", "N888", "T936", "Q934"]}29 tokenswrong_schema_or_value
{"ids":["T936","N888"]}16 tokenswrong_schema_or_value
{"ids":["N656","T936"]}16 tokenswrong_schema_or_value
{"ids":["N656","T936"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"G937","units":9,"price":11},{"id":"P173","units":4,"price":5},{"id":"V258","units":9,"price":11},{"id":"X904","units":1,"price":3},{"id":"O497","units":11,"price":3}]Checked answer{"total": 254}wrong_schema_or_value
{
"total": 110
}12 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"A517","priority":4},{"id":"S453","priority":2},{"id":"W209","priority":8},{"id":"X601","priority":4},{"id":"K545","priority":1}]Checked answer{"ids": ["W209", "A517"]}invalid_json_object
[{"id":"W209","priority":8},{"id":"X601","priority":4},{"id":"A517","priority":4},{"id":"S453","priority":2},{"id":"K545","priority":1}]62 tokenswrong_schema_or_value
{"ids":["W209","X601"]}16 tokensCorrect
{"ids":["W209","A517"]}16 tokenswrong_schema_or_value
{"ids":["W209","X601"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record F749. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"I822","city":"Lima","units":0},{"id":"C390","city":"Perth","units":4},{"id":"R752","city":"Accra","units":4},{"id":"F749","city":"Lima","units":8},{"id":"Z493","city":"Kyoto","units":8}]Checked answer{"city": "Lima", "units": 8}Correct
{
"city": "Lima",
"units": 8
}18 tokensCorrect
{"city":"Lima","units":8}11 tokensCorrect
{"city":"Lima","units":8}11 tokensCorrect
{"city":"Lima","units":8}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"I200","status":"pending"},{"id":"E734","status":"ready"},{"id":"M591","status":"ready"},{"id":"S308","status":"ready"},{"id":"B937","status":"pending"}]Checked answer{"ids": ["E734", "M591", "S308"]}wrong_schema_or_value
{"ids": ["E734", "M591", "S308", "B937", "I200"]}35 tokensCorrect
{"ids":["E734","M591","S308"]}21 tokensCorrect
{"ids":["E734","M591","S308"]}21 tokenswrong_schema_or_value
{"ids":["E734","S308","M591"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Y939","units":5,"price":8},{"id":"D555","units":0,"price":2},{"id":"L796","units":12,"price":16},{"id":"Y751","units":17,"price":3},{"id":"I847","units":4,"price":13}]Checked answer{"total": 335}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"N789","priority":1},{"id":"O537","priority":4},{"id":"J807","priority":9},{"id":"T576","priority":9},{"id":"N763","priority":5}]Checked answer{"ids": ["J807", "T576"]}wrong_schema_or_value
{"ids": ["N789", "O537", "J807", "T576", "N763"]}35 tokensCorrect
{"ids":["J807","T576"]}16 tokensCorrect
{"ids":["J807","T576"]}16 tokensCorrect
{"ids":["J807","T576"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"L947","city":"Lima","units":3},{"id":"G117","city":"Bern","units":4},{"id":"J482","city":"Bern","units":15},{"id":"J286","city":"Accra","units":13},{"id":"R766","city":"Oslo","units":5}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Lima","units":3}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"J289","status":"ready"},{"id":"T577","status":"closed"},{"id":"Y892","status":"ready"},{"id":"E794","status":"closed"},{"id":"G838","status":"pending"}]Checked answer{"ids": ["J289", "Y892"]}wrong_schema_or_value
{
"ids": ["J289", "Y892", "G838"]
}26 tokensCorrect
{"ids":["J289","Y892"]}16 tokensCorrect
{"ids":["J289","Y892"]}16 tokensCorrect
{"ids":["J289","Y892"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"P890","units":10,"price":6},{"id":"K773","units":7,"price":7},{"id":"T762","units":2,"price":12},{"id":"D645","units":3,"price":10},{"id":"B658","units":0,"price":7}]Checked answer{"total": 163}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":268}8 tokenswrong_schema_or_value
{"total":268}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"X824","priority":1},{"id":"D952","priority":6},{"id":"E938","priority":5},{"id":"A238","priority":1},{"id":"L597","priority":6}]Checked answer{"ids": ["D952", "L597"]}invalid_json_object
[{"id":"X824","priority":1},{"id":"A238","priority":1},{"id":"L597","priority":6},{"id":"D952","priority":6},{"id":"E938","priority":5}]62 tokensCorrect
{"ids":["D952","L597"]}16 tokensCorrect
{"ids":["D952","L597"]}16 tokensCorrect
{"ids":["D952","L597"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to L130? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"A304","city":"Riga","units":11},{"id":"I689","city":"Bern","units":2},{"id":"T384","city":"Riga","units":14},{"id":"L130","city":"Bern","units":14},{"id":"H902","city":"Riga","units":6}]Checked answer{"city": "Bern", "units": 14}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Bern","units":14}11 tokensCorrect
{"city":"Bern","units":14}11 tokensCorrect
{"city":"Bern","units":14}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"X614","status":"pending"},{"id":"U671","status":"closed"},{"id":"X289","status":"ready"},{"id":"G526","status":"ready"},{"id":"H926","status":"closed"}]Checked answer{"ids": ["G526", "X289"]}invalid_json_object
[{"id":"X289","status":"ready"},{"id":"X614","status":"pending"},{"id":"U671","status":"closed"},{"id":"G526","status":"ready"},{"id":"H926","status":"closed"}]63 tokensCorrect
{"ids":["G526","X289"]}16 tokensCorrect
{"ids":["G526","X289"]}16 tokensinvalid_json_object
["ids":["G526","X289"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"F529","units":16,"price":17},{"id":"W196","units":18,"price":8},{"id":"U561","units":4,"price":13},{"id":"L688","units":13,"price":7},{"id":"P900","units":9,"price":12}]Checked answer{"total": 667}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"A573","priority":6},{"id":"Y639","priority":1},{"id":"J161","priority":7},{"id":"Z593","priority":6},{"id":"C580","priority":7}]Checked answer{"ids": ["C580", "J161"]}wrong_schema_or_value
{"ids": ["A573", "Y639", "J161", "Z593", "C580"]}35 tokensCorrect
{"ids":["C580","J161"]}16 tokensCorrect
{"ids":["C580","J161"]}16 tokensCorrect
{"ids":["C580","J161"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=X493. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"K997","city":"Bern","units":5},{"id":"T796","city":"Accra","units":14},{"id":"G206","city":"Oslo","units":7},{"id":"U450","city":"Perth","units":4},{"id":"X493","city":"Oslo","units":5}]Checked answer{"city": "Oslo", "units": 5}Correct
{"city":"Oslo","units":5}11 tokensCorrect
{"city":"Oslo","units":5}11 tokensCorrect
{"city":"Oslo","units":5}11 tokensCorrect
{"city":"Oslo","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Q640","status":"ready"},{"id":"G534","status":"ready"},{"id":"K827","status":"closed"},{"id":"B616","status":"pending"},{"id":"F940","status":"ready"}]Checked answer{"ids": ["F940", "G534", "Q640"]}wrong_schema_or_value
{"ids": ["G534", "K827", "F940", "Q640", "B616"]}35 tokensCorrect
{"ids":["F940","G534","Q640"]}21 tokenswrong_schema_or_value
{"ids":["B616","F940","G534","Q640"]}26 tokenswrong_schema_or_value
{"ids":["F940","Q640","G534"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"D370","units":11,"price":13},{"id":"Z578","units":0,"price":14},{"id":"N604","units":4,"price":7},{"id":"L243","units":5,"price":4},{"id":"I427","units":8,"price":19}]Checked answer{"total": 343}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"V724","priority":8},{"id":"V182","priority":9},{"id":"T781","priority":6},{"id":"M269","priority":1},{"id":"U939","priority":8}]Checked answer{"ids": ["V182", "U939"]}invalid_json_object
[{"id":"V724","priority":8},{"id":"V182","priority":9},{"id":"T781","priority":6},{"id":"M269","priority":1},{"id":"U939","priority":8}]62 tokensCorrect
{"ids":["V182","U939"]}16 tokensCorrect
{"ids":["V182","U939"]}16 tokensCorrect
{"ids":["V182","U939"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record L635. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"U885","city":"Perth","units":0},{"id":"L635","city":"Accra","units":6},{"id":"F732","city":"Lima","units":10},{"id":"E321","city":"Perth","units":12},{"id":"Q110","city":"Lima","units":17}]Checked answer{"city": "Accra", "units": 6}Correct
{
"city": "Accra",
"units": 6
}18 tokensCorrect
{"city":"Accra","units":6}11 tokensCorrect
{"city":"Accra","units":6}11 tokensCorrect
{"city":"Accra","units":6}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"D780","status":"pending"},{"id":"H204","status":"pending"},{"id":"A647","status":"ready"},{"id":"O363","status":"pending"},{"id":"B673","status":"pending"}]Checked answer{"ids": ["A647"]}wrong_schema_or_value
{"ids": ["A647", "B673", "H204", "O363", "D780"]}35 tokensCorrect
{"ids":["A647"]}11 tokensCorrect
{"ids":["A647"]}11 tokensCorrect
{"ids":["A647"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"L268","units":19,"price":7},{"id":"O539","units":4,"price":18},{"id":"Z355","units":12,"price":13},{"id":"E231","units":6,"price":13},{"id":"W361","units":11,"price":5}]Checked answer{"total": 494}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"B937","priority":8},{"id":"B676","priority":9},{"id":"N933","priority":9},{"id":"A582","priority":4},{"id":"N139","priority":5}]Checked answer{"ids": ["B676", "N933"]}wrong_schema_or_value
{"ids": ["B937", "N933"]}17 tokensCorrect
{"ids":["B676","N933"]}16 tokensCorrect
{"ids":["B676","N933"]}16 tokenswrong_schema_or_value
{"ids":["N933","B676"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id N703 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"N703","city":"Oslo","units":0},{"id":"L245","city":"Bern","units":10},{"id":"Q661","city":"Suva","units":19},{"id":"H199","city":"Lima","units":17},{"id":"B679","city":"Lima","units":3}]Checked answer{"city": "Oslo", "units": 0}Correct
{"city":"Oslo","units":0}11 tokensCorrect
{"city":"Oslo","units":0}11 tokensCorrect
{"city":"Oslo","units":0}11 tokensCorrect
{"city":"Oslo","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"C808","status":"pending"},{"id":"K238","status":"ready"},{"id":"Y737","status":"closed"},{"id":"E926","status":"closed"},{"id":"W909","status":"ready"}]Checked answer{"ids": ["K238", "W909"]}wrong_schema_or_value
{
"ids": ["C808", "K238", "Y737", "E926", "W909"]
}38 tokenswrong_schema_or_value
{"ids":["W909","K238"]}16 tokensCorrect
{"ids":["K238","W909"]}16 tokenswrong_schema_or_value
{"ids":["C808","W909","K238"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"B474","units":17,"price":6},{"id":"O658","units":1,"price":3},{"id":"L770","units":13,"price":3},{"id":"K392","units":13,"price":1},{"id":"Y564","units":17,"price":7}]Checked answer{"total": 276}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"D278","priority":6},{"id":"F714","priority":4},{"id":"Z597","priority":9},{"id":"F427","priority":2},{"id":"B625","priority":8}]Checked answer{"ids": ["Z597", "B625"]}invalid_json_object
[{"id":"F714","priority":4},{"id":"F427","priority":2},{"id":"B625","priority":8},{"id":"D278","priority":6},{"id":"Z597","priority":9}]62 tokenswrong_schema_or_value
{"ids":["D278","B625"]}16 tokenswrong_schema_or_value
{"ids":["D278","B625"]}16 tokenswrong_schema_or_value
{"ids":["Z597","D278"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"K345","city":"Perth","units":14},{"id":"K659","city":"Riga","units":5},{"id":"O284","city":"Perth","units":14},{"id":"R999","city":"Suva","units":7},{"id":"K864","city":"Kyoto","units":14}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"Z936","status":"closed"},{"id":"H675","status":"ready"},{"id":"S991","status":"ready"},{"id":"B889","status":"pending"},{"id":"K626","status":"pending"}]Checked answer{"ids": ["H675", "S991"]}invalid_json_object
[{"id":"H675","status":"ready"},{"id":"S991","status":"ready"},{"id":"B889","status":"pending"},{"id":"Z936","status":"closed"}]51 tokensinvalid_json_object
["H675","S991"]12 tokens
Correct
{"ids":["H675","S991"]}16 tokensinvalid_json_object
["ids":["H675","S991"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"E257","units":10,"price":19},{"id":"A193","units":16,"price":15},{"id":"E145","units":9,"price":19},{"id":"T757","units":19,"price":17},{"id":"H747","units":9,"price":6}]Checked answer{"total": 978}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"P946","priority":5},{"id":"T203","priority":1},{"id":"S823","priority":1},{"id":"W691","priority":3},{"id":"K869","priority":1}]Checked answer{"ids": ["P946", "W691"]}wrong_schema_or_value
{"ids": ["P946", "T203", "S823", "W691", "K869"]}35 tokensCorrect
{"ids":["P946","W691"]}16 tokensCorrect
{"ids":["P946","W691"]}16 tokenswrong_schema_or_value
{"ids":["P946","S823"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=F670. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"F670","city":"Accra","units":2},{"id":"S610","city":"Oslo","units":13},{"id":"S771","city":"Riga","units":2},{"id":"O172","city":"Oslo","units":7},{"id":"T997","city":"Oslo","units":19}]Checked answer{"city": "Accra", "units": 2}Correct
{"city": "Accra", "units": 2}14 tokensCorrect
{"city":"Accra","units":2}11 tokensCorrect
{"city":"Accra","units":2}11 tokensCorrect
{"city":"Accra","units":2}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"M273","status":"ready"},{"id":"X836","status":"ready"},{"id":"U594","status":"pending"},{"id":"C239","status":"closed"},{"id":"G615","status":"ready"}]Checked answer{"ids": ["G615", "M273", "X836"]}wrong_schema_or_value
{"ids": ["M273", "X836", "U594", "C239", "G615"]}35 tokenswrong_schema_or_value
{"ids":["C239","G615","M273","X836"]}26 tokensCorrect
{"ids":["G615","M273","X836"]}21 tokenswrong_schema_or_value
{"ids":["M273","G615","X836"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"R320","units":4,"price":4},{"id":"A955","units":18,"price":19},{"id":"L701","units":19,"price":7},{"id":"V462","units":9,"price":13},{"id":"F367","units":11,"price":6}]Checked answer{"total": 674}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"N355","priority":5},{"id":"J553","priority":6},{"id":"G765","priority":3},{"id":"D329","priority":4},{"id":"W961","priority":3}]Checked answer{"ids": ["J553", "N355"]}invalid_json_object
[{"id":"N355","priority":5},{"id":"J553","priority":6},{"id":"G765","priority":3},{"id":"D329","priority":4},{"id":"W961","priority":3}]62 tokensCorrect
{"ids":["J553","N355"]}16 tokensCorrect
{"ids":["J553","N355"]}16 tokensCorrect
{"ids":["J553","N355"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record B255. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"I918","city":"Riga","units":10},{"id":"U267","city":"Kyoto","units":9},{"id":"B255","city":"Bern","units":11},{"id":"H419","city":"Suva","units":12},{"id":"S134","city":"Lima","units":2}]Checked answer{"city": "Bern", "units": 11}invalid_json_object
[{"city":"Riga","units":10},{"city":"Bern","units":11},{"city":"Suva","units":12},{"city":"Lima","units":2}]44 tokensCorrect
{"city":"Bern","units":11}11 tokensCorrect
{"city":"Bern","units":11}11 tokensCorrect
{"city":"Bern","units":11}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"P718","status":"pending"},{"id":"K307","status":"pending"},{"id":"R189","status":"pending"},{"id":"K556","status":"closed"},{"id":"X432","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["K307", "K556", "P718", "R189", "X432"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"C644","units":18,"price":9},{"id":"O112","units":0,"price":7},{"id":"Q439","units":4,"price":18},{"id":"R930","units":5,"price":17},{"id":"S443","units":18,"price":17}]Checked answer{"total": 625}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"J293","priority":7},{"id":"L837","priority":5},{"id":"V906","priority":9},{"id":"C473","priority":6},{"id":"Q477","priority":4}]Checked answer{"ids": ["V906", "J293"]}wrong_schema_or_value
{"ids": ["J293", "V906", "C473", "Q477", "L837"]}35 tokensCorrect
{"ids":["V906","J293"]}16 tokensCorrect
{"ids":["V906","J293"]}16 tokensCorrect
{"ids":["V906","J293"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id E410 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"C389","city":"Lima","units":0},{"id":"W573","city":"Riga","units":2},{"id":"E410","city":"Accra","units":3},{"id":"J972","city":"Kyoto","units":15},{"id":"L927","city":"Riga","units":18}]Checked answer{"city": "Accra", "units": 3}wrong_schema_or_value
{"city":"Lima","units":0}11 tokensCorrect
{"city":"Accra","units":3}11 tokensCorrect
{"city":"Accra","units":3}11 tokensCorrect
{"city":"Accra","units":3}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"O840","status":"pending"},{"id":"Y647","status":"ready"},{"id":"D360","status":"pending"},{"id":"W694","status":"closed"},{"id":"E837","status":"closed"}]Checked answer{"ids": ["Y647"]}wrong_schema_or_value
{
"ids": ["O840", "Y647", "D360", "W694", "E837"]
}38 tokensCorrect
{"ids":["Y647"]}11 tokensCorrect
{"ids":["Y647"]}11 tokensCorrect
{"ids":["Y647"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"U451","units":3,"price":12},{"id":"N442","units":18,"price":1},{"id":"H599","units":7,"price":12},{"id":"C185","units":13,"price":2},{"id":"Z179","units":11,"price":6}]Checked answer{"total": 230}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Q388","priority":9},{"id":"C266","priority":1},{"id":"L525","priority":1},{"id":"V702","priority":7},{"id":"K380","priority":1}]Checked answer{"ids": ["Q388", "V702"]}invalid_json_object
[{"id":"Q388","priority":9},{"id":"C266","priority":1},{"id":"L525","priority":1},{"id":"V702","priority":7},{"id":"K380","priority":1}]62 tokensCorrect
{"ids":["Q388","V702"]}16 tokenswrong_schema_or_value
{"ids":["Q388","C266"]}16 tokensCorrect
{"ids":["Q388","V702"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to S367? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"S367","city":"Bern","units":13},{"id":"N742","city":"Bern","units":12},{"id":"S105","city":"Perth","units":14},{"id":"A324","city":"Kyoto","units":7},{"id":"W715","city":"Oslo","units":4}]Checked answer{"city": "Bern", "units": 13}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Bern","units":13}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Bern","units":13}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"N979","status":"pending"},{"id":"W631","status":"pending"},{"id":"C140","status":"closed"},{"id":"M328","status":"pending"},{"id":"F618","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"C140","status":"closed"},{"id":"F618","status":"closed"},{"id":"M328","status":"pending"},{"id":"N979","status":"pending"},{"id":"W631","status":"pending"}]63 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"S976","units":19,"price":4},{"id":"N622","units":5,"price":8},{"id":"T354","units":4,"price":5},{"id":"A229","units":6,"price":14},{"id":"Q191","units":13,"price":19}]Checked answer{"total": 467}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"K506","priority":1},{"id":"A709","priority":9},{"id":"G256","priority":8},{"id":"R345","priority":4},{"id":"R751","priority":9}]Checked answer{"ids": ["A709", "R751"]}wrong_schema_or_value
{"ids": ["A709", "G256", "R345", "R751", "K506"]}35 tokensCorrect
{"ids":["A709","R751"]}16 tokensCorrect
{"ids":["A709","R751"]}16 tokenswrong_schema_or_value
{"ids":["A709","R345"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"O779","city":"Suva","units":7},{"id":"I493","city":"Bern","units":8},{"id":"B767","city":"Bern","units":1},{"id":"H839","city":"Kyoto","units":11},{"id":"O743","city":"Kyoto","units":7}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Suva","units":7}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Q791","status":"pending"},{"id":"T683","status":"closed"},{"id":"E631","status":"pending"},{"id":"A499","status":"closed"},{"id":"Q390","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["A499", "E631", "Q390", "Q791", "T683"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"H785","units":3,"price":18},{"id":"Q846","units":17,"price":13},{"id":"C568","units":17,"price":19},{"id":"O627","units":17,"price":10},{"id":"V530","units":16,"price":2}]Checked answer{"total": 800}invalid_json_object
{
"total": 18 + 13 + 19 + 10 + 2 = 68
}30 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"V879","priority":5},{"id":"A180","priority":9},{"id":"J850","priority":9},{"id":"K643","priority":9},{"id":"D676","priority":9}]Checked answer{"ids": ["A180", "D676"]}invalid_json_object
[{"id":"J850","priority":9},{"id":"K643","priority":9},{"id":"V879","priority":5},{"id":"A180","priority":9},{"id":"D676","priority":9}]62 tokensCorrect
{"ids":["A180","D676"]}16 tokenswrong_schema_or_value
{"ids":["A180","K643"]}16 tokenswrong_schema_or_value
{"ids":["J850","D676"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Y336. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"P670","city":"Oslo","units":11},{"id":"Y336","city":"Riga","units":5},{"id":"Y490","city":"Accra","units":0},{"id":"Z164","city":"Suva","units":13},{"id":"U459","city":"Riga","units":7}]Checked answer{"city": "Riga", "units": 5}wrong_schema_or_value
{
"city": "Oslo",
"units": 11
}19 tokensCorrect
{"city":"Riga","units":5}11 tokensCorrect
{"city":"Riga","units":5}11 tokensCorrect
{"city":"Riga","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"P903","status":"pending"},{"id":"T908","status":"pending"},{"id":"D803","status":"closed"},{"id":"M161","status":"pending"},{"id":"I782","status":"ready"}]Checked answer{"ids": ["I782"]}wrong_schema_or_value
{"ids": ["D803", "I782", "M161", "P903", "T908"]}35 tokensCorrect
{"ids":["I782"]}11 tokensCorrect
{"ids":["I782"]}11 tokensCorrect
{"ids":["I782"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"C268","units":9,"price":3},{"id":"W661","units":5,"price":17},{"id":"X534","units":5,"price":13},{"id":"R204","units":15,"price":13},{"id":"E214","units":2,"price":15}]Checked answer{"total": 402}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"C972","priority":9},{"id":"E607","priority":2},{"id":"B210","priority":9},{"id":"T345","priority":6},{"id":"W585","priority":5}]Checked answer{"ids": ["B210", "C972"]}wrong_schema_or_value
{"ids": ["C972", "B210", "T345", "W585"]}29 tokenswrong_schema_or_value
{"ids":["C972","B210"]}16 tokenswrong_schema_or_value
{"ids":["C972","B210"]}16 tokensCorrect
{"ids":["B210","C972"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id M498 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"L674","city":"Kyoto","units":8},{"id":"M498","city":"Perth","units":9},{"id":"L274","city":"Accra","units":9},{"id":"P477","city":"Lima","units":1},{"id":"B678","city":"Riga","units":7}]Checked answer{"city": "Perth", "units": 9}wrong_schema_or_value
{"city":"Kyoto","units":8}12 tokensCorrect
{"city":"Perth","units":9}11 tokensCorrect
{"city":"Perth","units":9}11 tokensCorrect
{"city":"Perth","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"P357","status":"pending"},{"id":"D573","status":"closed"},{"id":"I125","status":"pending"},{"id":"J328","status":"closed"},{"id":"M594","status":"ready"}]Checked answer{"ids": ["M594"]}wrong_schema_or_value
{
"ids": ["M594", "P357", "I125", "J328", "D573"]
}38 tokensCorrect
{"ids":["M594"]}11 tokensCorrect
{"ids":["M594"]}11 tokensCorrect
{"ids":["M594"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"Y187","units":1,"price":18},{"id":"B451","units":16,"price":19},{"id":"D697","units":4,"price":15},{"id":"A208","units":0,"price":19},{"id":"J644","units":11,"price":8}]Checked answer{"total": 470}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"P311","priority":6},{"id":"I472","priority":2},{"id":"Z255","priority":3},{"id":"U743","priority":8},{"id":"U675","priority":4}]Checked answer{"ids": ["U743", "P311"]}invalid_json_object
[{"id":"P311","priority":6},{"id":"U675","priority":4}]26 tokensCorrect
{"ids":["U743","P311"]}16 tokenswrong_schema_or_value
{"ids":["P311","U743"]}16 tokenswrong_schema_or_value
{"ids":["P311","Z255"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Y760? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"Z440","city":"Perth","units":7},{"id":"V427","city":"Kyoto","units":15},{"id":"T922","city":"Lima","units":13},{"id":"J993","city":"Bern","units":9},{"id":"Y760","city":"Lima","units":12}]Checked answer{"city": "Lima", "units": 12}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Lima","units":12}12 tokensCorrect
{"city":"Lima","units":12}12 tokensCorrect
{"city":"Lima","units":12}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"V557","status":"closed"},{"id":"S397","status":"pending"},{"id":"Q122","status":"closed"},{"id":"V117","status":"pending"},{"id":"T326","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"V557"},{"id":"S397"},{"id":"Q122"},{"id":"V117"},{"id":"T326"}]43 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"G775","units":17,"price":1},{"id":"X765","units":6,"price":4},{"id":"D718","units":8,"price":18},{"id":"P933","units":8,"price":7},{"id":"Q952","units":13,"price":17}]Checked answer{"total": 462}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"H533","priority":1},{"id":"Q520","priority":7},{"id":"D189","priority":7},{"id":"H539","priority":1},{"id":"R958","priority":6}]Checked answer{"ids": ["D189", "Q520"]}wrong_schema_or_value
{"ids": ["H533", "H539"]}17 tokensCorrect
{"ids":["D189","Q520"]}16 tokensCorrect
{"ids":["D189","Q520"]}16 tokensCorrect
{"ids":["D189","Q520"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=H971. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"C751","city":"Suva","units":9},{"id":"H971","city":"Suva","units":9},{"id":"T494","city":"Bern","units":18},{"id":"W232","city":"Oslo","units":15},{"id":"O319","city":"Oslo","units":19}]Checked answer{"city": "Suva", "units": 9}Correct
{"city":"Suva","units":9}11 tokensCorrect
{"city":"Suva","units":9}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Suva","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"P429","status":"pending"},{"id":"A267","status":"pending"},{"id":"O521","status":"pending"},{"id":"D136","status":"ready"},{"id":"I461","status":"ready"}]Checked answer{"ids": ["D136", "I461"]}wrong_schema_or_value
{"ids": ["A267", "D136", "I461", "P429", "O521"]}35 tokenswrong_schema_or_value
{"ids":["I461","D136"]}16 tokensCorrect
{"ids":["D136","I461"]}16 tokenswrong_schema_or_value
{"ids":["I461","D136"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"Y698","units":7,"price":15},{"id":"J774","units":7,"price":4},{"id":"F814","units":17,"price":7},{"id":"H340","units":13,"price":17},{"id":"J883","units":15,"price":14}]Checked answer{"total": 683}invalid_json_object
{
"total": 15 + 4 + 7 + 17 + 14 + 15 + 17 + 13 + 15 + 14 + 17 + 13 + 15 + 17 + 13 + 15 + 14 + 17 + 13 + 15 + 14 + 17 + 13 + 96 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"H256","priority":3},{"id":"Y156","priority":8},{"id":"P202","priority":8},{"id":"F870","priority":9},{"id":"R336","priority":9}]Checked answer{"ids": ["F870", "R336"]}invalid_json_object
[{"id":"H256","priority":3},{"id":"P202","priority":8},{"id":"F870","priority":9},{"id":"R336","priority":9}]50 tokensCorrect
{"ids":["F870","R336"]}16 tokenswrong_schema_or_value
{"ids":["R336","F870"]}16 tokensCorrect
{"ids":["F870","R336"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"V913","city":"Oslo","units":17},{"id":"L140","city":"Accra","units":15},{"id":"D396","city":"Lima","units":9},{"id":"D340","city":"Perth","units":12},{"id":"P377","city":"Oslo","units":13}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Oslo","units":17}]13 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"C730","status":"pending"},{"id":"H909","status":"closed"},{"id":"X156","status":"closed"},{"id":"L652","status":"closed"},{"id":"Y179","status":"ready"}]Checked answer{"ids": ["Y179"]}wrong_schema_or_value
{"ids": ["C730", "H909", "X156", "L652", "Y179"]}35 tokensCorrect
{"ids":["Y179"]}11 tokensCorrect
{"ids":["Y179"]}11 tokensCorrect
{"ids":["Y179"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"B268","units":15,"price":13},{"id":"L180","units":12,"price":6},{"id":"R259","units":6,"price":12},{"id":"O424","units":19,"price":2},{"id":"A344","units":4,"price":1}]Checked answer{"total": 381}wrong_schema_or_value
{"total": 130}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"O853","priority":2},{"id":"L629","priority":6},{"id":"H245","priority":7},{"id":"Z130","priority":8},{"id":"R443","priority":4}]Checked answer{"ids": ["Z130", "H245"]}wrong_schema_or_value
{"ids": ["O853", "L629", "H245", "Z130", "R443"]}35 tokenswrong_schema_or_value
{"ids":["H245","Z130"]}16 tokenswrong_schema_or_value
{"ids":["H245","Z130"]}16 tokenswrong_schema_or_value
{"ids":["H245","Z130"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id J801 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"J801","city":"Oslo","units":5},{"id":"H326","city":"Lima","units":11},{"id":"J650","city":"Oslo","units":13},{"id":"T265","city":"Riga","units":18},{"id":"X983","city":"Oslo","units":0}]Checked answer{"city": "Oslo", "units": 5}wrong_schema_or_value
{"city":"Oslo","units":13}12 tokensCorrect
{"city":"Oslo","units":5}11 tokensCorrect
{"city":"Oslo","units":5}11 tokensCorrect
{"city":"Oslo","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"C362","status":"ready"},{"id":"P333","status":"closed"},{"id":"G441","status":"closed"},{"id":"M934","status":"closed"},{"id":"F390","status":"closed"}]Checked answer{"ids": ["C362"]}wrong_schema_or_value
{
"ids": ["C362", "F390", "M934", "P333", "G441"]
}38 tokensCorrect
{"ids":["C362"]}11 tokensCorrect
{"ids":["C362"]}11 tokensCorrect
{"ids":["C362"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"K523","units":2,"price":2},{"id":"U939","units":7,"price":16},{"id":"S460","units":11,"price":19},{"id":"W894","units":2,"price":2},{"id":"H930","units":16,"price":16}]Checked answer{"total": 585}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"S808","priority":6},{"id":"M561","priority":7},{"id":"A569","priority":4},{"id":"C295","priority":2},{"id":"F292","priority":8}]Checked answer{"ids": ["F292", "M561"]}invalid_json_object
[{"id":"S808","priority":6},{"id":"M561","priority":7},{"id":"A569","priority":4},{"id":"C295","priority":2},{"id":"F292","priority":8}]62 tokensCorrect
{"ids":["F292","M561"]}16 tokensCorrect
{"ids":["F292","M561"]}16 tokenswrong_schema_or_value
{"ids":["F292","S808"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to R789? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"R789","city":"Accra","units":11},{"id":"N243","city":"Oslo","units":6},{"id":"N876","city":"Riga","units":2},{"id":"B215","city":"Bern","units":3},{"id":"H124","city":"Lima","units":7}]Checked answer{"city": "Accra", "units": 11}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Accra","units":11}12 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Accra","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"M893","status":"ready"},{"id":"S221","status":"pending"},{"id":"E106","status":"closed"},{"id":"D406","status":"ready"},{"id":"I171","status":"closed"}]Checked answer{"ids": ["D406", "M893"]}invalid_json_object
[{"id":"D406","status":"ready"},{"id":"I171","status":"closed"},{"id":"M893","status":"ready"},{"id":"S221","status":"pending"},{"id":"E106","status":"closed"}]63 tokensCorrect
{"ids":["D406","M893"]}16 tokensCorrect
{"ids":["D406","M893"]}16 tokensinvalid_json_object
["ids":["D406","M893"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"J443","units":3,"price":14},{"id":"C490","units":2,"price":8},{"id":"G761","units":14,"price":18},{"id":"J821","units":0,"price":16},{"id":"F769","units":0,"price":1}]Checked answer{"total": 310}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"L807","priority":9},{"id":"C310","priority":4},{"id":"Q623","priority":8},{"id":"K554","priority":3},{"id":"O150","priority":6}]Checked answer{"ids": ["L807", "Q623"]}Correct
{"ids": ["L807", "Q623"]}17 tokensCorrect
{"ids":["L807","Q623"]}16 tokensCorrect
{"ids":["L807","Q623"]}16 tokensCorrect
{"ids":["L807","Q623"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=I114. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"U814","city":"Perth","units":9},{"id":"I114","city":"Bern","units":19},{"id":"N374","city":"Lima","units":12},{"id":"W298","city":"Perth","units":17},{"id":"Q987","city":"Perth","units":12}]Checked answer{"city": "Bern", "units": 19}wrong_schema_or_value
{"city":"Perth","units":9}11 tokensCorrect
{"city":"Bern","units":19}11 tokensCorrect
{"city":"Bern","units":19}11 tokensCorrect
{"city":"Bern","units":19}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"K908","status":"ready"},{"id":"U498","status":"ready"},{"id":"T684","status":"pending"},{"id":"E516","status":"closed"},{"id":"Y735","status":"ready"}]Checked answer{"ids": ["K908", "U498", "Y735"]}wrong_schema_or_value
{"ids": ["K908", "U498", "T684", "E516", "Y735"]}35 tokenswrong_schema_or_value
{"ids":["E516","K908","Y735","U498"]}26 tokenswrong_schema_or_value
{"ids":["K908","Y735"]}16 tokenswrong_schema_or_value
{"ids":["K908","E516","Y735"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"Y767","units":16,"price":8},{"id":"S696","units":0,"price":4},{"id":"N964","units":14,"price":6},{"id":"Q989","units":7,"price":10},{"id":"M107","units":6,"price":1}]Checked answer{"total": 288}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"N581","priority":4},{"id":"H776","priority":4},{"id":"J601","priority":9},{"id":"H644","priority":5},{"id":"M930","priority":2}]Checked answer{"ids": ["J601", "H644"]}invalid_json_object
[{"id":"H776","priority":4},{"id":"H644","priority":5},{"id":"M930","priority":2},{"id":"N581","priority":4},{"id":"J601","priority":9}]62 tokensCorrect
{"ids":["J601","H644"]}16 tokensCorrect
{"ids":["J601","H644"]}16 tokenswrong_schema_or_value
{"ids":["J601","N581"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record D316. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Y922","city":"Suva","units":1},{"id":"D316","city":"Kyoto","units":17},{"id":"G878","city":"Perth","units":0},{"id":"V585","city":"Riga","units":17},{"id":"T785","city":"Bern","units":3}]Checked answer{"city": "Kyoto", "units": 17}Correct
{
"city": "Kyoto",
"units": 17
}20 tokensCorrect
{"city":"Kyoto","units":17}13 tokensCorrect
{"city":"Kyoto","units":17}13 tokensCorrect
{"city":"Kyoto","units":17}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"X686","status":"ready"},{"id":"Q605","status":"closed"},{"id":"T909","status":"pending"},{"id":"L896","status":"pending"},{"id":"D922","status":"closed"}]Checked answer{"ids": ["X686"]}wrong_schema_or_value
{"ids": ["X686", "Q605", "T909", "L896", "D922"]}35 tokensCorrect
{"ids":["X686"]}11 tokensCorrect
{"ids":["X686"]}11 tokensCorrect
{"ids":["X686"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"A671","units":12,"price":9},{"id":"P459","units":5,"price":7},{"id":"F143","units":8,"price":11},{"id":"E527","units":18,"price":2},{"id":"Y483","units":7,"price":11}]Checked answer{"total": 344}invalid_json_object
{"total": 12*9 + 5*7 + 8*11 + 18*2 + 7*11}33 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"E718","priority":7},{"id":"H579","priority":4},{"id":"Q637","priority":9},{"id":"X943","priority":1},{"id":"T560","priority":5}]Checked answer{"ids": ["Q637", "E718"]}wrong_schema_or_value
{"ids": ["H579", "Q637", "X943", "T560", "E718"]}35 tokenswrong_schema_or_value
{"ids":["E718","Q637"]}16 tokenswrong_schema_or_value
{"ids":["E718","Q637"]}16 tokensCorrect
{"ids":["Q637","E718"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"I834","city":"Oslo","units":18},{"id":"T940","city":"Perth","units":10},{"id":"C670","city":"Bern","units":14},{"id":"W954","city":"Kyoto","units":7},{"id":"U530","city":"Accra","units":2}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Oslo","units":18}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"R372","status":"pending"},{"id":"W679","status":"ready"},{"id":"X576","status":"closed"},{"id":"V492","status":"closed"},{"id":"Q280","status":"ready"}]Checked answer{"ids": ["Q280", "W679"]}wrong_schema_or_value
{
"ids": ["R372", "W679", "X576", "V492", "Q280"]
}38 tokensCorrect
{"ids":["Q280","W679"]}16 tokensCorrect
{"ids":["Q280","W679"]}16 tokenswrong_schema_or_value
{"ids":["W679","Q280"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"V918","units":8,"price":4},{"id":"K273","units":3,"price":7},{"id":"U916","units":11,"price":17},{"id":"K396","units":13,"price":11},{"id":"D744","units":14,"price":11}]Checked answer{"total": 537}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"J801","priority":5},{"id":"W649","priority":2},{"id":"S241","priority":7},{"id":"E474","priority":6},{"id":"A484","priority":1}]Checked answer{"ids": ["S241", "E474"]}invalid_json_object
[{"id":"J801","priority":5},{"id":"W649","priority":2},{"id":"S241","priority":7},{"id":"E474","priority":6},{"id":"A484","priority":1}]62 tokensCorrect
{"ids":["S241","E474"]}16 tokensCorrect
{"ids":["S241","E474"]}16 tokensCorrect
{"ids":["S241","E474"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to E560? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"V724","city":"Suva","units":19},{"id":"B143","city":"Perth","units":9},{"id":"T185","city":"Accra","units":0},{"id":"E560","city":"Lima","units":13},{"id":"J901","city":"Kyoto","units":17}]Checked answer{"city": "Lima", "units": 13}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Lima","units":13}12 tokensCorrect
{"city":"Lima","units":13}12 tokensCorrect
{"city":"Lima","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"H990","status":"closed"},{"id":"W595","status":"ready"},{"id":"J848","status":"ready"},{"id":"Z947","status":"closed"},{"id":"Q446","status":"closed"}]Checked answer{"ids": ["J848", "W595"]}invalid_json_object
[{"id":"W595","status":"ready"},{"id":"J848","status":"ready"},{"id":"Z947","status":"closed"},{"id":"Q446","status":"closed"}]51 tokensinvalid_json_object
["J848","W595"]12 tokens
Correct
{"ids":["J848","W595"]}16 tokensinvalid_json_object
["ids":["J848","W595"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"W309","units":16,"price":14},{"id":"M107","units":9,"price":7},{"id":"D117","units":1,"price":5},{"id":"G606","units":9,"price":13},{"id":"U558","units":1,"price":12}]Checked answer{"total": 421}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Y679","priority":5},{"id":"B249","priority":5},{"id":"K535","priority":5},{"id":"C902","priority":1},{"id":"V659","priority":8}]Checked answer{"ids": ["V659", "B249"]}wrong_schema_or_value
{"ids": ["Y679", "B249", "K535", "C902", "V659"]}35 tokensCorrect
{"ids":["V659","B249"]}16 tokenswrong_schema_or_value
{"ids":["V659","Y679"]}16 tokenswrong_schema_or_value
{"ids":["V659","K535"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=B636. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"Q794","city":"Accra","units":0},{"id":"B402","city":"Accra","units":9},{"id":"B822","city":"Oslo","units":3},{"id":"B636","city":"Perth","units":19},{"id":"H580","city":"Suva","units":17}]Checked answer{"city": "Perth", "units": 19}Correct
{"city":"Perth","units":19}12 tokensCorrect
{"city":"Perth","units":19}12 tokensCorrect
{"city":"Perth","units":19}12 tokensCorrect
{"city":"Perth","units":19}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"U972","status":"pending"},{"id":"K659","status":"closed"},{"id":"C741","status":"pending"},{"id":"W216","status":"ready"},{"id":"L816","status":"ready"}]Checked answer{"ids": ["L816", "W216"]}wrong_schema_or_value
{"ids": ["L816", "U972", "C741", "K659", "W216"]}35 tokensCorrect
{"ids":["L816","W216"]}16 tokensCorrect
{"ids":["L816","W216"]}16 tokensCorrect
{"ids":["L816","W216"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"S249","units":5,"price":2},{"id":"D389","units":10,"price":5},{"id":"A698","units":0,"price":19},{"id":"Y820","units":9,"price":6},{"id":"H852","units":15,"price":14}]Checked answer{"total": 324}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"X257","priority":8},{"id":"X354","priority":4},{"id":"R638","priority":5},{"id":"V760","priority":5},{"id":"U725","priority":1}]Checked answer{"ids": ["X257", "R638"]}invalid_json_object
[{"id":"X257"},{"id":"X354"},{"id":"R638"},{"id":"V760"},{"id":"U725"}]43 tokensCorrect
{"ids":["X257","R638"]}16 tokenswrong_schema_or_value
{"ids":["X257","X354"]}16 tokensCorrect
{"ids":["X257","R638"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record D689. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"D689","city":"Suva","units":15},{"id":"X624","city":"Riga","units":17},{"id":"I226","city":"Suva","units":14},{"id":"S954","city":"Perth","units":13},{"id":"R460","city":"Riga","units":6}]Checked answer{"city": "Suva", "units": 15}Correct
{
"city": "Suva",
"units": 15
}19 tokensCorrect
{"city":"Suva","units":15}12 tokensCorrect
{"city":"Suva","units":15}12 tokensCorrect
{"city":"Suva","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"H538","status":"ready"},{"id":"J246","status":"ready"},{"id":"U294","status":"pending"},{"id":"P278","status":"pending"},{"id":"W747","status":"ready"}]Checked answer{"ids": ["H538", "J246", "W747"]}wrong_schema_or_value
{"ids": ["H538", "J246", "U294", "P278", "W747"]}35 tokenswrong_schema_or_value
{"ids":["H538","W747","J246"]}21 tokenswrong_schema_or_value
{"ids":["H538","W747","J246"]}21 tokensCorrect
{"ids":["H538","J246","W747"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"D711","units":9,"price":13},{"id":"O985","units":16,"price":4},{"id":"Z339","units":13,"price":8},{"id":"J983","units":16,"price":11},{"id":"Q336","units":14,"price":9}]Checked answer{"total": 587}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"R561","priority":5},{"id":"J426","priority":7},{"id":"Z414","priority":9},{"id":"L551","priority":9},{"id":"V328","priority":6}]Checked answer{"ids": ["L551", "Z414"]}wrong_schema_or_value
{"ids": ["L551", "V328", "Z414", "J426", "R561"]}35 tokensCorrect
{"ids":["L551","Z414"]}16 tokensCorrect
{"ids":["L551","Z414"]}16 tokensCorrect
{"ids":["L551","Z414"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id T887 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"U585","city":"Perth","units":4},{"id":"T887","city":"Perth","units":17},{"id":"V950","city":"Suva","units":13},{"id":"I993","city":"Kyoto","units":7},{"id":"V843","city":"Oslo","units":2}]Checked answer{"city": "Perth", "units": 17}wrong_schema_or_value
{"city":"Perth","units":4}11 tokensCorrect
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"N860","status":"closed"},{"id":"Z407","status":"closed"},{"id":"A135","status":"pending"},{"id":"T907","status":"pending"},{"id":"L468","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{
"ids": ["A135", "L468", "N860", "T907", "Z407"]
}38 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"X312","units":0,"price":13},{"id":"O185","units":10,"price":1},{"id":"A365","units":10,"price":7},{"id":"A359","units":2,"price":13},{"id":"S290","units":19,"price":1}]Checked answer{"total": 125}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"K475","priority":3},{"id":"S105","priority":1},{"id":"G265","priority":2},{"id":"B553","priority":4},{"id":"K921","priority":6}]Checked answer{"ids": ["K921", "B553"]}invalid_json_object
[{"id":"K475","priority":3},{"id":"S105","priority":1},{"id":"G265","priority":2},{"id":"B553","priority":4},{"id":"K921","priority":6}]62 tokensCorrect
{"ids":["K921","B553"]}16 tokensCorrect
{"ids":["K921","B553"]}16 tokensCorrect
{"ids":["K921","B553"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"A308","city":"Riga","units":7},{"id":"J120","city":"Bern","units":18},{"id":"X612","city":"Bern","units":2},{"id":"U739","city":"Bern","units":8},{"id":"Q494","city":"Suva","units":10}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"M738","status":"pending"},{"id":"R662","status":"closed"},{"id":"I556","status":"pending"},{"id":"Q960","status":"closed"},{"id":"V743","status":"ready"}]Checked answer{"ids": ["V743"]}invalid_json_object
[{"id":"V743","status":"ready"}]15 tokensCorrect
{"ids":["V743"]}11 tokensCorrect
{"ids":["V743"]}11 tokensinvalid_json_object
["ids":["V743"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"R874","units":19,"price":18},{"id":"K325","units":2,"price":12},{"id":"A738","units":19,"price":7},{"id":"L228","units":8,"price":18},{"id":"K272","units":9,"price":2}]Checked answer{"total": 661}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"K344","priority":8},{"id":"M378","priority":6},{"id":"V338","priority":2},{"id":"V105","priority":3},{"id":"J856","priority":1}]Checked answer{"ids": ["K344", "M378"]}wrong_schema_or_value
{"ids": ["V105", "V338", "K344", "M378", "J856"]}35 tokensCorrect
{"ids":["K344","M378"]}16 tokensCorrect
{"ids":["K344","M378"]}16 tokensCorrect
{"ids":["K344","M378"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=E672. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"G855","city":"Riga","units":9},{"id":"J150","city":"Bern","units":8},{"id":"E672","city":"Oslo","units":10},{"id":"T617","city":"Accra","units":12},{"id":"R701","city":"Bern","units":0}]Checked answer{"city": "Oslo", "units": 10}Correct
{"city":"Oslo","units":10}12 tokensCorrect
{"city":"Oslo","units":10}12 tokensCorrect
{"city":"Oslo","units":10}12 tokensCorrect
{"city":"Oslo","units":10}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"O804","status":"ready"},{"id":"M638","status":"closed"},{"id":"F424","status":"ready"},{"id":"A448","status":"pending"},{"id":"D336","status":"ready"}]Checked answer{"ids": ["D336", "F424", "O804"]}wrong_schema_or_value
{"ids": ["A448", "D336", "F424", "O804", "M638"]}35 tokensCorrect
{"ids":["D336","F424","O804"]}21 tokensCorrect
{"ids":["D336","F424","O804"]}21 tokenswrong_schema_or_value
{"ids":["F424","D336","O804"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"X933","units":16,"price":16},{"id":"V222","units":14,"price":10},{"id":"B838","units":12,"price":6},{"id":"Z162","units":14,"price":19},{"id":"Z644","units":15,"price":13}]Checked answer{"total": 929}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"R723","priority":4},{"id":"F289","priority":8},{"id":"Y855","priority":7},{"id":"R559","priority":9},{"id":"R993","priority":2}]Checked answer{"ids": ["R559", "F289"]}invalid_json_object
[{"id":"R723","priority":4},{"id":"R559","priority":9},{"id":"R993","priority":2}]38 tokenswrong_schema_or_value
{"ids":["F289","R559"]}16 tokenswrong_schema_or_value
{"ids":["F289","R559"]}16 tokenswrong_schema_or_value
{"ids":["F289","R559"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Q492. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"O313","city":"Oslo","units":2},{"id":"P767","city":"Lima","units":14},{"id":"X400","city":"Kyoto","units":6},{"id":"Q492","city":"Bern","units":13},{"id":"D710","city":"Bern","units":0}]Checked answer{"city": "Bern", "units": 13}Correct
{
"city": "Bern",
"units": 13
}18 tokensCorrect
{"city":"Bern","units":13}11 tokensCorrect
{"city":"Bern","units":13}11 tokensCorrect
{"city":"Bern","units":13}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"C245","status":"pending"},{"id":"C773","status":"ready"},{"id":"M701","status":"ready"},{"id":"E451","status":"closed"},{"id":"G492","status":"pending"}]Checked answer{"ids": ["C773", "M701"]}wrong_schema_or_value
{"ids": ["C245", "C773", "M701", "E451", "G492"]}35 tokensCorrect
{"ids":["C773","M701"]}16 tokensCorrect
{"ids":["C773","M701"]}16 tokensCorrect
{"ids":["C773","M701"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"I264","units":10,"price":15},{"id":"A512","units":19,"price":4},{"id":"E748","units":4,"price":11},{"id":"M684","units":14,"price":18},{"id":"V559","units":19,"price":5}]Checked answer{"total": 617}wrong_schema_or_value
{"total": 150}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"I833","priority":2},{"id":"J695","priority":7},{"id":"U662","priority":3},{"id":"K981","priority":7},{"id":"X708","priority":3}]Checked answer{"ids": ["J695", "K981"]}wrong_schema_or_value
{"ids": ["U662", "J695", "I833", "K981", "X708"]}35 tokenswrong_schema_or_value
{"ids":["K981","J695"]}16 tokenswrong_schema_or_value
{"ids":["K981","J695"]}16 tokenswrong_schema_or_value
{"ids":["K981","J695"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id B802 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"B802","city":"Perth","units":11},{"id":"F425","city":"Lima","units":15},{"id":"G278","city":"Accra","units":12},{"id":"N884","city":"Kyoto","units":6},{"id":"Q806","city":"Riga","units":5}]Checked answer{"city": "Perth", "units": 11}Correct
{"city":"Perth","units":11}12 tokensCorrect
{"city":"Perth","units":11}12 tokensCorrect
{"city":"Perth","units":11}12 tokensCorrect
{"city":"Perth","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"I240","status":"closed"},{"id":"E705","status":"closed"},{"id":"C478","status":"pending"},{"id":"B687","status":"ready"},{"id":"W166","status":"ready"}]Checked answer{"ids": ["B687", "W166"]}wrong_schema_or_value
{
"ids": ["B687", "C478", "E705", "I240", "W166"]
}38 tokensCorrect
{"ids":["B687","W166"]}16 tokensCorrect
{"ids":["B687","W166"]}16 tokensCorrect
{"ids":["B687","W166"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"J722","units":6,"price":15},{"id":"W864","units":19,"price":19},{"id":"P912","units":7,"price":13},{"id":"D450","units":9,"price":17},{"id":"G957","units":4,"price":11}]Checked answer{"total": 739}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"V725","priority":9},{"id":"S559","priority":5},{"id":"W920","priority":6},{"id":"B109","priority":2},{"id":"W469","priority":9}]Checked answer{"ids": ["V725", "W469"]}invalid_json_object
[{"id":"V725","priority":9},{"id":"W469","priority":9}]26 tokensCorrect
{"ids":["V725","W469"]}16 tokenswrong_schema_or_value
{"ids":["W469","V725"]}16 tokensCorrect
{"ids":["V725","W469"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to U863? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"F924","city":"Accra","units":8},{"id":"F534","city":"Kyoto","units":2},{"id":"U863","city":"Kyoto","units":14},{"id":"K831","city":"Bern","units":16},{"id":"F479","city":"Perth","units":10}]Checked answer{"city": "Kyoto", "units": 14}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Kyoto","units":14}13 tokensCorrect
{"city":"Kyoto","units":14}13 tokensCorrect
{"city":"Kyoto","units":14}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"P167","status":"ready"},{"id":"F520","status":"closed"},{"id":"J600","status":"closed"},{"id":"P345","status":"pending"},{"id":"U928","status":"pending"}]Checked answer{"ids": ["P167"]}invalid_json_object
[{"id":"P167","status":"ready"},{"id":"F520","status":"closed"},{"id":"J600","status":"closed"},{"id":"P345","status":"pending"},{"id":"U928","status":"pending"}]63 tokensCorrect
{"ids":["P167"]}11 tokensCorrect
{"ids":["P167"]}11 tokensinvalid_json_object
["ids":["P167"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"R293","units":18,"price":18},{"id":"M242","units":9,"price":18},{"id":"A662","units":9,"price":15},{"id":"X519","units":6,"price":14},{"id":"J895","units":7,"price":18}]Checked answer{"total": 831}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"A362","priority":6},{"id":"U992","priority":1},{"id":"K126","priority":9},{"id":"K507","priority":3},{"id":"Z431","priority":7}]Checked answer{"ids": ["K126", "Z431"]}wrong_schema_or_value
{"ids": ["U992", "K126"]}17 tokenswrong_schema_or_value
{"ids":["K126","A362"]}16 tokenswrong_schema_or_value
{"ids":["K126","A362"]}16 tokenswrong_schema_or_value
{"ids":["K126","A362"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"E334","city":"Accra","units":19},{"id":"N510","city":"Bern","units":2},{"id":"C771","city":"Riga","units":17},{"id":"P658","city":"Perth","units":16},{"id":"C827","city":"Suva","units":0}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Accra","units":19}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"D831","status":"closed"},{"id":"W418","status":"closed"},{"id":"Z293","status":"closed"},{"id":"E221","status":"ready"},{"id":"P524","status":"pending"}]Checked answer{"ids": ["E221"]}wrong_schema_or_value
{"ids": ["E221", "P524", "D831", "W418", "Z293"]}35 tokensCorrect
{"ids":["E221"]}11 tokensCorrect
{"ids":["E221"]}11 tokensCorrect
{"ids":["E221"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"J943","units":8,"price":13},{"id":"E489","units":6,"price":9},{"id":"F389","units":12,"price":3},{"id":"I154","units":4,"price":1},{"id":"P544","units":3,"price":18}]Checked answer{"total": 252}wrong_schema_or_value
{
"total": 104
}12 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"Y208","priority":4},{"id":"W807","priority":4},{"id":"Z591","priority":6},{"id":"D326","priority":1},{"id":"X912","priority":8}]Checked answer{"ids": ["X912", "Z591"]}invalid_json_object
[{"id":"Y208","priority":4},{"id":"Z591","priority":6},{"id":"D326","priority":1},{"id":"X912","priority":8}]50 tokensCorrect
{"ids":["X912","Z591"]}16 tokensCorrect
{"ids":["X912","Z591"]}16 tokensCorrect
{"ids":["X912","Z591"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record U440. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"U742","city":"Lima","units":9},{"id":"T738","city":"Suva","units":11},{"id":"B350","city":"Riga","units":1},{"id":"J308","city":"Perth","units":10},{"id":"U440","city":"Suva","units":11}]Checked answer{"city": "Suva", "units": 11}Correct
{
"city": "Suva",
"units": 11
}19 tokensCorrect
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Suva","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"I607","status":"closed"},{"id":"X911","status":"pending"},{"id":"U203","status":"closed"},{"id":"N335","status":"closed"},{"id":"X790","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["U203", "N335", "X911", "X790", "I607"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"P154","units":19,"price":7},{"id":"V361","units":19,"price":15},{"id":"G223","units":8,"price":13},{"id":"M712","units":15,"price":9},{"id":"L276","units":6,"price":6}]Checked answer{"total": 693}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"E677","priority":7},{"id":"J496","priority":2},{"id":"Y825","priority":1},{"id":"Z651","priority":5},{"id":"D700","priority":5}]Checked answer{"ids": ["E677", "D700"]}wrong_schema_or_value
{"ids": ["Y825", "J496", "E677", "Z651", "D700"]}35 tokensCorrect
{"ids":["E677","D700"]}16 tokensCorrect
{"ids":["E677","D700"]}16 tokensCorrect
{"ids":["E677","D700"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id B731 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"O806","city":"Lima","units":6},{"id":"R119","city":"Accra","units":14},{"id":"G675","city":"Oslo","units":2},{"id":"Y811","city":"Bern","units":15},{"id":"B731","city":"Riga","units":7}]Checked answer{"city": "Riga", "units": 7}wrong_schema_or_value
{"city":"Lima","units":6}11 tokensCorrect
{"city":"Riga","units":7}11 tokensCorrect
{"city":"Riga","units":7}11 tokensCorrect
{"city":"Riga","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"M334","status":"pending"},{"id":"I579","status":"closed"},{"id":"B504","status":"pending"},{"id":"S746","status":"pending"},{"id":"Q462","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{
"ids": ["M334", "I579", "B504", "S746", "Q462"]
}38 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"I778","units":4,"price":17},{"id":"D646","units":8,"price":18},{"id":"Z666","units":4,"price":11},{"id":"R168","units":14,"price":11},{"id":"P472","units":9,"price":4}]Checked answer{"total": 446}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"V772","priority":3},{"id":"C372","priority":1},{"id":"Y274","priority":1},{"id":"K107","priority":5},{"id":"E916","priority":5}]Checked answer{"ids": ["E916", "K107"]}invalid_json_object
[{"id":"V772","priority":3},{"id":"C372","priority":1},{"id":"Y274","priority":1},{"id":"K107","priority":5},{"id":"E916","priority":5}]62 tokenswrong_schema_or_value
{"ids":["K107","E916"]}16 tokensCorrect
{"ids":["E916","K107"]}16 tokensCorrect
{"ids":["E916","K107"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to B817? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"V186","city":"Bern","units":0},{"id":"B817","city":"Kyoto","units":18},{"id":"G407","city":"Lima","units":11},{"id":"O934","city":"Perth","units":10},{"id":"X104","city":"Bern","units":18}]Checked answer{"city": "Kyoto", "units": 18}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Kyoto","units":18}13 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Kyoto","units":18}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"V336","status":"closed"},{"id":"R797","status":"ready"},{"id":"U701","status":"ready"},{"id":"W194","status":"closed"},{"id":"H451","status":"ready"}]Checked answer{"ids": ["H451", "R797", "U701"]}invalid_json_object
[{"id":"R797","status":"ready"},{"id":"U701","status":"ready"},{"id":"W194","status":"closed"},{"id":"H451","status":"ready"}]51 tokensinvalid_json_object
["H451","R797","U701"]17 tokens
Correct
{"ids":["H451","R797","U701"]}21 tokensinvalid_json_object
["ids":["H451","R797","V336"]]21 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"U280","units":10,"price":5},{"id":"C998","units":12,"price":8},{"id":"Q287","units":19,"price":7},{"id":"S419","units":1,"price":7},{"id":"K466","units":10,"price":13}]Checked answer{"total": 416}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":548}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"M122","priority":1},{"id":"L776","priority":3},{"id":"L212","priority":5},{"id":"C237","priority":4},{"id":"R159","priority":9}]Checked answer{"ids": ["R159", "L212"]}invalid_json_object
{"ids": ["M122", "L212", "C237", "R159", "L776", "L212", "L776", "L212", "L776", "L212", "L776", "L212", "L776", "L212", "L776", "L296 tokenswrong_schema_or_value
{"ids":["R159","C237"]}16 tokenswrong_schema_or_value
{"ids":["R159","C237"]}16 tokenswrong_schema_or_value
{"ids":["R159","C237"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=E851. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"C314","city":"Riga","units":3},{"id":"E851","city":"Perth","units":17},{"id":"C799","city":"Perth","units":13},{"id":"F516","city":"Riga","units":13},{"id":"M257","city":"Bern","units":4}]Checked answer{"city": "Perth", "units": 17}wrong_schema_or_value
{"city": "Riga", "units": 3}14 tokensCorrect
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"U218","status":"pending"},{"id":"S752","status":"pending"},{"id":"E755","status":"closed"},{"id":"P742","status":"ready"},{"id":"Y907","status":"pending"}]Checked answer{"ids": ["P742"]}wrong_schema_or_value
{"ids": ["U218", "S752", "E755", "P742", "Y907"]}35 tokensCorrect
{"ids":["P742"]}11 tokensCorrect
{"ids":["P742"]}11 tokensCorrect
{"ids":["P742"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"E705","units":14,"price":15},{"id":"Q829","units":4,"price":19},{"id":"H123","units":7,"price":19},{"id":"N675","units":18,"price":7},{"id":"E113","units":19,"price":12}]Checked answer{"total": 773}invalid_json_object
{
"total": 15 + 72 + 133 + 114 + 135 = 427
}35 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"N243","priority":7},{"id":"T512","priority":1},{"id":"I187","priority":3},{"id":"Z312","priority":6},{"id":"X332","priority":6}]Checked answer{"ids": ["N243", "X332"]}invalid_json_object
[{"id":"N243","priority":7},{"id":"T512","priority":1},{"id":"I187","priority":3},{"id":"Z312","priority":6},{"id":"X332","priority":6}]62 tokensCorrect
{"ids":["N243","X332"]}16 tokensCorrect
{"ids":["N243","X332"]}16 tokenswrong_schema_or_value
{"ids":["N243","Z312"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"X871","city":"Bern","units":15},{"id":"K341","city":"Suva","units":13},{"id":"B842","city":"Kyoto","units":16},{"id":"Z269","city":"Perth","units":9},{"id":"F471","city":"Bern","units":1}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Bern","units":15}]12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"G411","status":"closed"},{"id":"K648","status":"closed"},{"id":"O293","status":"pending"},{"id":"Q777","status":"ready"},{"id":"T179","status":"ready"}]Checked answer{"ids": ["Q777", "T179"]}wrong_schema_or_value
{"ids": ["O293", "Q777", "T179", "G411", "K648"]}35 tokensCorrect
{"ids":["Q777","T179"]}16 tokensCorrect
{"ids":["Q777","T179"]}16 tokensCorrect
{"ids":["Q777","T179"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"W347","units":2,"price":5},{"id":"Y441","units":9,"price":16},{"id":"Q195","units":19,"price":5},{"id":"B867","units":8,"price":13},{"id":"W615","units":8,"price":7}]Checked answer{"total": 409}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"C829","priority":4},{"id":"I723","priority":8},{"id":"F298","priority":9},{"id":"L400","priority":5},{"id":"X282","priority":2}]Checked answer{"ids": ["F298", "I723"]}wrong_schema_or_value
{"ids": ["I723", "F298", "L400", "C829"]}29 tokenswrong_schema_or_value
{"ids":["I723","F298"]}16 tokensCorrect
{"ids":["F298","I723"]}16 tokensCorrect
{"ids":["F298","I723"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z418 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"M127","city":"Riga","units":16},{"id":"Z418","city":"Lima","units":14},{"id":"Z986","city":"Accra","units":9},{"id":"U983","city":"Bern","units":16},{"id":"W495","city":"Riga","units":10}]Checked answer{"city": "Lima", "units": 14}wrong_schema_or_value
{"city":"Riga","units":16}12 tokensCorrect
{"city":"Lima","units":14}12 tokensCorrect
{"city":"Lima","units":14}12 tokensCorrect
{"city":"Lima","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"H710","status":"pending"},{"id":"Q416","status":"ready"},{"id":"P202","status":"pending"},{"id":"A315","status":"closed"},{"id":"B688","status":"pending"}]Checked answer{"ids": ["Q416"]}wrong_schema_or_value
{
"ids": ["A315", "B688", "H710", "Q416"]
}32 tokensCorrect
{"ids":["Q416"]}11 tokensCorrect
{"ids":["Q416"]}11 tokensCorrect
{"ids":["Q416"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"T854","units":17,"price":19},{"id":"S958","units":2,"price":8},{"id":"N243","units":8,"price":18},{"id":"Y942","units":12,"price":2},{"id":"Y468","units":0,"price":1}]Checked answer{"total": 507}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"C669","priority":5},{"id":"I236","priority":2},{"id":"W966","priority":9},{"id":"B275","priority":7},{"id":"A414","priority":4}]Checked answer{"ids": ["W966", "B275"]}invalid_json_object
[{"id":"C669","priority":5},{"id":"I236","priority":2},{"id":"W966","priority":9},{"id":"B275","priority":7},{"id":"A414","priority":4}]62 tokensCorrect
{"ids":["W966","B275"]}16 tokenswrong_schema_or_value
{"ids":["W966","C669"]}16 tokensCorrect
{"ids":["W966","B275"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to T721? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"M348","city":"Lima","units":16},{"id":"T721","city":"Accra","units":13},{"id":"W388","city":"Riga","units":6},{"id":"Z819","city":"Lima","units":18},{"id":"G934","city":"Perth","units":17}]Checked answer{"city": "Accra", "units": 13}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Accra","units":13}12 tokensCorrect
{"city":"Accra","units":13}12 tokensCorrect
{"city":"Accra","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"T312","status":"closed"},{"id":"U500","status":"closed"},{"id":"G282","status":"ready"},{"id":"O348","status":"closed"},{"id":"K385","status":"ready"}]Checked answer{"ids": ["G282", "K385"]}invalid_json_object
[{"id":"G282"},{"id":"K385"},{"id":"O348"},{"id":"T312"},{"id":"U500"}]43 tokensinvalid_json_object
["K385","G282"]12 tokens
wrong_schema_or_value
{"ids":["K385","G282"]}16 tokensinvalid_json_object
["ids":["G282","K385"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"P974","units":19,"price":1},{"id":"Q253","units":12,"price":17},{"id":"H887","units":3,"price":15},{"id":"W935","units":19,"price":16},{"id":"J627","units":4,"price":17}]Checked answer{"total": 640}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"T783","priority":6},{"id":"V728","priority":7},{"id":"P420","priority":2},{"id":"V918","priority":8},{"id":"S501","priority":5}]Checked answer{"ids": ["V918", "V728"]}wrong_schema_or_value
{"ids": ["V728", "V918", "P420", "T783", "S501"]}35 tokenswrong_schema_or_value
{"ids":["V918","T783"]}16 tokenswrong_schema_or_value
{"ids":["V728","T783"]}16 tokenswrong_schema_or_value
{"ids":["V918","T783"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=C185. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"R549","city":"Riga","units":2},{"id":"L675","city":"Oslo","units":17},{"id":"V632","city":"Kyoto","units":3},{"id":"D559","city":"Oslo","units":7},{"id":"C185","city":"Oslo","units":0}]Checked answer{"city": "Oslo", "units": 0}wrong_schema_or_value
{"city":"Riga","units":2}11 tokensCorrect
{"city":"Oslo","units":0}11 tokenswrong_schema_or_value
{"city":null,"units":0}10 tokensCorrect
{"city":"Oslo","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"W732","status":"ready"},{"id":"F215","status":"closed"},{"id":"A130","status":"pending"},{"id":"W282","status":"pending"},{"id":"J985","status":"pending"}]Checked answer{"ids": ["W732"]}wrong_schema_or_value
{"ids": ["A130", "F215", "J985", "W282", "W732"]}35 tokensCorrect
{"ids":["W732"]}11 tokensCorrect
{"ids":["W732"]}11 tokensCorrect
{"ids":["W732"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"D408","units":8,"price":5},{"id":"Z220","units":5,"price":15},{"id":"D617","units":8,"price":9},{"id":"Y257","units":14,"price":2},{"id":"G975","units":11,"price":10}]Checked answer{"total": 325}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"J821","priority":8},{"id":"J165","priority":7},{"id":"Q357","priority":5},{"id":"V944","priority":9},{"id":"D364","priority":4}]Checked answer{"ids": ["V944", "J821"]}invalid_json_object
[{"id":"J821"},{"id":"J165"},{"id":"Q357"},{"id":"V944"},{"id":"D364"}]43 tokensCorrect
{"ids":["V944","J821"]}16 tokenswrong_schema_or_value
{"ids":["J821","V944"]}16 tokenswrong_schema_or_value
{"ids":["J821","V944"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record J411. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"J411","city":"Riga","units":16},{"id":"M699","city":"Accra","units":13},{"id":"I998","city":"Kyoto","units":16},{"id":"G698","city":"Bern","units":3},{"id":"Y700","city":"Suva","units":3}]Checked answer{"city": "Riga", "units": 16}Correct
{
"city": "Riga",
"units": 16
}19 tokensCorrect
{"city":"Riga","units":16}12 tokensCorrect
{"city":"Riga","units":16}12 tokensCorrect
{"city":"Riga","units":16}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"G547","status":"pending"},{"id":"M409","status":"ready"},{"id":"B935","status":"pending"},{"id":"H405","status":"closed"},{"id":"H304","status":"pending"}]Checked answer{"ids": ["M409"]}wrong_schema_or_value
{"ids": ["B935", "G547", "M409", "H304", "H405"]}35 tokensCorrect
{"ids":["M409"]}11 tokensCorrect
{"ids":["M409"]}11 tokensCorrect
{"ids":["M409"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"F553","units":15,"price":11},{"id":"R844","units":8,"price":10},{"id":"O199","units":9,"price":15},{"id":"P188","units":17,"price":11},{"id":"V592","units":0,"price":16}]Checked answer{"total": 567}invalid_json_object
{"total": 11*15 + 10*8 + 15*9 + 11*17 + 16*0 = 165}41 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"W314","priority":7},{"id":"P612","priority":3},{"id":"M834","priority":2},{"id":"Q280","priority":3},{"id":"D889","priority":8}]Checked answer{"ids": ["D889", "W314"]}wrong_schema_or_value
{"ids": ["P612", "W314", "D889", "M834", "Q280"]}35 tokensCorrect
{"ids":["D889","W314"]}16 tokensCorrect
{"ids":["D889","W314"]}16 tokensCorrect
{"ids":["D889","W314"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"U255","city":"Kyoto","units":19},{"id":"P832","city":"Oslo","units":7},{"id":"R298","city":"Suva","units":1},{"id":"W902","city":"Suva","units":6},{"id":"Q640","city":"Accra","units":0}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Kyoto","units":19}13 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"A364","status":"ready"},{"id":"D388","status":"ready"},{"id":"N316","status":"closed"},{"id":"E874","status":"pending"},{"id":"D422","status":"pending"}]Checked answer{"ids": ["A364", "D388"]}wrong_schema_or_value
{
"ids": ["A364", "D388", "N316", "E874", "D422"]
}38 tokensCorrect
{"ids":["A364","D388"]}16 tokensCorrect
{"ids":["A364","D388"]}16 tokensCorrect
{"ids":["A364","D388"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"R897","units":1,"price":9},{"id":"V567","units":15,"price":10},{"id":"B117","units":8,"price":9},{"id":"W196","units":3,"price":2},{"id":"R980","units":11,"price":1}]Checked answer{"total": 248}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Y350","priority":3},{"id":"T798","priority":9},{"id":"K874","priority":5},{"id":"L856","priority":6},{"id":"V730","priority":7}]Checked answer{"ids": ["T798", "V730"]}invalid_json_object
[{"id":"Y350","priority":3},{"id":"V730","priority":7}]26 tokensCorrect
{"ids":["T798","V730"]}16 tokensCorrect
{"ids":["T798","V730"]}16 tokenswrong_schema_or_value
{"ids":["T798","K874"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to K850? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"K850","city":"Kyoto","units":2},{"id":"J191","city":"Oslo","units":3},{"id":"S905","city":"Kyoto","units":7},{"id":"G431","city":"Bern","units":16},{"id":"W275","city":"Oslo","units":14}]Checked answer{"city": "Kyoto", "units": 2}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Kyoto","units":2}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"M330","status":"pending"},{"id":"T388","status":"closed"},{"id":"F764","status":"pending"},{"id":"J614","status":"ready"},{"id":"G113","status":"closed"}]Checked answer{"ids": ["J614"]}invalid_json_object
[{"id":"J614","status":"ready"},{"id":"G113","status":"closed"},{"id":"M330","status":"pending"},{"id":"T388","status":"closed"},{"id":"F764","status":"pending"}]63 tokensCorrect
{"ids":["J614"]}11 tokensCorrect
{"ids":["J614"]}11 tokensinvalid_json_object
["ids":["J614"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"U948","units":4,"price":7},{"id":"P799","units":4,"price":7},{"id":"V292","units":2,"price":10},{"id":"T401","units":14,"price":13},{"id":"E840","units":4,"price":16}]Checked answer{"total": 322}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"R801","priority":4},{"id":"K978","priority":6},{"id":"G245","priority":7},{"id":"J766","priority":9},{"id":"T811","priority":6}]Checked answer{"ids": ["J766", "G245"]}wrong_schema_or_value
{"ids": ["R801", "K978", "G245", "J766", "T811"]}35 tokensCorrect
{"ids":["J766","G245"]}16 tokensCorrect
{"ids":["J766","G245"]}16 tokensCorrect
{"ids":["J766","G245"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=R859. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"V186","city":"Kyoto","units":8},{"id":"X837","city":"Oslo","units":10},{"id":"Z922","city":"Accra","units":1},{"id":"R859","city":"Bern","units":5},{"id":"R423","city":"Suva","units":16}]Checked answer{"city": "Bern", "units": 5}Correct
{"city":"Bern","units":5}10 tokensCorrect
{"city":"Bern","units":5}10 tokensCorrect
{"city":"Bern","units":5}10 tokensCorrect
{"city":"Bern","units":5}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Q542","status":"closed"},{"id":"P244","status":"pending"},{"id":"C582","status":"pending"},{"id":"O283","status":"closed"},{"id":"J765","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["C582", "J765", "P244", "O283", "Q542"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"O517","units":10,"price":9},{"id":"D896","units":2,"price":3},{"id":"V778","units":0,"price":14},{"id":"J149","units":4,"price":16},{"id":"Q856","units":16,"price":12}]Checked answer{"total": 352}invalid_json_object
{
"total": 10 * 9 + 2 * 3 + 0 * 14 + 4 * 16 + 16 * 12
}42 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"K820","priority":9},{"id":"N659","priority":4},{"id":"L981","priority":9},{"id":"E808","priority":3},{"id":"A179","priority":4}]Checked answer{"ids": ["K820", "L981"]}invalid_json_object
[{"id":"K820","priority":9},{"id":"N659","priority":4},{"id":"L981","priority":9},{"id":"E808","priority":3},{"id":"A179","priority":4}]62 tokenswrong_schema_or_value
{"ids":["L981","K820"]}16 tokenswrong_schema_or_value
{"ids":["L981","K820"]}16 tokenswrong_schema_or_value
{"ids":["L981","K820"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Y129. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Y836","city":"Accra","units":12},{"id":"X765","city":"Accra","units":19},{"id":"B191","city":"Perth","units":14},{"id":"S184","city":"Accra","units":13},{"id":"Y129","city":"Accra","units":7}]Checked answer{"city": "Accra", "units": 7}invalid_json_object
[{"city":"Accra","units":12}]13 tokensCorrect
{"city":"Accra","units":7}11 tokensCorrect
{"city":"Accra","units":7}11 tokensCorrect
{"city":"Accra","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"X108","status":"ready"},{"id":"V419","status":"closed"},{"id":"V112","status":"ready"},{"id":"U180","status":"pending"},{"id":"D990","status":"pending"}]Checked answer{"ids": ["V112", "X108"]}wrong_schema_or_value
{"ids": ["X108", "V112", "U180", "D990"]}29 tokenswrong_schema_or_value
{"ids":["X108","V112"]}16 tokenswrong_schema_or_value
{"ids":["X108","V112"]}16 tokenswrong_schema_or_value
{"ids":["X108","V112"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"G170","units":17,"price":13},{"id":"W206","units":19,"price":13},{"id":"Y795","units":11,"price":15},{"id":"U350","units":10,"price":16},{"id":"U792","units":13,"price":2}]Checked answer{"total": 819}wrong_schema_or_value
{"total": 130}9 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"R222","priority":1},{"id":"Y946","priority":3},{"id":"H716","priority":5},{"id":"P546","priority":6},{"id":"B745","priority":9}]Checked answer{"ids": ["B745", "P546"]}wrong_schema_or_value
{"ids": ["R222", "Y946", "H716", "P546", "B745"]}35 tokensCorrect
{"ids":["B745","P546"]}16 tokenswrong_schema_or_value
{"ids":["B745","H716"]}16 tokenswrong_schema_or_value
{"ids":["B745","H716"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id A660 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"F697","city":"Perth","units":4},{"id":"E917","city":"Suva","units":19},{"id":"I252","city":"Kyoto","units":11},{"id":"D948","city":"Bern","units":15},{"id":"A660","city":"Riga","units":18}]Checked answer{"city": "Riga", "units": 18}Correct
{"city":"Riga","units":18}12 tokensCorrect
{"city":"Riga","units":18}12 tokensCorrect
{"city":"Riga","units":18}12 tokensCorrect
{"city":"Riga","units":18}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"B458","status":"closed"},{"id":"U662","status":"pending"},{"id":"M311","status":"ready"},{"id":"E912","status":"ready"},{"id":"T753","status":"ready"}]Checked answer{"ids": ["E912", "M311", "T753"]}wrong_schema_or_value
{
"ids": ["B458", "M311", "E912", "T753"]
}32 tokensCorrect
{"ids":["E912","M311","T753"]}21 tokensCorrect
{"ids":["E912","M311","T753"]}21 tokensCorrect
{"ids":["E912","M311","T753"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"M754","units":6,"price":17},{"id":"I597","units":6,"price":15},{"id":"A403","units":7,"price":5},{"id":"U910","units":12,"price":5},{"id":"O840","units":3,"price":16}]Checked answer{"total": 335}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":278}8 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"G740","priority":3},{"id":"P538","priority":9},{"id":"O237","priority":9},{"id":"F189","priority":4},{"id":"G539","priority":3}]Checked answer{"ids": ["O237", "P538"]}invalid_json_object
[{"id":"G740","priority":3},{"id":"P538","priority":9},{"id":"O237","priority":9},{"id":"F189","priority":4},{"id":"G539","priority":3}]62 tokensCorrect
{"ids":["O237","P538"]}16 tokensCorrect
{"ids":["O237","P538"]}16 tokensCorrect
{"ids":["O237","P538"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"E774","city":"Suva","units":14},{"id":"E986","city":"Bern","units":4},{"id":"P886","city":"Kyoto","units":16},{"id":"K208","city":"Accra","units":19},{"id":"P762","city":"Perth","units":17}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"E674","status":"closed"},{"id":"F246","status":"pending"},{"id":"T956","status":"ready"},{"id":"L641","status":"pending"},{"id":"T476","status":"closed"}]Checked answer{"ids": ["T956"]}invalid_json_object
[{"id":"T476","status":"closed"},{"id":"T956","status":"ready"},{"id":"F246","status":"pending"},{"id":"E674","status":"closed"}]51 tokensCorrect
{"ids":["T956"]}11 tokensCorrect
{"ids":["T956"]}11 tokensinvalid_json_object
["ids":["T956"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Y997","units":11,"price":7},{"id":"E274","units":18,"price":9},{"id":"N337","units":8,"price":10},{"id":"K891","units":11,"price":1},{"id":"N305","units":0,"price":13}]Checked answer{"total": 330}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"J661","priority":8},{"id":"Q599","priority":5},{"id":"T927","priority":7},{"id":"V245","priority":2},{"id":"H524","priority":9}]Checked answer{"ids": ["H524", "J661"]}invalid_json_object
{"ids": ["J661", "V245", "H524", "T927", "Q599", "V245", "H524", "T927", "Q599", "V245", "H524", "T927", "Q599", "V245", "H524", "T996 tokensCorrect
{"ids":["H524","J661"]}16 tokensCorrect
{"ids":["H524","J661"]}16 tokensCorrect
{"ids":["H524","J661"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=B434. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"Y961","city":"Riga","units":0},{"id":"K255","city":"Riga","units":14},{"id":"L868","city":"Riga","units":3},{"id":"E747","city":"Accra","units":11},{"id":"B434","city":"Kyoto","units":4}]Checked answer{"city": "Kyoto", "units": 4}wrong_schema_or_value
{"city": "Riga", "units": 14}15 tokensCorrect
{"city":"Kyoto","units":4}12 tokensCorrect
{"city":"Kyoto","units":4}12 tokensCorrect
{"city":"Kyoto","units":4}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"A902","status":"closed"},{"id":"M630","status":"pending"},{"id":"W637","status":"closed"},{"id":"T972","status":"ready"},{"id":"Z326","status":"ready"}]Checked answer{"ids": ["T972", "Z326"]}wrong_schema_or_value
{"ids": ["A902", "M630", "W637", "T972", "Z326"]}35 tokensCorrect
{"ids":["T972","Z326"]}16 tokensCorrect
{"ids":["T972","Z326"]}16 tokenswrong_schema_or_value
{"ids":["Z326","T972"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"H516","units":10,"price":6},{"id":"D762","units":1,"price":4},{"id":"U351","units":12,"price":1},{"id":"X917","units":12,"price":14},{"id":"V868","units":8,"price":15}]Checked answer{"total": 364}invalid_json_object
{
"total": 10 * 6 + 1 * 4 + 12 * 1 + 12 * 14 + 8 * 15
}42 tokenswrong_schema_or_value
{"total":388}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"G746","priority":9},{"id":"P121","priority":9},{"id":"B978","priority":6},{"id":"Q508","priority":8},{"id":"P706","priority":5}]Checked answer{"ids": ["G746", "P121"]}invalid_json_object
[{"id":"P706","priority":5},{"id":"P121","priority":9},{"id":"B978","priority":6},{"id":"G746","priority":9},{"id":"Q508","priority":8}]62 tokensCorrect
{"ids":["G746","P121"]}16 tokensCorrect
{"ids":["G746","P121"]}16 tokensCorrect
{"ids":["G746","P121"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record O675. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"O675","city":"Oslo","units":9},{"id":"P458","city":"Oslo","units":5},{"id":"X647","city":"Oslo","units":9},{"id":"K952","city":"Kyoto","units":16},{"id":"N481","city":"Lima","units":12}]Checked answer{"city": "Oslo", "units": 9}Correct
{
"city": "Oslo",
"units": 9
}18 tokensCorrect
{"city":"Oslo","units":9}11 tokensCorrect
{"city":"Oslo","units":9}11 tokensCorrect
{"city":"Oslo","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"P606","status":"closed"},{"id":"Q486","status":"ready"},{"id":"J629","status":"pending"},{"id":"O349","status":"pending"},{"id":"I215","status":"ready"}]Checked answer{"ids": ["I215", "Q486"]}wrong_schema_or_value
{"ids": ["J629", "O349", "P606", "Q486", "I215"]}35 tokensCorrect
{"ids":["I215","Q486"]}16 tokensCorrect
{"ids":["I215","Q486"]}16 tokensCorrect
{"ids":["I215","Q486"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"X585","units":4,"price":15},{"id":"U277","units":19,"price":18},{"id":"K451","units":7,"price":19},{"id":"O361","units":11,"price":9},{"id":"K726","units":7,"price":17}]Checked answer{"total": 753}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"X598","priority":2},{"id":"Z990","priority":4},{"id":"P918","priority":3},{"id":"S125","priority":2},{"id":"Y123","priority":2}]Checked answer{"ids": ["Z990", "P918"]}wrong_schema_or_value
{"ids": ["X598", "Y123", "S125", "Z990"]}29 tokenswrong_schema_or_value
{"ids":["P918","Z990"]}16 tokensCorrect
{"ids":["Z990","P918"]}16 tokensCorrect
{"ids":["Z990","P918"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id O723 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"K312","city":"Bern","units":14},{"id":"Y150","city":"Oslo","units":1},{"id":"M373","city":"Lima","units":12},{"id":"O723","city":"Bern","units":2},{"id":"O832","city":"Lima","units":4}]Checked answer{"city": "Bern", "units": 2}Correct
{"city":"Bern","units":2}10 tokensCorrect
{"city":"Bern","units":2}10 tokensCorrect
{"city":"Bern","units":2}10 tokensCorrect
{"city":"Bern","units":2}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"G472","status":"closed"},{"id":"E603","status":"ready"},{"id":"A439","status":"closed"},{"id":"Y251","status":"pending"},{"id":"Z144","status":"ready"}]Checked answer{"ids": ["E603", "Z144"]}wrong_schema_or_value
{
"ids": ["A439", "E603", "G472", "Y251"]
}32 tokensCorrect
{"ids":["E603","Z144"]}16 tokensCorrect
{"ids":["E603","Z144"]}16 tokensCorrect
{"ids":["E603","Z144"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"D938","units":6,"price":13},{"id":"Q422","units":19,"price":19},{"id":"M196","units":18,"price":1},{"id":"V342","units":15,"price":6},{"id":"N312","units":14,"price":17}]Checked answer{"total": 785}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"G297","priority":8},{"id":"V199","priority":3},{"id":"E560","priority":2},{"id":"R783","priority":2},{"id":"O329","priority":2}]Checked answer{"ids": ["G297", "V199"]}invalid_json_object
[{"id":"G297","priority":8},{"id":"V199","priority":3},{"id":"E560","priority":2},{"id":"R783","priority":2},{"id":"O329","priority":2}]62 tokenswrong_schema_or_value
{"ids":["G297","E560"]}16 tokenswrong_schema_or_value
{"ids":["G297","E560"]}16 tokenswrong_schema_or_value
{"ids":["G297","E560"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to F977? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"T485","city":"Perth","units":18},{"id":"J646","city":"Perth","units":2},{"id":"L357","city":"Lima","units":7},{"id":"A363","city":"Riga","units":7},{"id":"F977","city":"Kyoto","units":17}]Checked answer{"city": "Kyoto", "units": 17}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Kyoto","units":17}13 tokensCorrect
{"city":"Kyoto","units":17}13 tokensCorrect
{"city":"Kyoto","units":17}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"Q269","status":"pending"},{"id":"S805","status":"pending"},{"id":"A911","status":"pending"},{"id":"T568","status":"ready"},{"id":"I395","status":"closed"}]Checked answer{"ids": ["T568"]}invalid_json_object
[{"id":"T568","status":"ready"}]15 tokensCorrect
{"ids":["T568"]}11 tokensCorrect
{"ids":["T568"]}11 tokensinvalid_json_object
["ids":["T568"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"M505","units":15,"price":6},{"id":"J623","units":12,"price":12},{"id":"K831","units":16,"price":6},{"id":"E456","units":6,"price":9},{"id":"U294","units":14,"price":8}]Checked answer{"total": 496}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"L280","priority":1},{"id":"P533","priority":7},{"id":"B284","priority":8},{"id":"F368","priority":6},{"id":"N502","priority":5}]Checked answer{"ids": ["B284", "P533"]}wrong_schema_or_value
{"ids": ["L280", "P533", "B284", "F368", "N502"]}35 tokensCorrect
{"ids":["B284","P533"]}16 tokensCorrect
{"ids":["B284","P533"]}16 tokensCorrect
{"ids":["B284","P533"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"Z703","city":"Accra","units":15},{"id":"B387","city":"Riga","units":8},{"id":"M704","city":"Bern","units":1},{"id":"D492","city":"Accra","units":9},{"id":"H133","city":"Oslo","units":11}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Accra","units":15}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Y794","status":"ready"},{"id":"R346","status":"closed"},{"id":"M694","status":"closed"},{"id":"E287","status":"pending"},{"id":"O444","status":"ready"}]Checked answer{"ids": ["O444", "Y794"]}wrong_schema_or_value
{"ids": ["O444", "Y794", "R346", "M694", "E287"]}35 tokenswrong_schema_or_value
{"ids":["E287","Y794","O444"]}21 tokenswrong_schema_or_value
{"ids":["E287","O444","Y794"]}21 tokensCorrect
{"ids":["O444","Y794"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"Y807","units":5,"price":17},{"id":"M994","units":2,"price":17},{"id":"B224","units":17,"price":18},{"id":"C629","units":18,"price":5},{"id":"L202","units":6,"price":1}]Checked answer{"total": 521}invalid_json_object
{
"total": 17 + 34 + 30 + 95 + 108
}28 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"B486","priority":4},{"id":"Y489","priority":5},{"id":"S834","priority":1},{"id":"E362","priority":6},{"id":"S397","priority":3}]Checked answer{"ids": ["E362", "Y489"]}invalid_json_object
[{"id":"S397","priority":3},{"id":"S834","priority":1},{"id":"Y489","priority":5},{"id":"B486","priority":4},{"id":"E362","priority":6}]62 tokensCorrect
{"ids":["E362","Y489"]}16 tokensCorrect
{"ids":["E362","Y489"]}16 tokensCorrect
{"ids":["E362","Y489"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Y963. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"W495","city":"Suva","units":8},{"id":"Y963","city":"Riga","units":13},{"id":"S208","city":"Suva","units":7},{"id":"A698","city":"Lima","units":2},{"id":"R398","city":"Kyoto","units":18}]Checked answer{"city": "Riga", "units": 13}Correct
{
"city": "Riga",
"units": 13
}19 tokensCorrect
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"A433","status":"closed"},{"id":"S345","status":"ready"},{"id":"Q532","status":"pending"},{"id":"M350","status":"pending"},{"id":"E536","status":"pending"}]Checked answer{"ids": ["S345"]}wrong_schema_or_value
{"ids": ["S345", "M350", "E536", "A433", "Q532"]}35 tokensCorrect
{"ids":["S345"]}11 tokensCorrect
{"ids":["S345"]}11 tokensCorrect
{"ids":["S345"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"S721","units":10,"price":1},{"id":"R676","units":11,"price":1},{"id":"I601","units":14,"price":1},{"id":"Y566","units":12,"price":19},{"id":"Y105","units":4,"price":1}]Checked answer{"total": 267}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":468}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"J421","priority":6},{"id":"R989","priority":5},{"id":"O640","priority":8},{"id":"F552","priority":9},{"id":"K905","priority":2}]Checked answer{"ids": ["F552", "O640"]}wrong_schema_or_value
{"ids": ["R989", "J421", "O640", "F552", "K905"]}35 tokensCorrect
{"ids":["F552","O640"]}16 tokensCorrect
{"ids":["F552","O640"]}16 tokenswrong_schema_or_value
{"ids":["F552","J421"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Y665 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"O304","city":"Accra","units":3},{"id":"D164","city":"Bern","units":3},{"id":"I811","city":"Lima","units":4},{"id":"Y665","city":"Oslo","units":0},{"id":"L402","city":"Perth","units":3}]Checked answer{"city": "Oslo", "units": 0}Correct
{"city":"Oslo","units":0}11 tokensCorrect
{"city":"Oslo","units":0}11 tokensCorrect
{"city":"Oslo","units":0}11 tokensCorrect
{"city":"Oslo","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"P280","status":"ready"},{"id":"J438","status":"ready"},{"id":"K781","status":"ready"},{"id":"I804","status":"closed"},{"id":"L317","status":"pending"}]Checked answer{"ids": ["J438", "K781", "P280"]}wrong_schema_or_value
{
"ids": ["P280", "J438", "K781"]
}26 tokensCorrect
{"ids":["J438","K781","P280"]}21 tokensCorrect
{"ids":["J438","K781","P280"]}21 tokensinvalid_json_object
{"ids":["J438","K781","P280"]]}22 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"H940","units":5,"price":18},{"id":"P420","units":3,"price":15},{"id":"E759","units":11,"price":12},{"id":"V715","units":16,"price":15},{"id":"U710","units":19,"price":12}]Checked answer{"total": 735}wrong_schema_or_value
{"total": 108}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"T961","priority":9},{"id":"I566","priority":4},{"id":"N476","priority":6},{"id":"N584","priority":9},{"id":"T694","priority":2}]Checked answer{"ids": ["N584", "T961"]}invalid_json_object
[{"id":"T961","priority":9},{"id":"N584","priority":9},{"id":"N476","priority":6},{"id":"I566","priority":4},{"id":"T694","priority":2}]62 tokensCorrect
{"ids":["N584","T961"]}16 tokensCorrect
{"ids":["N584","T961"]}16 tokenswrong_schema_or_value
{"ids":["N476","T961"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to M255? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"R102","city":"Accra","units":18},{"id":"H709","city":"Accra","units":14},{"id":"I483","city":"Lima","units":19},{"id":"A392","city":"Riga","units":18},{"id":"M255","city":"Perth","units":0}]Checked answer{"city": "Perth", "units": 0}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Perth","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"D721","status":"pending"},{"id":"M690","status":"pending"},{"id":"Q960","status":"closed"},{"id":"G575","status":"ready"},{"id":"F989","status":"closed"}]Checked answer{"ids": ["G575"]}invalid_json_object
[{"id":"G575"},{"id":"F989"},{"id":"M690"},{"id":"Q960"},{"id":"D721"}]43 tokensCorrect
{"ids":["G575"]}11 tokensCorrect
{"ids":["G575"]}11 tokensinvalid_json_object
["ids":["G575"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"H523","units":4,"price":17},{"id":"N492","units":7,"price":19},{"id":"G584","units":3,"price":19},{"id":"C398","units":4,"price":8},{"id":"Q632","units":4,"price":14}]Checked answer{"total": 346}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":278}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"K875","priority":3},{"id":"E783","priority":6},{"id":"Z206","priority":5},{"id":"H162","priority":3},{"id":"S969","priority":1}]Checked answer{"ids": ["E783", "Z206"]}wrong_schema_or_value
{"ids": ["K875", "E783", "Z206", "H162", "S969"]}35 tokenswrong_schema_or_value
{"ids":["E783","K875"]}16 tokenswrong_schema_or_value
{"ids":["E783","K875"]}16 tokenswrong_schema_or_value
{"ids":["E783","K875"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=T348. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"C506","city":"Accra","units":10},{"id":"U750","city":"Lima","units":5},{"id":"T348","city":"Lima","units":18},{"id":"Y591","city":"Suva","units":17},{"id":"B547","city":"Accra","units":8}]Checked answer{"city": "Lima", "units": 18}Correct
{"city": "Lima", "units": 18}15 tokensCorrect
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"C156","status":"closed"},{"id":"H363","status":"closed"},{"id":"I375","status":"closed"},{"id":"U987","status":"pending"},{"id":"N487","status":"ready"}]Checked answer{"ids": ["N487"]}wrong_schema_or_value
{"ids": ["I375", "N487", "U987", "C156", "H363"]}35 tokensCorrect
{"ids":["N487"]}11 tokensCorrect
{"ids":["N487"]}11 tokensCorrect
{"ids":["N487"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"K721","units":1,"price":3},{"id":"B477","units":5,"price":14},{"id":"O833","units":5,"price":12},{"id":"W317","units":7,"price":16},{"id":"Y787","units":19,"price":13}]Checked answer{"total": 492}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"W535","priority":3},{"id":"E898","priority":9},{"id":"Z680","priority":6},{"id":"Y596","priority":9},{"id":"M938","priority":2}]Checked answer{"ids": ["E898", "Y596"]}invalid_json_object
[{"id":"W535","priority":3},{"id":"Y596","priority":9},{"id":"M938","priority":2}]38 tokensCorrect
{"ids":["E898","Y596"]}16 tokensCorrect
{"ids":["E898","Y596"]}16 tokensCorrect
{"ids":["E898","Y596"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"W340","city":"Suva","units":7},{"id":"M361","city":"Riga","units":6},{"id":"I752","city":"Accra","units":16},{"id":"M331","city":"Kyoto","units":5},{"id":"H364","city":"Kyoto","units":15}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Suva","units":7},{"city":"Riga","units":6},{"city":"Accra","units":16},{"city":"Kyoto","units":5},{"city":"Kyoto","units":15}]56 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Q700","status":"pending"},{"id":"I771","status":"closed"},{"id":"V949","status":"pending"},{"id":"U862","status":"closed"},{"id":"S575","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["U862", "S575", "V949", "I771", "Q700"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Q169","units":13,"price":15},{"id":"W455","units":17,"price":9},{"id":"X370","units":10,"price":12},{"id":"G597","units":18,"price":14},{"id":"J111","units":15,"price":1}]Checked answer{"total": 735}invalid_json_object
{"total": 13*15 + 17*9 + 10*12 + 18*14 + 15*1}37 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Z843","priority":6},{"id":"B686","priority":6},{"id":"X715","priority":9},{"id":"P657","priority":1},{"id":"P567","priority":3}]Checked answer{"ids": ["X715", "B686"]}wrong_schema_or_value
{"ids": ["X715", "P657", "P567", "B686", "Z843"]}35 tokenswrong_schema_or_value
{"ids":["X715","Z843"]}16 tokenswrong_schema_or_value
{"ids":["X715","Z843"]}16 tokensCorrect
{"ids":["X715","B686"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id O189 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"Z162","city":"Perth","units":1},{"id":"M397","city":"Lima","units":15},{"id":"J273","city":"Oslo","units":15},{"id":"S575","city":"Lima","units":3},{"id":"O189","city":"Kyoto","units":0}]Checked answer{"city": "Kyoto", "units": 0}wrong_schema_or_value
{"city":"Perth","units":1}11 tokensCorrect
{"city":"Kyoto","units":0}12 tokensCorrect
{"city":"Kyoto","units":0}12 tokensCorrect
{"city":"Kyoto","units":0}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"B125","status":"pending"},{"id":"V162","status":"ready"},{"id":"D704","status":"ready"},{"id":"R422","status":"ready"},{"id":"M173","status":"ready"}]Checked answer{"ids": ["D704", "M173", "R422", "V162"]}wrong_schema_or_value
{
"ids": ["B125", "D704", "M173", "R422"]
}32 tokenswrong_schema_or_value
{"ids":["D704","M173","R422"]}21 tokensCorrect
{"ids":["D704","M173","R422","V162"]}26 tokenswrong_schema_or_value
{"ids":["D704","M173","V162"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"K850","units":7,"price":6},{"id":"Z482","units":10,"price":16},{"id":"N867","units":2,"price":9},{"id":"P327","units":12,"price":14},{"id":"B974","units":6,"price":2}]Checked answer{"total": 400}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"H115","priority":7},{"id":"D458","priority":8},{"id":"S349","priority":3},{"id":"I532","priority":6},{"id":"F475","priority":6}]Checked answer{"ids": ["D458", "H115"]}invalid_json_object
[{"id":"H115","priority":7},{"id":"D458","priority":8},{"id":"S349","priority":3},{"id":"I532","priority":6},{"id":"F475","priority":6}]62 tokensCorrect
{"ids":["D458","H115"]}16 tokensCorrect
{"ids":["D458","H115"]}16 tokensCorrect
{"ids":["D458","H115"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Y839? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"W719","city":"Bern","units":6},{"id":"X629","city":"Lima","units":16},{"id":"Y839","city":"Suva","units":10},{"id":"N495","city":"Kyoto","units":0},{"id":"U511","city":"Suva","units":0}]Checked answer{"city": "Suva", "units": 10}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Suva","units":10}12 tokensCorrect
{"city":"Suva","units":10}12 tokensCorrect
{"city":"Suva","units":10}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"R588","status":"closed"},{"id":"W378","status":"ready"},{"id":"X176","status":"ready"},{"id":"M154","status":"pending"},{"id":"M190","status":"pending"}]Checked answer{"ids": ["W378", "X176"]}invalid_json_object
[{"id":"W378","status":"ready"},{"id":"X176","status":"ready"},{"id":"M154","status":"pending"},{"id":"M190","status":"pending"}]51 tokenswrong_schema_or_value
{"ids":["X176","W378"]}16 tokensCorrect
{"ids":["W378","X176"]}16 tokensinvalid_json_object
["ids":["W378","X176"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"R283","units":16,"price":9},{"id":"Q870","units":13,"price":11},{"id":"U210","units":10,"price":19},{"id":"A640","units":19,"price":16},{"id":"N399","units":14,"price":13}]Checked answer{"total": 963}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"X339","priority":7},{"id":"E781","priority":5},{"id":"V146","priority":3},{"id":"W826","priority":1},{"id":"X481","priority":3}]Checked answer{"ids": ["X339", "E781"]}wrong_schema_or_value
{"ids": ["X339", "V146", "W826", "X481", "E781"]}35 tokensCorrect
{"ids":["X339","E781"]}16 tokensCorrect
{"ids":["X339","E781"]}16 tokensCorrect
{"ids":["X339","E781"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=O242. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"D738","city":"Suva","units":3},{"id":"M362","city":"Kyoto","units":11},{"id":"O813","city":"Lima","units":8},{"id":"L165","city":"Riga","units":3},{"id":"O242","city":"Perth","units":13}]Checked answer{"city": "Perth", "units": 13}wrong_schema_or_value
{"city":"Suva","units":3}11 tokensCorrect
{"city":"Perth","units":13}12 tokensCorrect
{"city":"Perth","units":13}12 tokensCorrect
{"city":"Perth","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"T874","status":"closed"},{"id":"S878","status":"closed"},{"id":"A302","status":"pending"},{"id":"S367","status":"closed"},{"id":"I760","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["A302", "I760", "S367", "S878", "T874"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"S617","units":7,"price":4},{"id":"P972","units":14,"price":11},{"id":"W575","units":14,"price":16},{"id":"P708","units":5,"price":17},{"id":"V598","units":18,"price":19}]Checked answer{"total": 833}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"P802","priority":6},{"id":"E835","priority":3},{"id":"K659","priority":1},{"id":"O437","priority":5},{"id":"D137","priority":7}]Checked answer{"ids": ["D137", "P802"]}invalid_json_object
[{"id":"P802","priority":6},{"id":"O437","priority":5},{"id":"D137","priority":7},{"id":"K659","priority":1},{"id":"E835","priority":3}]62 tokensCorrect
{"ids":["D137","P802"]}16 tokensCorrect
{"ids":["D137","P802"]}16 tokenswrong_schema_or_value
{"ids":["D137","E835"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record M268. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"T747","city":"Accra","units":9},{"id":"E161","city":"Accra","units":7},{"id":"M268","city":"Riga","units":14},{"id":"W840","city":"Bern","units":6},{"id":"S872","city":"Lima","units":5}]Checked answer{"city": "Riga", "units": 14}invalid_json_object
[{"city":"Riga","units":14}]13 tokensCorrect
{"city":"Riga","units":14}12 tokensCorrect
{"city":"Riga","units":14}12 tokensCorrect
{"city":"Riga","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"A637","status":"ready"},{"id":"J191","status":"ready"},{"id":"E700","status":"closed"},{"id":"H824","status":"pending"},{"id":"C922","status":"closed"}]Checked answer{"ids": ["A637", "J191"]}wrong_schema_or_value
{"ids": ["A637", "J191", "C922"]}23 tokensCorrect
{"ids":["A637","J191"]}16 tokensCorrect
{"ids":["A637","J191"]}16 tokensCorrect
{"ids":["A637","J191"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"H507","units":3,"price":14},{"id":"E516","units":1,"price":11},{"id":"P814","units":7,"price":9},{"id":"J429","units":1,"price":17},{"id":"B661","units":9,"price":16}]Checked answer{"total": 277}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":278}8 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"B734","priority":5},{"id":"B797","priority":8},{"id":"K396","priority":8},{"id":"H796","priority":5},{"id":"P817","priority":6}]Checked answer{"ids": ["B797", "K396"]}wrong_schema_or_value
{"ids": ["B797", "B734", "H796", "K396", "P817"]}35 tokenswrong_schema_or_value
{"ids":["K396","B797"]}16 tokensCorrect
{"ids":["B797","K396"]}16 tokenswrong_schema_or_value
{"ids":["K396","B797"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"C624","city":"Oslo","units":19},{"id":"S405","city":"Bern","units":17},{"id":"S157","city":"Oslo","units":14},{"id":"U336","city":"Bern","units":8},{"id":"S179","city":"Riga","units":7}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Oslo","units":19}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"O904","status":"pending"},{"id":"W446","status":"closed"},{"id":"G460","status":"pending"},{"id":"T195","status":"pending"},{"id":"P653","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{
"ids": ["O904", "W446", "G460", "T195", "P653"]
}38 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"C925","units":9,"price":4},{"id":"V247","units":15,"price":7},{"id":"H653","units":16,"price":8},{"id":"T966","units":1,"price":14},{"id":"B699","units":14,"price":10}]Checked answer{"total": 423}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"W538","priority":8},{"id":"A515","priority":8},{"id":"U713","priority":5},{"id":"E817","priority":3},{"id":"Y714","priority":7}]Checked answer{"ids": ["A515", "W538"]}invalid_json_object
[{"id":"W538","priority":8},{"id":"A515","priority":8},{"id":"U713","priority":5},{"id":"E817","priority":3},{"id":"Y714","priority":7}]62 tokensCorrect
{"ids":["A515","W538"]}16 tokenswrong_schema_or_value
{"ids":["W538","A515"]}16 tokensCorrect
{"ids":["A515","W538"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to L724? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"E457","city":"Kyoto","units":0},{"id":"J733","city":"Perth","units":14},{"id":"L724","city":"Perth","units":3},{"id":"F467","city":"Riga","units":0},{"id":"I851","city":"Perth","units":19}]Checked answer{"city": "Perth", "units": 3}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Perth","units":3}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"J745","status":"closed"},{"id":"H159","status":"pending"},{"id":"D235","status":"closed"},{"id":"K784","status":"closed"},{"id":"Q164","status":"ready"}]Checked answer{"ids": ["Q164"]}invalid_json_object
[{"id":"Q164"}]11 tokensinvalid_json_object
["Q164"]7 tokens
Correct
{"ids":["Q164"]}11 tokensinvalid_json_object
["ids":["Q164"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"T683","units":19,"price":19},{"id":"L971","units":16,"price":13},{"id":"Q853","units":13,"price":7},{"id":"A482","units":4,"price":18},{"id":"K987","units":19,"price":13}]Checked answer{"total": 979}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"K458","priority":8},{"id":"O751","priority":1},{"id":"G580","priority":8},{"id":"C200","priority":1},{"id":"O183","priority":4}]Checked answer{"ids": ["G580", "K458"]}wrong_schema_or_value
{"ids": ["O183", "O751", "C200", "G580", "K458"]}35 tokensCorrect
{"ids":["G580","K458"]}16 tokensCorrect
{"ids":["G580","K458"]}16 tokensCorrect
{"ids":["G580","K458"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=F324. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"E127","city":"Oslo","units":5},{"id":"F324","city":"Oslo","units":14},{"id":"X705","city":"Bern","units":16},{"id":"U596","city":"Perth","units":15},{"id":"O937","city":"Oslo","units":16}]Checked answer{"city": "Oslo", "units": 14}Correct
{"city":"Oslo","units":14}12 tokensCorrect
{"city":"Oslo","units":14}12 tokensCorrect
{"city":"Oslo","units":14}12 tokensCorrect
{"city":"Oslo","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"N665","status":"ready"},{"id":"U774","status":"closed"},{"id":"X961","status":"ready"},{"id":"X845","status":"ready"},{"id":"S234","status":"pending"}]Checked answer{"ids": ["N665", "X845", "X961"]}wrong_schema_or_value
{"ids": ["N665", "X845", "X961", "U774", "S234"]}35 tokenswrong_schema_or_value
{"ids":["N665","X961","X845"]}21 tokenswrong_schema_or_value
{"ids":["N665","X961","X845"]}21 tokenswrong_schema_or_value
{"ids":["N665","X961","X845"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"Y695","units":3,"price":19},{"id":"Q231","units":11,"price":17},{"id":"T870","units":2,"price":13},{"id":"C997","units":12,"price":8},{"id":"A184","units":8,"price":5}]Checked answer{"total": 406}invalid_json_object
{
"total": 19 + 17 + 13 + 8 + 5 = 69
}29 tokenswrong_schema_or_value
{"total":378}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"U578","priority":8},{"id":"Y533","priority":5},{"id":"N287","priority":7},{"id":"O735","priority":8},{"id":"J243","priority":7}]Checked answer{"ids": ["O735", "U578"]}invalid_json_object
[{"id":"U578"},{"id":"Y533"},{"id":"N287"},{"id":"O735"},{"id":"J243"}]43 tokensCorrect
{"ids":["O735","U578"]}16 tokensCorrect
{"ids":["O735","U578"]}16 tokensCorrect
{"ids":["O735","U578"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record R434. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"T680","city":"Lima","units":1},{"id":"H445","city":"Bern","units":1},{"id":"T756","city":"Bern","units":4},{"id":"R434","city":"Perth","units":6},{"id":"X415","city":"Lima","units":16}]Checked answer{"city": "Perth", "units": 6}Correct
{
"city": "Perth",
"units": 6
}18 tokensCorrect
{"city":"Perth","units":6}11 tokensCorrect
{"city":"Perth","units":6}11 tokensCorrect
{"city":"Perth","units":6}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"D578","status":"closed"},{"id":"R601","status":"closed"},{"id":"A723","status":"ready"},{"id":"D729","status":"pending"},{"id":"B339","status":"pending"}]Checked answer{"ids": ["A723"]}wrong_schema_or_value
{"ids": ["A723", "D729", "D578", "R601", "B339"]}35 tokensCorrect
{"ids":["A723"]}11 tokensCorrect
{"ids":["A723"]}11 tokensCorrect
{"ids":["A723"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"B197","units":4,"price":9},{"id":"J829","units":13,"price":4},{"id":"W783","units":5,"price":3},{"id":"Q228","units":7,"price":11},{"id":"O743","units":3,"price":3}]Checked answer{"total": 189}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"A313","priority":7},{"id":"N397","priority":3},{"id":"Z328","priority":1},{"id":"X478","priority":1},{"id":"F785","priority":2}]Checked answer{"ids": ["A313", "N397"]}wrong_schema_or_value
{"ids": ["N397", "X478", "F785", "Z328", "A313"]}35 tokenswrong_schema_or_value
{"ids":["A313","F785"]}16 tokensCorrect
{"ids":["A313","N397"]}16 tokensCorrect
{"ids":["A313","N397"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id G230 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"G230","city":"Suva","units":14},{"id":"L492","city":"Perth","units":7},{"id":"R349","city":"Riga","units":12},{"id":"V821","city":"Bern","units":1},{"id":"H364","city":"Suva","units":1}]Checked answer{"city": "Suva", "units": 14}Correct
{"city":"Suva","units":14}12 tokensCorrect
{"city":"Suva","units":14}12 tokensCorrect
{"city":"Suva","units":14}12 tokensCorrect
{"city":"Suva","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"E253","status":"pending"},{"id":"X581","status":"closed"},{"id":"D269","status":"ready"},{"id":"M513","status":"pending"},{"id":"K177","status":"pending"}]Checked answer{"ids": ["D269"]}wrong_schema_or_value
{
"ids": ["D269", "E253", "K177", "M513", "X581"]
}38 tokensCorrect
{"ids":["D269"]}11 tokensCorrect
{"ids":["D269"]}11 tokensCorrect
{"ids":["D269"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"Q200","units":15,"price":16},{"id":"O912","units":19,"price":12},{"id":"E317","units":10,"price":10},{"id":"W584","units":2,"price":8},{"id":"D342","units":13,"price":14}]Checked answer{"total": 766}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Z885","priority":3},{"id":"L920","priority":2},{"id":"N730","priority":8},{"id":"H504","priority":2},{"id":"S685","priority":1}]Checked answer{"ids": ["N730", "Z885"]}invalid_json_object
[{"id":"Z885","priority":3},{"id":"L920","priority":2},{"id":"N730","priority":8},{"id":"H504","priority":2},{"id":"S685","priority":1}]62 tokensCorrect
{"ids":["N730","Z885"]}16 tokensCorrect
{"ids":["N730","Z885"]}16 tokensCorrect
{"ids":["N730","Z885"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"Z968","city":"Oslo","units":17},{"id":"C443","city":"Oslo","units":19},{"id":"J260","city":"Perth","units":15},{"id":"P990","city":"Kyoto","units":1},{"id":"U593","city":"Lima","units":2}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"I831","status":"ready"},{"id":"R867","status":"pending"},{"id":"Q143","status":"closed"},{"id":"I536","status":"pending"},{"id":"X687","status":"closed"}]Checked answer{"ids": ["I831"]}invalid_json_object
[{"id":"I831","status":"ready"},{"id":"R867","status":"pending"},{"id":"Q143","status":"closed"},{"id":"I536","status":"pending"},{"id":"X687","status":"closed"}]63 tokensCorrect
{"ids":["I831"]}11 tokensCorrect
{"ids":["I831"]}11 tokensinvalid_json_object
["ids":["I831"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"N488","units":1,"price":13},{"id":"Q723","units":6,"price":13},{"id":"M534","units":17,"price":2},{"id":"X126","units":9,"price":11},{"id":"H250","units":13,"price":11}]Checked answer{"total": 367}wrong_schema_or_value
{"total": 13}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Y962","priority":8},{"id":"F682","priority":6},{"id":"Y605","priority":9},{"id":"K731","priority":5},{"id":"N595","priority":1}]Checked answer{"ids": ["Y605", "Y962"]}wrong_schema_or_value
{"ids": ["Y962", "K731"]}17 tokensCorrect
{"ids":["Y605","Y962"]}16 tokensCorrect
{"ids":["Y605","Y962"]}16 tokenswrong_schema_or_value
{"ids":["F682","Y605"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=W554. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"G618","city":"Kyoto","units":17},{"id":"Y933","city":"Riga","units":12},{"id":"I989","city":"Bern","units":1},{"id":"S143","city":"Suva","units":15},{"id":"W554","city":"Lima","units":18}]Checked answer{"city": "Lima", "units": 18}wrong_schema_or_value
{"city":"Kyoto","units":17}13 tokensCorrect
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"T569","status":"pending"},{"id":"Z731","status":"pending"},{"id":"H377","status":"pending"},{"id":"T902","status":"pending"},{"id":"V319","status":"ready"}]Checked answer{"ids": ["V319"]}wrong_schema_or_value
{"ids": ["H377", "T902", "V319", "T569", "Z731"]}35 tokensCorrect
{"ids":["V319"]}11 tokensCorrect
{"ids":["V319"]}11 tokensCorrect
{"ids":["V319"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"S125","units":11,"price":8},{"id":"Q759","units":6,"price":10},{"id":"Y886","units":9,"price":12},{"id":"I135","units":15,"price":8},{"id":"M580","units":3,"price":16}]Checked answer{"total": 424}wrong_schema_or_value
{
"total": 30
}11 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"T473","priority":7},{"id":"G191","priority":9},{"id":"I575","priority":4},{"id":"Z421","priority":4},{"id":"T746","priority":9}]Checked answer{"ids": ["G191", "T746"]}invalid_json_object
[{"id":"T473","priority":7},{"id":"G191","priority":9},{"id":"I575","priority":4},{"id":"Z421","priority":4},{"id":"T746","priority":9}]62 tokensCorrect
{"ids":["G191","T746"]}16 tokensCorrect
{"ids":["G191","T746"]}16 tokensCorrect
{"ids":["G191","T746"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record L149. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"S929","city":"Lima","units":1},{"id":"L149","city":"Kyoto","units":19},{"id":"W538","city":"Riga","units":2},{"id":"U279","city":"Kyoto","units":18},{"id":"V155","city":"Riga","units":2}]Checked answer{"city": "Kyoto", "units": 19}wrong_schema_or_value
{
"city": "Lima",
"units": 1
}18 tokensCorrect
{"city":"Kyoto","units":19}13 tokensCorrect
{"city":"Kyoto","units":19}13 tokensCorrect
{"city":"Kyoto","units":19}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"W366","status":"pending"},{"id":"C510","status":"ready"},{"id":"R158","status":"ready"},{"id":"O501","status":"closed"},{"id":"N177","status":"ready"}]Checked answer{"ids": ["C510", "N177", "R158"]}wrong_schema_or_value
{"ids": ["C510", "R158", "N177"]}23 tokensCorrect
{"ids":["C510","N177","R158"]}21 tokensCorrect
{"ids":["C510","N177","R158"]}21 tokensCorrect
{"ids":["C510","N177","R158"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"W810","units":8,"price":5},{"id":"Y586","units":10,"price":14},{"id":"V932","units":11,"price":3},{"id":"V815","units":4,"price":17},{"id":"B821","units":2,"price":18}]Checked answer{"total": 317}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"U104","priority":9},{"id":"T432","priority":2},{"id":"D702","priority":9},{"id":"S475","priority":3},{"id":"F544","priority":8}]Checked answer{"ids": ["D702", "U104"]}wrong_schema_or_value
{"ids": ["U104", "T432", "D702", "S475", "F544"]}35 tokensCorrect
{"ids":["D702","U104"]}16 tokensCorrect
{"ids":["D702","U104"]}16 tokensCorrect
{"ids":["D702","U104"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id B553 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"B553","city":"Bern","units":18},{"id":"P477","city":"Lima","units":4},{"id":"S992","city":"Suva","units":11},{"id":"K994","city":"Lima","units":19},{"id":"S753","city":"Oslo","units":16}]Checked answer{"city": "Bern", "units": 18}Correct
{"city":"Bern","units":18}11 tokensCorrect
{"city":"Bern","units":18}11 tokensCorrect
{"city":"Bern","units":18}11 tokensCorrect
{"city":"Bern","units":18}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"A862","status":"closed"},{"id":"T516","status":"closed"},{"id":"O223","status":"closed"},{"id":"X867","status":"pending"},{"id":"B449","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{
"ids": ["A862", "T516", "O223", "X867"]
}32 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"P698","units":10,"price":3},{"id":"E925","units":15,"price":5},{"id":"Y774","units":18,"price":17},{"id":"K543","units":12,"price":18},{"id":"I629","units":11,"price":12}]Checked answer{"total": 759}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"G440","priority":3},{"id":"O240","priority":8},{"id":"R256","priority":2},{"id":"W481","priority":4},{"id":"U149","priority":3}]Checked answer{"ids": ["O240", "W481"]}invalid_json_object
[{"id":"G440","priority":3},{"id":"O240","priority":8},{"id":"R256","priority":2},{"id":"W481","priority":4},{"id":"U149","priority":3}]62 tokensCorrect
{"ids":["O240","W481"]}16 tokensCorrect
{"ids":["O240","W481"]}16 tokensCorrect
{"ids":["O240","W481"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to J343? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"J343","city":"Riga","units":13},{"id":"O892","city":"Suva","units":9},{"id":"O365","city":"Kyoto","units":6},{"id":"G728","city":"Lima","units":4},{"id":"V759","city":"Accra","units":12}]Checked answer{"city": "Riga", "units": 13}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"F990","status":"ready"},{"id":"J219","status":"closed"},{"id":"J142","status":"pending"},{"id":"O518","status":"ready"},{"id":"V621","status":"closed"}]Checked answer{"ids": ["F990", "O518"]}invalid_json_object
[{"id":"F990"},{"id":"O518"},{"id":"V621"},{"id":"J142"},{"id":"J219"}]43 tokensCorrect
{"ids":["F990","O518"]}16 tokensCorrect
{"ids":["F990","O518"]}16 tokensinvalid_json_object
["ids":["F990","O518"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"I807","units":3,"price":13},{"id":"G594","units":7,"price":16},{"id":"F147","units":5,"price":2},{"id":"O621","units":4,"price":2},{"id":"C485","units":14,"price":5}]Checked answer{"total": 239}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"D255","priority":9},{"id":"R659","priority":8},{"id":"A714","priority":5},{"id":"L888","priority":9},{"id":"Z339","priority":5}]Checked answer{"ids": ["D255", "L888"]}wrong_schema_or_value
{"ids": ["D255", "A714", "L888", "Z339", "R659"]}35 tokensCorrect
{"ids":["D255","L888"]}16 tokenswrong_schema_or_value
{"ids":["L888","D255"]}16 tokensCorrect
{"ids":["D255","L888"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"J115","city":"Oslo","units":7},{"id":"J733","city":"Perth","units":11},{"id":"J290","city":"Perth","units":4},{"id":"N690","city":"Lima","units":19},{"id":"S834","city":"Accra","units":19}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Oslo","units":7}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"K258","status":"ready"},{"id":"G343","status":"ready"},{"id":"T491","status":"pending"},{"id":"Z721","status":"closed"},{"id":"I754","status":"ready"}]Checked answer{"ids": ["G343", "I754", "K258"]}wrong_schema_or_value
{"ids": ["K258", "G343", "T491", "Z721", "I754"]}35 tokensCorrect
{"ids":["G343","I754","K258"]}21 tokenswrong_schema_or_value
{"ids":["I754","K258","G343"]}21 tokenswrong_schema_or_value
{"ids":["I754","K258","G343"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"C788","units":0,"price":12},{"id":"X579","units":8,"price":16},{"id":"S921","units":8,"price":4},{"id":"N523","units":17,"price":13},{"id":"G707","units":11,"price":16}]Checked answer{"total": 557}invalid_json_object
{
"total": 12 + 128 + 12 + 132 + 176
}30 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"P626","priority":7},{"id":"F579","priority":5},{"id":"L141","priority":2},{"id":"G843","priority":3},{"id":"N210","priority":9}]Checked answer{"ids": ["N210", "P626"]}invalid_json_object
[{"id":"P626","priority":7},{"id":"F579","priority":5},{"id":"L141","priority":2},{"id":"G843","priority":3},{"id":"N210","priority":9}]62 tokensCorrect
{"ids":["N210","P626"]}16 tokensCorrect
{"ids":["N210","P626"]}16 tokensCorrect
{"ids":["N210","P626"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record M795. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Q112","city":"Accra","units":18},{"id":"N156","city":"Oslo","units":3},{"id":"X914","city":"Riga","units":18},{"id":"P661","city":"Kyoto","units":13},{"id":"M795","city":"Bern","units":9}]Checked answer{"city": "Bern", "units": 9}Correct
{
"city": "Bern",
"units": 9
}17 tokensCorrect
{"city":"Bern","units":9}10 tokensCorrect
{"city":"Bern","units":9}10 tokensCorrect
{"city":"Bern","units":9}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"O681","status":"pending"},{"id":"Y922","status":"ready"},{"id":"Y316","status":"closed"},{"id":"Y114","status":"pending"},{"id":"T319","status":"pending"}]Checked answer{"ids": ["Y922"]}wrong_schema_or_value
{"ids": ["Y114", "Y316", "Y922", "O681"]}29 tokensCorrect
{"ids":["Y922"]}11 tokensCorrect
{"ids":["Y922"]}11 tokensCorrect
{"ids":["Y922"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"P313","units":0,"price":10},{"id":"E113","units":16,"price":13},{"id":"C713","units":5,"price":10},{"id":"J784","units":11,"price":3},{"id":"K999","units":2,"price":2}]Checked answer{"total": 295}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":418}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"O163","priority":1},{"id":"T427","priority":3},{"id":"C741","priority":4},{"id":"D338","priority":4},{"id":"Z195","priority":3}]Checked answer{"ids": ["C741", "D338"]}wrong_schema_or_value
{"ids": ["O163", "T427", "C741", "D338", "Z195"]}35 tokensCorrect
{"ids":["C741","D338"]}16 tokensCorrect
{"ids":["C741","D338"]}16 tokensCorrect
{"ids":["C741","D338"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id N100 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"D169","city":"Suva","units":14},{"id":"Y749","city":"Lima","units":13},{"id":"N100","city":"Perth","units":14},{"id":"J422","city":"Perth","units":1},{"id":"E858","city":"Lima","units":7}]Checked answer{"city": "Perth", "units": 14}wrong_schema_or_value
{"city":"Suva","units":14}12 tokensCorrect
{"city":"Perth","units":14}12 tokensCorrect
{"city":"Perth","units":14}12 tokensCorrect
{"city":"Perth","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"J980","status":"ready"},{"id":"B733","status":"closed"},{"id":"A148","status":"pending"},{"id":"D184","status":"closed"},{"id":"D961","status":"ready"}]Checked answer{"ids": ["D961", "J980"]}wrong_schema_or_value
{
"ids": ["A148", "B733", "D184", "D961", "J980"]
}38 tokensCorrect
{"ids":["D961","J980"]}16 tokensCorrect
{"ids":["D961","J980"]}16 tokenswrong_schema_or_value
{"ids":["D184","J980"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"J339","units":12,"price":18},{"id":"U329","units":11,"price":6},{"id":"L543","units":11,"price":14},{"id":"E426","units":17,"price":18},{"id":"J852","units":10,"price":7}]Checked answer{"total": 812}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"J928","priority":2},{"id":"F971","priority":8},{"id":"D515","priority":6},{"id":"R144","priority":8},{"id":"H528","priority":4}]Checked answer{"ids": ["F971", "R144"]}invalid_json_object
[{"id":"J928","priority":2},{"id":"F971","priority":8},{"id":"D515","priority":6},{"id":"R144","priority":8},{"id":"H528","priority":4}]62 tokensCorrect
{"ids":["F971","R144"]}16 tokensCorrect
{"ids":["F971","R144"]}16 tokenswrong_schema_or_value
{"ids":["D515","R144"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to F136? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"F136","city":"Perth","units":0},{"id":"G600","city":"Oslo","units":14},{"id":"N434","city":"Accra","units":2},{"id":"U869","city":"Bern","units":16},{"id":"C908","city":"Bern","units":14}]Checked answer{"city": "Perth", "units": 0}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Perth","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"X401","status":"pending"},{"id":"W825","status":"pending"},{"id":"G923","status":"ready"},{"id":"I141","status":"pending"},{"id":"K611","status":"closed"}]Checked answer{"ids": ["G923"]}invalid_json_object
[{"id":"G923","status":"ready"},{"id":"I141","status":"pending"},{"id":"K611","status":"closed"}]39 tokensCorrect
{"ids":["G923"]}11 tokensCorrect
{"ids":["G923"]}11 tokensinvalid_json_object
["ids":["G923"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Q371","units":4,"price":9},{"id":"D295","units":11,"price":19},{"id":"T452","units":0,"price":17},{"id":"A189","units":14,"price":2},{"id":"J418","units":4,"price":3}]Checked answer{"total": 285}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":408}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"K820","priority":3},{"id":"T710","priority":4},{"id":"R208","priority":4},{"id":"D374","priority":9},{"id":"D836","priority":3}]Checked answer{"ids": ["D374", "R208"]}wrong_schema_or_value
{"ids": ["K820", "T710", "R208", "D374", "D836"]}35 tokensCorrect
{"ids":["D374","R208"]}16 tokenswrong_schema_or_value
{"ids":["D374","T710"]}16 tokenswrong_schema_or_value
{"ids":["D374","K820"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=T964. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"C248","city":"Riga","units":13},{"id":"X356","city":"Oslo","units":14},{"id":"T964","city":"Riga","units":15},{"id":"Q284","city":"Accra","units":16},{"id":"B478","city":"Bern","units":15}]Checked answer{"city": "Riga", "units": 15}Correct
{"city":"Riga","units":15}12 tokensCorrect
{"city":"Riga","units":15}12 tokensCorrect
{"city":"Riga","units":15}12 tokensCorrect
{"city":"Riga","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"X835","status":"closed"},{"id":"T498","status":"pending"},{"id":"S963","status":"closed"},{"id":"Z402","status":"pending"},{"id":"W270","status":"ready"}]Checked answer{"ids": ["W270"]}wrong_schema_or_value
{"ids": ["W270", "X835", "S963", "T498", "Z402"]}35 tokensCorrect
{"ids":["W270"]}11 tokensCorrect
{"ids":["W270"]}11 tokensCorrect
{"ids":["W270"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"V731","units":2,"price":7},{"id":"L800","units":7,"price":11},{"id":"K145","units":4,"price":7},{"id":"H773","units":12,"price":4},{"id":"A545","units":3,"price":3}]Checked answer{"total": 176}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":288}8 tokenswrong_schema_or_value
{"total":278}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"J688","priority":4},{"id":"R821","priority":9},{"id":"X517","priority":2},{"id":"R224","priority":8},{"id":"H886","priority":3}]Checked answer{"ids": ["R821", "R224"]}invalid_json_object
[{"id":"J688"},{"id":"R821"},{"id":"X517"},{"id":"R224"},{"id":"H886"}]43 tokenswrong_schema_or_value
{"ids":["R224","R821"]}16 tokenswrong_schema_or_value
{"ids":["R821","J688"]}16 tokenswrong_schema_or_value
{"ids":["R821","J688"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"P392","city":"Kyoto","units":5},{"id":"H624","city":"Lima","units":19},{"id":"J943","city":"Perth","units":19},{"id":"M414","city":"Oslo","units":8},{"id":"B558","city":"Bern","units":12}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{
"city": "Kyoto",
"units": 5
}19 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"F738","status":"ready"},{"id":"X369","status":"ready"},{"id":"R406","status":"pending"},{"id":"H487","status":"pending"},{"id":"E663","status":"pending"}]Checked answer{"ids": ["F738", "X369"]}wrong_schema_or_value
{"ids": ["F738", "X369", "H487", "E663"]}29 tokensCorrect
{"ids":["F738","X369"]}16 tokensCorrect
{"ids":["F738","X369"]}16 tokensCorrect
{"ids":["F738","X369"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"D537","units":14,"price":9},{"id":"C664","units":12,"price":15},{"id":"U624","units":12,"price":13},{"id":"Z359","units":11,"price":13},{"id":"P707","units":19,"price":5}]Checked answer{"total": 700}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"U655","priority":8},{"id":"S594","priority":9},{"id":"Z866","priority":3},{"id":"U871","priority":7},{"id":"N809","priority":7}]Checked answer{"ids": ["S594", "U655"]}wrong_schema_or_value
{"ids": ["U655", "S594", "Z866", "U871", "N809"]}35 tokensCorrect
{"ids":["S594","U655"]}16 tokensCorrect
{"ids":["S594","U655"]}16 tokensCorrect
{"ids":["S594","U655"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z822 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"Z822","city":"Perth","units":10},{"id":"A193","city":"Bern","units":17},{"id":"Q656","city":"Lima","units":16},{"id":"Z815","city":"Suva","units":10},{"id":"Y988","city":"Suva","units":3}]Checked answer{"city": "Perth", "units": 10}Correct
{"city":"Perth","units":10}12 tokensCorrect
{"city":"Perth","units":10}12 tokensCorrect
{"city":"Perth","units":10}12 tokensCorrect
{"city":"Perth","units":10}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"U355","status":"pending"},{"id":"K466","status":"ready"},{"id":"I984","status":"ready"},{"id":"H714","status":"ready"},{"id":"G370","status":"ready"}]Checked answer{"ids": ["G370", "H714", "I984", "K466"]}wrong_schema_or_value
{
"ids": ["U355", "K466", "I984", "H714", "G370"]
}38 tokenswrong_schema_or_value
{"ids":["H714","I984","K466"]}21 tokensCorrect
{"ids":["G370","H714","I984","K466"]}26 tokenswrong_schema_or_value
{"ids":["I984","H714","G370"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"N835","units":14,"price":12},{"id":"P348","units":3,"price":6},{"id":"D397","units":0,"price":18},{"id":"X778","units":4,"price":15},{"id":"F431","units":17,"price":2}]Checked answer{"total": 280}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":408}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Z497","priority":5},{"id":"C946","priority":5},{"id":"Z179","priority":9},{"id":"T713","priority":8},{"id":"V102","priority":7}]Checked answer{"ids": ["Z179", "T713"]}invalid_json_object
[{"id":"Z497","priority":5},{"id":"C946","priority":5},{"id":"Z179","priority":9},{"id":"T713","priority":8},{"id":"V102","priority":7}]62 tokensCorrect
{"ids":["Z179","T713"]}16 tokensCorrect
{"ids":["Z179","T713"]}16 tokensCorrect
{"ids":["Z179","T713"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to D745? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"P384","city":"Kyoto","units":10},{"id":"W628","city":"Lima","units":0},{"id":"D745","city":"Kyoto","units":13},{"id":"L610","city":"Perth","units":12},{"id":"N526","city":"Accra","units":10}]Checked answer{"city": "Kyoto", "units": 13}wrong_schema_or_value
{
"city": null,
"units": 13
}17 tokensCorrect
{"city":"Kyoto","units":13}13 tokensCorrect
{"city":"Kyoto","units":13}13 tokensCorrect
{"city":"Kyoto","units":13}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"S447","status":"closed"},{"id":"U416","status":"pending"},{"id":"A618","status":"pending"},{"id":"V434","status":"pending"},{"id":"V157","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"A618","status":"pending"},{"id":"V157","status":"closed"},{"id":"V434","status":"pending"},{"id":"S447","status":"closed"}]51 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"T591","units":14,"price":7},{"id":"S192","units":16,"price":9},{"id":"X175","units":14,"price":6},{"id":"H683","units":4,"price":10},{"id":"F629","units":7,"price":8}]Checked answer{"total": 422}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"F864","priority":6},{"id":"P527","priority":1},{"id":"N714","priority":3},{"id":"Y726","priority":3},{"id":"D885","priority":8}]Checked answer{"ids": ["D885", "F864"]}wrong_schema_or_value
{"ids": ["F864", "P527", "N714", "Y726", "D885"]}35 tokensCorrect
{"ids":["D885","F864"]}16 tokensCorrect
{"ids":["D885","F864"]}16 tokensCorrect
{"ids":["D885","F864"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=R704. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"T373","city":"Oslo","units":10},{"id":"R704","city":"Suva","units":10},{"id":"F957","city":"Oslo","units":3},{"id":"L535","city":"Oslo","units":14},{"id":"P165","city":"Riga","units":0}]Checked answer{"city": "Suva", "units": 10}wrong_schema_or_value
{"city":"Oslo","units":10}12 tokensCorrect
{"city":"Suva","units":10}12 tokensCorrect
{"city":"Suva","units":10}12 tokensCorrect
{"city":"Suva","units":10}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"W704","status":"closed"},{"id":"X692","status":"ready"},{"id":"H480","status":"ready"},{"id":"U389","status":"closed"},{"id":"M818","status":"pending"}]Checked answer{"ids": ["H480", "X692"]}wrong_schema_or_value
{"ids": ["X692", "H480", "U389", "M818", "W704"]}35 tokensCorrect
{"ids":["H480","X692"]}16 tokensCorrect
{"ids":["H480","X692"]}16 tokensCorrect
{"ids":["H480","X692"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"J835","units":15,"price":19},{"id":"S573","units":8,"price":8},{"id":"W496","units":12,"price":2},{"id":"M548","units":4,"price":17},{"id":"X347","units":5,"price":1}]Checked answer{"total": 446}invalid_json_object
{
"total": 19 + 8 + 2 + 17 + 1 = 47
}28 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"J135","priority":8},{"id":"K105","priority":4},{"id":"X828","priority":1},{"id":"Z995","priority":1},{"id":"L284","priority":1}]Checked answer{"ids": ["J135", "K105"]}invalid_json_object
[{"id":"Z995","priority":1},{"id":"L284","priority":1},{"id":"J135","priority":8},{"id":"K105","priority":4},{"id":"X828","priority":1}]62 tokensCorrect
{"ids":["J135","K105"]}16 tokensCorrect
{"ids":["J135","K105"]}16 tokensCorrect
{"ids":["J135","K105"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record G956. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"A537","city":"Kyoto","units":17},{"id":"G956","city":"Accra","units":12},{"id":"Z461","city":"Riga","units":2},{"id":"T373","city":"Suva","units":14},{"id":"E837","city":"Bern","units":11}]Checked answer{"city": "Accra", "units": 12}wrong_schema_or_value
{
"city": "Kyoto",
"units": 17
}20 tokensCorrect
{"city":"Accra","units":12}12 tokensCorrect
{"city":"Accra","units":12}12 tokensCorrect
{"city":"Accra","units":12}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"C626","status":"closed"},{"id":"U343","status":"ready"},{"id":"C569","status":"ready"},{"id":"J676","status":"closed"},{"id":"G990","status":"pending"}]Checked answer{"ids": ["C569", "U343"]}wrong_schema_or_value
{"ids": ["C569", "C626", "U343", "J676", "G990"]}35 tokensCorrect
{"ids":["C569","U343"]}16 tokenswrong_schema_or_value
{"ids":["C569","J676"]}16 tokensCorrect
{"ids":["C569","U343"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"T883","units":7,"price":16},{"id":"W237","units":0,"price":5},{"id":"J467","units":7,"price":3},{"id":"W465","units":17,"price":2},{"id":"M158","units":15,"price":7}]Checked answer{"total": 272}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"K483","priority":8},{"id":"R263","priority":2},{"id":"Q974","priority":8},{"id":"Q663","priority":4},{"id":"T453","priority":3}]Checked answer{"ids": ["K483", "Q974"]}wrong_schema_or_value
{"ids": ["R263", "K483", "T453", "Q663", "Q974"]}35 tokensCorrect
{"ids":["K483","Q974"]}16 tokensCorrect
{"ids":["K483","Q974"]}16 tokenswrong_schema_or_value
{"ids":["K483","Q663"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"K968","city":"Bern","units":12},{"id":"B259","city":"Accra","units":15},{"id":"C247","city":"Suva","units":1},{"id":"R857","city":"Kyoto","units":3},{"id":"Z457","city":"Oslo","units":11}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Bern","units":12}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"F743","status":"closed"},{"id":"T808","status":"closed"},{"id":"Z145","status":"pending"},{"id":"J672","status":"ready"},{"id":"X110","status":"pending"}]Checked answer{"ids": ["J672"]}wrong_schema_or_value
{
"ids": ["F743", "T808", "Z145", "J672", "X110"]
}38 tokensCorrect
{"ids":["J672"]}11 tokensCorrect
{"ids":["J672"]}11 tokensCorrect
{"ids":["J672"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"E729","units":9,"price":5},{"id":"Z755","units":2,"price":9},{"id":"S440","units":13,"price":1},{"id":"K264","units":2,"price":9},{"id":"R737","units":6,"price":12}]Checked answer{"total": 166}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"P961","priority":6},{"id":"Y328","priority":3},{"id":"M750","priority":6},{"id":"J376","priority":3},{"id":"O985","priority":4}]Checked answer{"ids": ["M750", "P961"]}invalid_json_object
[{"id":"P961","priority":6},{"id":"M750","priority":6},{"id":"J376","priority":3},{"id":"O985","priority":4}]50 tokensCorrect
{"ids":["M750","P961"]}16 tokensCorrect
{"ids":["M750","P961"]}16 tokensCorrect
{"ids":["M750","P961"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to X610? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"S137","city":"Riga","units":1},{"id":"U521","city":"Lima","units":16},{"id":"K403","city":"Lima","units":6},{"id":"X610","city":"Bern","units":0},{"id":"Y286","city":"Kyoto","units":8}]Checked answer{"city": "Bern", "units": 0}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokenswrong_schema_or_value
{"city":null,"units":0}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Bern","units":0}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"I110","status":"closed"},{"id":"L598","status":"pending"},{"id":"W240","status":"pending"},{"id":"N829","status":"ready"},{"id":"V746","status":"closed"}]Checked answer{"ids": ["N829"]}invalid_json_object
[{"id":"N829"}]11 tokensCorrect
{"ids":["N829"]}11 tokensCorrect
{"ids":["N829"]}11 tokensinvalid_json_object
["ids":["N829"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"M898","units":9,"price":19},{"id":"K942","units":2,"price":7},{"id":"E350","units":17,"price":9},{"id":"V944","units":8,"price":5},{"id":"E309","units":4,"price":6}]Checked answer{"total": 402}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"S439","priority":8},{"id":"X422","priority":3},{"id":"U768","priority":2},{"id":"Y833","priority":2},{"id":"B287","priority":4}]Checked answer{"ids": ["S439", "B287"]}wrong_schema_or_value
{"ids": ["U768", "Y833", "S439", "X422", "B287"]}35 tokensCorrect
{"ids":["S439","B287"]}16 tokensCorrect
{"ids":["S439","B287"]}16 tokenswrong_schema_or_value
{"ids":["S439","X422"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=D551. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"J908","city":"Bern","units":10},{"id":"M109","city":"Accra","units":16},{"id":"K294","city":"Riga","units":3},{"id":"F975","city":"Perth","units":17},{"id":"D551","city":"Suva","units":16}]Checked answer{"city": "Suva", "units": 16}Correct
{"city": "Suva", "units": 16}15 tokensCorrect
{"city":"Suva","units":16}12 tokensCorrect
{"city":"Suva","units":16}12 tokensCorrect
{"city":"Suva","units":16}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"N493","status":"pending"},{"id":"S258","status":"closed"},{"id":"L382","status":"closed"},{"id":"B311","status":"ready"},{"id":"H516","status":"closed"}]Checked answer{"ids": ["B311"]}wrong_schema_or_value
{"ids": ["B311", "H516", "N493", "S258", "L382"]}35 tokensCorrect
{"ids":["B311"]}11 tokensCorrect
{"ids":["B311"]}11 tokensCorrect
{"ids":["B311"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"L288","units":9,"price":3},{"id":"J835","units":13,"price":12},{"id":"N401","units":10,"price":13},{"id":"J462","units":6,"price":6},{"id":"T367","units":17,"price":7}]Checked answer{"total": 468}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"X364","priority":1},{"id":"D255","priority":1},{"id":"T219","priority":3},{"id":"C671","priority":8},{"id":"G860","priority":7}]Checked answer{"ids": ["C671", "G860"]}invalid_json_object
[{"id":"X364","priority":1},{"id":"D255","priority":1},{"id":"T219","priority":3},{"id":"C671","priority":8},{"id":"G860","priority":7}]62 tokensCorrect
{"ids":["C671","G860"]}16 tokensCorrect
{"ids":["C671","G860"]}16 tokenswrong_schema_or_value
{"ids":["C671","D255"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record S608. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"F852","city":"Lima","units":18},{"id":"K758","city":"Lima","units":17},{"id":"S608","city":"Accra","units":12},{"id":"W545","city":"Riga","units":4},{"id":"E334","city":"Kyoto","units":18}]Checked answer{"city": "Accra", "units": 12}Correct
{
"city": "Accra",
"units": 12
}19 tokensCorrect
{"city":"Accra","units":12}12 tokensCorrect
{"city":"Accra","units":12}12 tokensCorrect
{"city":"Accra","units":12}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"N223","status":"ready"},{"id":"P278","status":"closed"},{"id":"L710","status":"pending"},{"id":"E461","status":"ready"},{"id":"U399","status":"ready"}]Checked answer{"ids": ["E461", "N223", "U399"]}wrong_schema_or_value
{"ids": ["N223", "P278", "L710", "E461", "U399"]}35 tokensCorrect
{"ids":["E461","N223","U399"]}21 tokensCorrect
{"ids":["E461","N223","U399"]}21 tokenswrong_schema_or_value
{"ids":["E461","N223","N223"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Z521","units":14,"price":8},{"id":"B792","units":15,"price":15},{"id":"P277","units":13,"price":14},{"id":"A232","units":14,"price":1},{"id":"B172","units":19,"price":1}]Checked answer{"total": 552}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"T225","priority":9},{"id":"Q576","priority":2},{"id":"L291","priority":6},{"id":"D231","priority":3},{"id":"C782","priority":1}]Checked answer{"ids": ["T225", "L291"]}wrong_schema_or_value
{"ids": ["T225", "Q576", "L291", "D231", "C782"]}35 tokensCorrect
{"ids":["T225","L291"]}16 tokensCorrect
{"ids":["T225","L291"]}16 tokenswrong_schema_or_value
{"ids":["T225","D231"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id D923 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"D923","city":"Accra","units":5},{"id":"P442","city":"Bern","units":14},{"id":"A204","city":"Suva","units":8},{"id":"F678","city":"Riga","units":2},{"id":"R120","city":"Suva","units":4}]Checked answer{"city": "Accra", "units": 5}Correct
{"city":"Accra","units":5}11 tokensCorrect
{"city":"Accra","units":5}11 tokensCorrect
{"city":"Accra","units":5}11 tokensCorrect
{"city":"Accra","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"T905","status":"closed"},{"id":"J414","status":"closed"},{"id":"C656","status":"pending"},{"id":"P962","status":"ready"},{"id":"O126","status":"pending"}]Checked answer{"ids": ["P962"]}wrong_schema_or_value
{
"ids": ["C656", "J414", "P962", "O126"]
}32 tokensCorrect
{"ids":["P962"]}11 tokensCorrect
{"ids":["P962"]}11 tokensCorrect
{"ids":["P962"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"Y185","units":6,"price":18},{"id":"V549","units":19,"price":7},{"id":"H691","units":18,"price":11},{"id":"M576","units":6,"price":8},{"id":"T829","units":9,"price":7}]Checked answer{"total": 550}wrong_schema_or_value
{"total": 102}9 tokenswrong_schema_or_value
{"total":548}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"B133","priority":9},{"id":"G781","priority":9},{"id":"R647","priority":6},{"id":"M360","priority":7},{"id":"X775","priority":3}]Checked answer{"ids": ["B133", "G781"]}invalid_json_object
[{"id":"B133","priority":9},{"id":"G781","priority":9},{"id":"R647","priority":6},{"id":"M360","priority":7},{"id":"X775","priority":3}]62 tokensCorrect
{"ids":["B133","G781"]}16 tokensCorrect
{"ids":["B133","G781"]}16 tokenswrong_schema_or_value
{"ids":["G781","B133"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"V249","city":"Kyoto","units":5},{"id":"U242","city":"Oslo","units":13},{"id":"P414","city":"Suva","units":19},{"id":"X863","city":"Kyoto","units":14},{"id":"O274","city":"Suva","units":13}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"Q497","status":"ready"},{"id":"I192","status":"closed"},{"id":"Z964","status":"closed"},{"id":"T197","status":"ready"},{"id":"N115","status":"closed"}]Checked answer{"ids": ["Q497", "T197"]}invalid_json_object
[{"id":"T197","status":"ready"},{"id":"Z964","status":"closed"},{"id":"Q497","status":"ready"},{"id":"N115","status":"closed"}]51 tokensCorrect
{"ids":["Q497","T197"]}16 tokensCorrect
{"ids":["Q497","T197"]}16 tokensinvalid_json_object
["ids":["Q497","T197"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"R879","units":11,"price":6},{"id":"S228","units":12,"price":15},{"id":"T707","units":6,"price":1},{"id":"Z890","units":17,"price":19},{"id":"X247","units":9,"price":5}]Checked answer{"total": 620}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"S691","priority":9},{"id":"G144","priority":3},{"id":"N990","priority":1},{"id":"R922","priority":4},{"id":"V119","priority":8}]Checked answer{"ids": ["S691", "V119"]}wrong_schema_or_value
{"ids": ["S691", "G144", "N990", "R922", "V119"]}35 tokensCorrect
{"ids":["S691","V119"]}16 tokensCorrect
{"ids":["S691","V119"]}16 tokensCorrect
{"ids":["S691","V119"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=T672. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"E446","city":"Riga","units":18},{"id":"K627","city":"Bern","units":16},{"id":"T672","city":"Accra","units":9},{"id":"S936","city":"Riga","units":9},{"id":"H127","city":"Lima","units":2}]Checked answer{"city": "Accra", "units": 9}wrong_schema_or_value
{"city":"Riga","units":18}12 tokensCorrect
{"city":"Accra","units":9}11 tokensCorrect
{"city":"Accra","units":9}11 tokensCorrect
{"city":"Accra","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"B842","status":"closed"},{"id":"F338","status":"ready"},{"id":"H588","status":"ready"},{"id":"R235","status":"ready"},{"id":"M142","status":"pending"}]Checked answer{"ids": ["F338", "H588", "R235"]}wrong_schema_or_value
{"ids": ["B842", "F338", "H588", "R235", "M142"]}35 tokensCorrect
{"ids":["F338","H588","R235"]}21 tokensCorrect
{"ids":["F338","H588","R235"]}21 tokensCorrect
{"ids":["F338","H588","R235"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"M961","units":0,"price":17},{"id":"T511","units":5,"price":11},{"id":"K542","units":9,"price":2},{"id":"O598","units":14,"price":19},{"id":"O828","units":7,"price":8}]Checked answer{"total": 395}invalid_json_object
{
"total": 17 + 11 + 2 + 19 + 8 = 59
}29 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"O284","priority":5},{"id":"P448","priority":6},{"id":"P614","priority":1},{"id":"P515","priority":9},{"id":"F277","priority":7}]Checked answer{"ids": ["P515", "F277"]}invalid_json_object
[{"id":"O284","priority":5},{"id":"P614","priority":1},{"id":"P448","priority":6},{"id":"P515","priority":9},{"id":"F277","priority":7}]62 tokensCorrect
{"ids":["P515","F277"]}16 tokenswrong_schema_or_value
{"ids":["P515","O284"]}16 tokenswrong_schema_or_value
{"ids":["O284","F277"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record H756. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"H756","city":"Lima","units":15},{"id":"P394","city":"Lima","units":5},{"id":"N824","city":"Lima","units":3},{"id":"I847","city":"Riga","units":15},{"id":"M833","city":"Bern","units":14}]Checked answer{"city": "Lima", "units": 15}Correct
{
"city": "Lima",
"units": 15
}19 tokensCorrect
{"city":"Lima","units":15}12 tokensCorrect
{"city":"Lima","units":15}12 tokensCorrect
{"city":"Lima","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"R542","status":"pending"},{"id":"B690","status":"pending"},{"id":"T228","status":"pending"},{"id":"P954","status":"closed"},{"id":"C461","status":"ready"}]Checked answer{"ids": ["C461"]}wrong_schema_or_value
{"ids": ["C461", "P954", "T228", "R542", "B690"]}35 tokensCorrect
{"ids":["C461"]}11 tokensCorrect
{"ids":["C461"]}11 tokensCorrect
{"ids":["C461"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"H201","units":17,"price":2},{"id":"C697","units":10,"price":5},{"id":"Q167","units":7,"price":19},{"id":"N941","units":14,"price":5},{"id":"Y908","units":15,"price":1}]Checked answer{"total": 302}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"J323","priority":7},{"id":"R579","priority":2},{"id":"H330","priority":6},{"id":"V271","priority":1},{"id":"G877","priority":7}]Checked answer{"ids": ["G877", "J323"]}wrong_schema_or_value
{"ids": ["R579", "J323", "H330", "V271", "G877"]}35 tokensCorrect
{"ids":["G877","J323"]}16 tokensCorrect
{"ids":["G877","J323"]}16 tokenswrong_schema_or_value
{"ids":["J323","G877"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id R444 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"M553","city":"Kyoto","units":19},{"id":"Z431","city":"Perth","units":11},{"id":"W633","city":"Bern","units":17},{"id":"R444","city":"Oslo","units":19},{"id":"J154","city":"Oslo","units":16}]Checked answer{"city": "Oslo", "units": 19}wrong_schema_or_value
{"city":"Kyoto","units":19}13 tokensCorrect
{"city":"Oslo","units":19}12 tokensCorrect
{"city":"Oslo","units":19}12 tokensCorrect
{"city":"Oslo","units":19}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"I980","status":"ready"},{"id":"B201","status":"pending"},{"id":"V111","status":"closed"},{"id":"S151","status":"closed"},{"id":"B621","status":"pending"}]Checked answer{"ids": ["I980"]}wrong_schema_or_value
{
"ids": ["I980", "B201", "V111", "S151", "B621"]
}38 tokensCorrect
{"ids":["I980"]}11 tokensCorrect
{"ids":["I980"]}11 tokensCorrect
{"ids":["I980"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"F504","units":16,"price":12},{"id":"E891","units":3,"price":9},{"id":"C490","units":2,"price":8},{"id":"P849","units":7,"price":12},{"id":"K366","units":10,"price":10}]Checked answer{"total": 419}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"X557","priority":4},{"id":"G482","priority":6},{"id":"W131","priority":3},{"id":"J151","priority":8},{"id":"F922","priority":3}]Checked answer{"ids": ["J151", "G482"]}invalid_json_object
[{"id":"X557","priority":4},{"id":"G482","priority":6},{"id":"W131","priority":3},{"id":"J151","priority":8},{"id":"F922","priority":3}]62 tokensCorrect
{"ids":["J151","G482"]}16 tokensCorrect
{"ids":["J151","G482"]}16 tokensCorrect
{"ids":["J151","G482"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to A664? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"O779","city":"Lima","units":3},{"id":"Y810","city":"Perth","units":5},{"id":"U544","city":"Perth","units":11},{"id":"A664","city":"Riga","units":7},{"id":"U169","city":"Oslo","units":1}]Checked answer{"city": "Riga", "units": 7}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Riga","units":7}11 tokensCorrect
{"city":"Riga","units":7}11 tokensCorrect
{"city":"Riga","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"B717","status":"closed"},{"id":"D570","status":"closed"},{"id":"T194","status":"pending"},{"id":"G119","status":"ready"},{"id":"Y573","status":"closed"}]Checked answer{"ids": ["G119"]}invalid_json_object
[{"id":"G119"}]11 tokensinvalid_json_object
["G119"]7 tokens
Correct
{"ids":["G119"]}11 tokensinvalid_json_object
["ids":["G119"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"F696","units":15,"price":15},{"id":"V590","units":7,"price":18},{"id":"J455","units":12,"price":15},{"id":"H281","units":1,"price":3},{"id":"Q790","units":10,"price":14}]Checked answer{"total": 674}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"C674","priority":7},{"id":"B836","priority":4},{"id":"H888","priority":6},{"id":"O982","priority":4},{"id":"G194","priority":9}]Checked answer{"ids": ["G194", "C674"]}wrong_schema_or_value
{"ids": ["C674", "H888"]}17 tokensCorrect
{"ids":["G194","C674"]}16 tokensCorrect
{"ids":["G194","C674"]}16 tokensCorrect
{"ids":["G194","C674"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"X151","city":"Oslo","units":19},{"id":"C181","city":"Kyoto","units":5},{"id":"E815","city":"Riga","units":1},{"id":"L698","city":"Oslo","units":10},{"id":"A326","city":"Accra","units":18}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Oslo","units":10}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Z647","status":"pending"},{"id":"I128","status":"pending"},{"id":"L485","status":"closed"},{"id":"D374","status":"pending"},{"id":"A681","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["A681", "D374", "I128", "L485", "Z647"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"S576","units":13,"price":2},{"id":"S942","units":0,"price":14},{"id":"W105","units":12,"price":2},{"id":"W714","units":0,"price":16},{"id":"Q606","units":6,"price":2}]Checked answer{"total": 62}wrong_schema_or_value
{
"total": 40
}11 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":388}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"I402","priority":8},{"id":"J912","priority":8},{"id":"U730","priority":5},{"id":"K527","priority":3},{"id":"D232","priority":9}]Checked answer{"ids": ["D232", "I402"]}invalid_json_object
[{"id":"I402","priority":8},{"id":"J912","priority":8},{"id":"U730","priority":5},{"id":"K527","priority":3},{"id":"D232","priority":9}]62 tokenswrong_schema_or_value
{"ids":["D232","J912"]}16 tokensCorrect
{"ids":["D232","I402"]}16 tokenswrong_schema_or_value
{"ids":["D232","J912"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record R685. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"R685","city":"Riga","units":3},{"id":"V157","city":"Suva","units":8},{"id":"W906","city":"Kyoto","units":16},{"id":"M135","city":"Riga","units":4},{"id":"N455","city":"Perth","units":11}]Checked answer{"city": "Riga", "units": 3}Correct
{
"city": "Riga",
"units": 3
}18 tokensCorrect
{"city":"Riga","units":3}11 tokensCorrect
{"city":"Riga","units":3}11 tokensCorrect
{"city":"Riga","units":3}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"P683","status":"ready"},{"id":"N364","status":"pending"},{"id":"N561","status":"ready"},{"id":"P575","status":"closed"},{"id":"V783","status":"closed"}]Checked answer{"ids": ["N561", "P683"]}wrong_schema_or_value
{"ids": ["P683", "N364", "N561", "P575", "V783"]}35 tokenswrong_schema_or_value
{"ids":["P683","N561"]}16 tokenswrong_schema_or_value
{"ids":["P683","N561"]}16 tokenswrong_schema_or_value
{"ids":["N364","P683"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Y346","units":10,"price":17},{"id":"O585","units":15,"price":1},{"id":"T305","units":10,"price":2},{"id":"N549","units":3,"price":7},{"id":"X837","units":1,"price":6}]Checked answer{"total": 232}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"J410","priority":4},{"id":"M497","priority":8},{"id":"V721","priority":6},{"id":"K881","priority":6},{"id":"R588","priority":2}]Checked answer{"ids": ["M497", "K881"]}wrong_schema_or_value
{"ids": ["M497", "V721", "K881", "J410"]}29 tokensCorrect
{"ids":["M497","K881"]}16 tokenswrong_schema_or_value
{"ids":["M497","V721"]}16 tokenswrong_schema_or_value
{"ids":["M497","J410"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id T395 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"T395","city":"Accra","units":11},{"id":"D961","city":"Lima","units":17},{"id":"J909","city":"Suva","units":11},{"id":"D255","city":"Accra","units":2},{"id":"T872","city":"Lima","units":4}]Checked answer{"city": "Accra", "units": 11}Correct
{"city":"Accra","units":11}12 tokensCorrect
{"city":"Accra","units":11}12 tokensCorrect
{"city":"Accra","units":11}12 tokensCorrect
{"city":"Accra","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"Z587","status":"pending"},{"id":"D374","status":"pending"},{"id":"E220","status":"pending"},{"id":"F320","status":"closed"},{"id":"I694","status":"ready"}]Checked answer{"ids": ["I694"]}wrong_schema_or_value
{
"ids": ["I694", "F320", "D374", "E220", "Z587"]
}38 tokensCorrect
{"ids":["I694"]}11 tokensCorrect
{"ids":["I694"]}11 tokensCorrect
{"ids":["I694"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"R472","units":5,"price":1},{"id":"K241","units":11,"price":19},{"id":"K808","units":17,"price":4},{"id":"V248","units":11,"price":15},{"id":"L605","units":18,"price":8}]Checked answer{"total": 591}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":468}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"K558","priority":7},{"id":"N431","priority":3},{"id":"Z469","priority":1},{"id":"I793","priority":6},{"id":"B280","priority":4}]Checked answer{"ids": ["K558", "I793"]}invalid_json_object
[{"id":"Z469","priority":1},{"id":"I793","priority":6},{"id":"K558","priority":7},{"id":"N431","priority":3},{"id":"B280","priority":4}]62 tokensCorrect
{"ids":["K558","I793"]}16 tokensCorrect
{"ids":["K558","I793"]}16 tokensCorrect
{"ids":["K558","I793"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to A440? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"T784","city":"Oslo","units":14},{"id":"A440","city":"Riga","units":14},{"id":"T777","city":"Riga","units":17},{"id":"L315","city":"Oslo","units":7},{"id":"F366","city":"Bern","units":0}]Checked answer{"city": "Riga", "units": 14}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Riga","units":14}12 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Riga","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"U312","status":"ready"},{"id":"U588","status":"pending"},{"id":"Z499","status":"ready"},{"id":"G802","status":"ready"},{"id":"D355","status":"ready"}]Checked answer{"ids": ["D355", "G802", "U312", "Z499"]}invalid_json_object
[{"id":"U312","status":"ready"},{"id":"U588","status":"pending"},{"id":"Z499","status":"ready"},{"id":"G802","status":"ready"},{"id":"D355","status":"ready"}]63 tokenswrong_schema_or_value
{"ids":["D355","G802","Z499"]}21 tokenswrong_schema_or_value
{"ids":["D355","G802","Z499","U312"]}26 tokensinvalid_json_object
["ids":["D355","G802","U312","Z499"]]26 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"U432","units":1,"price":18},{"id":"Y249","units":11,"price":6},{"id":"W647","units":3,"price":7},{"id":"T915","units":19,"price":4},{"id":"L192","units":10,"price":19}]Checked answer{"total": 371}wrong_schema_or_value
{"total": 108}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Z960","priority":6},{"id":"D369","priority":3},{"id":"K653","priority":4},{"id":"D196","priority":2},{"id":"G938","priority":1}]Checked answer{"ids": ["Z960", "K653"]}wrong_schema_or_value
{"ids": ["D369", "D196", "G938", "K653", "Z960"]}35 tokenswrong_schema_or_value
{"ids":["Z960","D369"]}16 tokenswrong_schema_or_value
{"ids":["Z960","D369"]}16 tokenswrong_schema_or_value
{"ids":["Z960","D369"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=P935. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"P935","city":"Oslo","units":6},{"id":"E763","city":"Suva","units":12},{"id":"B865","city":"Oslo","units":4},{"id":"N695","city":"Accra","units":11},{"id":"Y775","city":"Perth","units":4}]Checked answer{"city": "Oslo", "units": 6}Correct
{"city":"Oslo","units":6}11 tokensCorrect
{"city":"Oslo","units":6}11 tokensCorrect
{"city":"Oslo","units":6}11 tokensCorrect
{"city":"Oslo","units":6}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"R244","status":"ready"},{"id":"Z873","status":"closed"},{"id":"D106","status":"closed"},{"id":"F488","status":"pending"},{"id":"I716","status":"pending"}]Checked answer{"ids": ["R244"]}wrong_schema_or_value
{"ids": ["R244", "Z873", "D106", "F488", "I716"]}35 tokensCorrect
{"ids":["R244"]}11 tokensCorrect
{"ids":["R244"]}11 tokensCorrect
{"ids":["R244"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"H945","units":10,"price":6},{"id":"A393","units":5,"price":9},{"id":"B636","units":6,"price":15},{"id":"X841","units":1,"price":10},{"id":"P130","units":13,"price":5}]Checked answer{"total": 270}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":378}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"P148","priority":6},{"id":"N382","priority":9},{"id":"R876","priority":7},{"id":"V576","priority":6},{"id":"X810","priority":4}]Checked answer{"ids": ["N382", "R876"]}invalid_json_object
[{"id":"P148","priority":6},{"id":"R876","priority":7},{"id":"V576","priority":6},{"id":"N382","priority":9},{"id":"X810","priority":4}]62 tokenswrong_schema_or_value
{"ids":["N382","P148"]}16 tokenswrong_schema_or_value
{"ids":["N382","P148"]}16 tokenswrong_schema_or_value
{"ids":["N382","P148"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"C742","city":"Oslo","units":16},{"id":"N446","city":"Kyoto","units":7},{"id":"W293","city":"Perth","units":5},{"id":"V441","city":"Bern","units":16},{"id":"D421","city":"Perth","units":2}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{
"city": "Oslo",
"units": 16
}19 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Q643","status":"ready"},{"id":"Q999","status":"closed"},{"id":"T787","status":"closed"},{"id":"E210","status":"ready"},{"id":"Q613","status":"closed"}]Checked answer{"ids": ["E210", "Q643"]}wrong_schema_or_value
{"ids": ["E210", "Q613", "Q643", "Q999", "T787"]}35 tokensCorrect
{"ids":["E210","Q643"]}16 tokensCorrect
{"ids":["E210","Q643"]}16 tokensCorrect
{"ids":["E210","Q643"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"P191","units":6,"price":4},{"id":"K469","units":3,"price":12},{"id":"J450","units":8,"price":7},{"id":"U563","units":16,"price":1},{"id":"H854","units":8,"price":4}]Checked answer{"total": 164}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"J466","priority":9},{"id":"M925","priority":1},{"id":"V691","priority":1},{"id":"P937","priority":2},{"id":"U984","priority":9}]Checked answer{"ids": ["J466", "U984"]}wrong_schema_or_value
{"ids": ["M925", "V691", "P937", "J466"]}29 tokensCorrect
{"ids":["J466","U984"]}16 tokensCorrect
{"ids":["J466","U984"]}16 tokensCorrect
{"ids":["J466","U984"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id E354 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"E354","city":"Perth","units":16},{"id":"G537","city":"Perth","units":18},{"id":"P482","city":"Perth","units":2},{"id":"L504","city":"Oslo","units":4},{"id":"C505","city":"Oslo","units":8}]Checked answer{"city": "Perth", "units": 16}Correct
{"city":"Perth","units":16}12 tokensCorrect
{"city":"Perth","units":16}12 tokensCorrect
{"city":"Perth","units":16}12 tokensCorrect
{"city":"Perth","units":16}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"N531","status":"ready"},{"id":"K916","status":"ready"},{"id":"M185","status":"ready"},{"id":"E973","status":"closed"},{"id":"S307","status":"pending"}]Checked answer{"ids": ["K916", "M185", "N531"]}wrong_schema_or_value
{
"ids": ["M185", "N531", "K916", "E973"]
}32 tokensCorrect
{"ids":["K916","M185","N531"]}21 tokensCorrect
{"ids":["K916","M185","N531"]}21 tokensCorrect
{"ids":["K916","M185","N531"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"D106","units":3,"price":17},{"id":"J980","units":5,"price":12},{"id":"X994","units":19,"price":11},{"id":"R553","units":3,"price":18},{"id":"D350","units":0,"price":3}]Checked answer{"total": 374}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"H592","priority":4},{"id":"A621","priority":8},{"id":"H500","priority":4},{"id":"X433","priority":7},{"id":"Z157","priority":2}]Checked answer{"ids": ["A621", "X433"]}invalid_json_object
[{"id":"H592","priority":4},{"id":"A621","priority":8},{"id":"H500","priority":4},{"id":"X433","priority":7},{"id":"Z157","priority":2}]62 tokensCorrect
{"ids":["A621","X433"]}16 tokenswrong_schema_or_value
{"ids":["A621","H500"]}16 tokensCorrect
{"ids":["A621","X433"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to M657? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"M657","city":"Oslo","units":0},{"id":"U581","city":"Oslo","units":19},{"id":"C132","city":"Perth","units":9},{"id":"T227","city":"Accra","units":15},{"id":"D668","city":"Accra","units":9}]Checked answer{"city": "Oslo", "units": 0}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Oslo","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"G534","status":"closed"},{"id":"L840","status":"ready"},{"id":"U678","status":"closed"},{"id":"U340","status":"pending"},{"id":"U796","status":"pending"}]Checked answer{"ids": ["L840"]}invalid_json_object
[{"id":"U340","status":"pending"},{"id":"U796","status":"pending"},{"id":"U678","status":"closed"},{"id":"G534","status":"closed"},{"id":"L840","status":"ready"}]63 tokensinvalid_json_object
["L840"]7 tokens
Correct
{"ids":["L840"]}11 tokensinvalid_json_object
["ids":["L840"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Y499","units":12,"price":14},{"id":"Y999","units":2,"price":15},{"id":"V178","units":4,"price":11},{"id":"X247","units":10,"price":11},{"id":"W657","units":10,"price":2}]Checked answer{"total": 372}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"T891","priority":7},{"id":"X196","priority":5},{"id":"B412","priority":4},{"id":"P711","priority":7},{"id":"R570","priority":3}]Checked answer{"ids": ["P711", "T891"]}wrong_schema_or_value
{"ids": ["B412", "P711"]}17 tokensCorrect
{"ids":["P711","T891"]}16 tokensCorrect
{"ids":["P711","T891"]}16 tokensCorrect
{"ids":["P711","T891"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=S148. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"S333","city":"Perth","units":3},{"id":"S148","city":"Accra","units":4},{"id":"A270","city":"Suva","units":16},{"id":"F786","city":"Bern","units":14},{"id":"H695","city":"Kyoto","units":17}]Checked answer{"city": "Accra", "units": 4}wrong_schema_or_value
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Accra","units":4}11 tokensCorrect
{"city":"Accra","units":4}11 tokensCorrect
{"city":"Accra","units":4}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"K950","status":"ready"},{"id":"J742","status":"ready"},{"id":"F727","status":"pending"},{"id":"X206","status":"closed"},{"id":"Y995","status":"pending"}]Checked answer{"ids": ["J742", "K950"]}wrong_schema_or_value
{"ids": ["J742", "K950", "F727", "X206", "Y995"]}35 tokensCorrect
{"ids":["J742","K950"]}16 tokensCorrect
{"ids":["J742","K950"]}16 tokensCorrect
{"ids":["J742","K950"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"U107","units":12,"price":15},{"id":"E579","units":13,"price":7},{"id":"T222","units":3,"price":5},{"id":"T754","units":13,"price":2},{"id":"M178","units":19,"price":11}]Checked answer{"total": 521}wrong_schema_or_value
{
"total": 34
}11 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"W920","priority":7},{"id":"E454","priority":3},{"id":"A497","priority":2},{"id":"D822","priority":4},{"id":"L853","priority":5}]Checked answer{"ids": ["W920", "L853"]}invalid_json_object
[{"id":"W920","priority":7},{"id":"D822","priority":4}]26 tokensCorrect
{"ids":["W920","L853"]}16 tokensCorrect
{"ids":["W920","L853"]}16 tokenswrong_schema_or_value
{"ids":["W920","E454"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record D407. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"U812","city":"Riga","units":2},{"id":"E342","city":"Oslo","units":7},{"id":"P585","city":"Perth","units":11},{"id":"D407","city":"Perth","units":13},{"id":"E254","city":"Lima","units":6}]Checked answer{"city": "Perth", "units": 13}invalid_json_object
[{"city":"Riga","units":2},{"city":"Riga","units":2},{"city":"Riga","units":2},{"city":"Riga","units":2},{"city":"Riga","units":2},{"city":"Riga","units":2},{"city":"Riga","units":2},{"city":"Riga","units":2},{"city":"Riga","units":2},{"city":"Riga96 tokensCorrect
{"city":"Perth","units":13}12 tokensCorrect
{"city":"Perth","units":13}12 tokensCorrect
{"city":"Perth","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"W420","status":"ready"},{"id":"E646","status":"pending"},{"id":"J606","status":"closed"},{"id":"I223","status":"closed"},{"id":"X344","status":"pending"}]Checked answer{"ids": ["W420"]}wrong_schema_or_value
{"ids": ["E646", "J606", "I223", "W420", "X344"]}35 tokensCorrect
{"ids":["W420"]}11 tokensCorrect
{"ids":["W420"]}11 tokensCorrect
{"ids":["W420"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"P620","units":2,"price":7},{"id":"W684","units":0,"price":11},{"id":"N321","units":10,"price":9},{"id":"J365","units":10,"price":10},{"id":"F634","units":5,"price":14}]Checked answer{"total": 274}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"O555","priority":1},{"id":"Q200","priority":5},{"id":"L246","priority":5},{"id":"I498","priority":9},{"id":"J728","priority":5}]Checked answer{"ids": ["I498", "J728"]}wrong_schema_or_value
{"ids": ["O555", "J728", "I498", "L246", "Q200"]}35 tokensCorrect
{"ids":["I498","J728"]}16 tokensCorrect
{"ids":["I498","J728"]}16 tokenswrong_schema_or_value
{"ids":["I498","O555"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"X633","city":"Riga","units":8},{"id":"A839","city":"Kyoto","units":18},{"id":"U666","city":"Oslo","units":2},{"id":"F414","city":"Kyoto","units":13},{"id":"H403","city":"Kyoto","units":1}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Riga","units":8}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"V706","status":"closed"},{"id":"O206","status":"pending"},{"id":"P586","status":"ready"},{"id":"U244","status":"ready"},{"id":"O870","status":"pending"}]Checked answer{"ids": ["P586", "U244"]}wrong_schema_or_value
{
"ids": ["O206", "O870", "P586", "U244"]
}32 tokenswrong_schema_or_value
{"ids":["U244","P586"]}16 tokensCorrect
{"ids":["P586","U244"]}16 tokenswrong_schema_or_value
{"ids":["O206","U244"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"G308","units":0,"price":1},{"id":"D489","units":0,"price":1},{"id":"F751","units":4,"price":11},{"id":"E915","units":8,"price":6},{"id":"G508","units":15,"price":18}]Checked answer{"total": 362}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":288}8 tokenswrong_schema_or_value
{"total":288}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"O743","priority":2},{"id":"C949","priority":8},{"id":"Z794","priority":4},{"id":"R935","priority":3},{"id":"W113","priority":5}]Checked answer{"ids": ["C949", "W113"]}invalid_json_object
[{"id":"O743","priority":2},{"id":"Z794","priority":4},{"id":"R935","priority":3},{"id":"W113","priority":5}]50 tokensCorrect
{"ids":["C949","W113"]}16 tokenswrong_schema_or_value
{"ids":["C949","O743"]}16 tokenswrong_schema_or_value
{"ids":["C949","O743"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to A920? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"W629","city":"Kyoto","units":8},{"id":"Z843","city":"Lima","units":6},{"id":"K438","city":"Kyoto","units":6},{"id":"A920","city":"Riga","units":12},{"id":"B105","city":"Riga","units":3}]Checked answer{"city": "Riga", "units": 12}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Riga","units":12}12 tokensCorrect
{"city":"Riga","units":12}12 tokensCorrect
{"city":"Riga","units":12}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"Y237","status":"pending"},{"id":"M122","status":"ready"},{"id":"B202","status":"closed"},{"id":"Z859","status":"ready"},{"id":"S971","status":"ready"}]Checked answer{"ids": ["M122", "S971", "Z859"]}invalid_json_object
[{"id":"M122","status":"ready"},{"id":"S971","status":"ready"},{"id":"Y237","status":"pending"},{"id":"B202","status":"closed"},{"id":"Z859","status":"ready"}]63 tokensCorrect
{"ids":["M122","S971","Z859"]}21 tokensCorrect
{"ids":["M122","S971","Z859"]}21 tokensinvalid_json_object
["ids":["S971","M122","Z859"]]21 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"R164","units":15,"price":1},{"id":"L714","units":12,"price":17},{"id":"D820","units":4,"price":12},{"id":"N587","units":13,"price":13},{"id":"C991","units":10,"price":3}]Checked answer{"total": 466}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"X893","priority":7},{"id":"K348","priority":1},{"id":"B288","priority":5},{"id":"L747","priority":3},{"id":"P478","priority":6}]Checked answer{"ids": ["X893", "P478"]}wrong_schema_or_value
{"ids": ["X893", "K348", "B288", "L747", "P478"]}35 tokensCorrect
{"ids":["X893","P478"]}16 tokensCorrect
{"ids":["X893","P478"]}16 tokensCorrect
{"ids":["X893","P478"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=R481. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"R481","city":"Lima","units":0},{"id":"I244","city":"Lima","units":6},{"id":"P392","city":"Accra","units":2},{"id":"I950","city":"Riga","units":19},{"id":"D684","city":"Kyoto","units":17}]Checked answer{"city": "Lima", "units": 0}Correct
{"city": "Lima", "units": 0}14 tokensCorrect
{"city":"Lima","units":0}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Lima","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"A593","status":"pending"},{"id":"O556","status":"closed"},{"id":"T913","status":"closed"},{"id":"F901","status":"pending"},{"id":"F865","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["A593", "O556", "T913", "F865", "F901"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"J845","units":5,"price":4},{"id":"R908","units":15,"price":7},{"id":"P270","units":9,"price":10},{"id":"E540","units":11,"price":6},{"id":"I861","units":14,"price":8}]Checked answer{"total": 393}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"P366","priority":2},{"id":"R709","priority":2},{"id":"J387","priority":9},{"id":"H531","priority":8},{"id":"G873","priority":9}]Checked answer{"ids": ["G873", "J387"]}invalid_json_object
[{"id":"P366"},{"id":"R709"},{"id":"J387"},{"id":"H531"},{"id":"G873"}]43 tokensCorrect
{"ids":["G873","J387"]}16 tokensCorrect
{"ids":["G873","J387"]}16 tokenswrong_schema_or_value
{"ids":["J387","G873"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record P964. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"R986","city":"Lima","units":13},{"id":"W827","city":"Perth","units":6},{"id":"P964","city":"Bern","units":5},{"id":"W487","city":"Accra","units":4},{"id":"I698","city":"Oslo","units":14}]Checked answer{"city": "Bern", "units": 5}Correct
{
"city": "Bern",
"units": 5
}17 tokensCorrect
{"city":"Bern","units":5}10 tokensCorrect
{"city":"Bern","units":5}10 tokensCorrect
{"city":"Bern","units":5}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"T181","status":"ready"},{"id":"Y678","status":"pending"},{"id":"S726","status":"pending"},{"id":"X979","status":"ready"},{"id":"L400","status":"pending"}]Checked answer{"ids": ["T181", "X979"]}wrong_schema_or_value
{"ids": ["X979", "L400", "T181", "S726", "Y678"]}35 tokenswrong_schema_or_value
{"ids":["X979","T181"]}16 tokenswrong_schema_or_value
{"ids":["X979","T181"]}16 tokensCorrect
{"ids":["T181","X979"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"D270","units":14,"price":18},{"id":"Q506","units":4,"price":15},{"id":"F993","units":16,"price":1},{"id":"V671","units":4,"price":2},{"id":"N620","units":7,"price":6}]Checked answer{"total": 378}invalid_json_object
{"total": 14 * 18 + 4 * 15 + 16 * 1 + 4 * 2 + 7 * 6}38 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"I729","priority":9},{"id":"I163","priority":8},{"id":"V293","priority":7},{"id":"J521","priority":6},{"id":"M867","priority":4}]Checked answer{"ids": ["I729", "I163"]}wrong_schema_or_value
{"ids": ["I729", "V293", "J521", "M867"]}29 tokensCorrect
{"ids":["I729","I163"]}16 tokensCorrect
{"ids":["I729","I163"]}16 tokenswrong_schema_or_value
{"ids":["I729","J521"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id W390 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"G799","city":"Oslo","units":1},{"id":"W390","city":"Riga","units":9},{"id":"E411","city":"Oslo","units":7},{"id":"G131","city":"Kyoto","units":4},{"id":"H936","city":"Kyoto","units":17}]Checked answer{"city": "Riga", "units": 9}wrong_schema_or_value
{"city":"Oslo","units":1}11 tokensCorrect
{"city":"Riga","units":9}11 tokensCorrect
{"city":"Riga","units":9}11 tokensCorrect
{"city":"Riga","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"S488","status":"pending"},{"id":"D819","status":"closed"},{"id":"Q878","status":"pending"},{"id":"Z292","status":"closed"},{"id":"T692","status":"ready"}]Checked answer{"ids": ["T692"]}wrong_schema_or_value
{
"ids": ["S488", "D819", "Q878", "Z292", "T692"]
}38 tokensCorrect
{"ids":["T692"]}11 tokensCorrect
{"ids":["T692"]}11 tokensCorrect
{"ids":["T692"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"M384","units":19,"price":7},{"id":"V397","units":10,"price":8},{"id":"S506","units":18,"price":3},{"id":"D594","units":4,"price":9},{"id":"I654","units":15,"price":13}]Checked answer{"total": 498}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Q641","priority":2},{"id":"Z565","priority":9},{"id":"G318","priority":1},{"id":"C642","priority":6},{"id":"R529","priority":5}]Checked answer{"ids": ["Z565", "C642"]}invalid_json_object
[{"id":"Q641","priority":2},{"id":"G318","priority":1},{"id":"C642","priority":6},{"id":"R529","priority":5}]50 tokensCorrect
{"ids":["Z565","C642"]}16 tokensCorrect
{"ids":["Z565","C642"]}16 tokensCorrect
{"ids":["Z565","C642"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"H999","city":"Accra","units":15},{"id":"K245","city":"Suva","units":3},{"id":"N421","city":"Suva","units":5},{"id":"U189","city":"Suva","units":12},{"id":"E804","city":"Accra","units":4}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"E760","status":"pending"},{"id":"O785","status":"pending"},{"id":"Y629","status":"closed"},{"id":"X668","status":"closed"},{"id":"F238","status":"pending"}]Checked answer{"ids": []}invalid_json_object
[{"id":"E760"},{"id":"O785"},{"id":"Y629"},{"id":"X668"},{"id":"F238"}]43 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensinvalid_json_object
["ids":[]}6 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"E499","units":18,"price":11},{"id":"H866","units":10,"price":10},{"id":"R753","units":12,"price":7},{"id":"X724","units":8,"price":8},{"id":"K919","units":11,"price":16}]Checked answer{"total": 622}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"N996","priority":6},{"id":"S133","priority":7},{"id":"Q902","priority":2},{"id":"P736","priority":6},{"id":"G534","priority":6}]Checked answer{"ids": ["S133", "G534"]}wrong_schema_or_value
{"ids": ["N996", "S133", "Q902", "P736", "G534"]}35 tokenswrong_schema_or_value
{"ids":["S133","N996"]}16 tokenswrong_schema_or_value
{"ids":["S133","N996"]}16 tokenswrong_schema_or_value
{"ids":["S133","N996"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=G839. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"G427","city":"Riga","units":7},{"id":"X925","city":"Accra","units":1},{"id":"G839","city":"Perth","units":11},{"id":"T806","city":"Suva","units":12},{"id":"X627","city":"Accra","units":8}]Checked answer{"city": "Perth", "units": 11}wrong_schema_or_value
{"city":"Riga","units":7}11 tokensCorrect
{"city":"Perth","units":11}12 tokensCorrect
{"city":"Perth","units":11}12 tokensCorrect
{"city":"Perth","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"S433","status":"pending"},{"id":"J732","status":"pending"},{"id":"J195","status":"ready"},{"id":"I934","status":"ready"},{"id":"N862","status":"pending"}]Checked answer{"ids": ["I934", "J195"]}wrong_schema_or_value
{"ids": ["I934", "J195", "J732", "S433", "N862"]}35 tokensCorrect
{"ids":["I934","J195"]}16 tokensCorrect
{"ids":["I934","J195"]}16 tokensCorrect
{"ids":["I934","J195"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"I287","units":2,"price":19},{"id":"X896","units":7,"price":13},{"id":"T586","units":15,"price":11},{"id":"M528","units":5,"price":11},{"id":"C729","units":14,"price":12}]Checked answer{"total": 517}invalid_json_object
{
"total": 19 + 13 + 15 + 11 + 11 + 12 + 14 + 12 = 99
}43 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"I792","priority":6},{"id":"E896","priority":5},{"id":"R967","priority":8},{"id":"V467","priority":8},{"id":"N816","priority":8}]Checked answer{"ids": ["N816", "R967"]}invalid_json_object
[{"id":"I792","priority":6},{"id":"R967","priority":8},{"id":"V467","priority":8},{"id":"N816","priority":8}]50 tokensCorrect
{"ids":["N816","R967"]}16 tokenswrong_schema_or_value
{"ids":["R967","N816"]}16 tokenswrong_schema_or_value
{"ids":["N816","V467"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record A905. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"M563","city":"Lima","units":10},{"id":"M690","city":"Suva","units":8},{"id":"A905","city":"Accra","units":2},{"id":"W136","city":"Lima","units":0},{"id":"Z274","city":"Oslo","units":1}]Checked answer{"city": "Accra", "units": 2}Correct
{
"city": "Accra",
"units": 2
}18 tokensCorrect
{"city":"Accra","units":2}11 tokensCorrect
{"city":"Accra","units":2}11 tokensCorrect
{"city":"Accra","units":2}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"F639","status":"ready"},{"id":"G866","status":"closed"},{"id":"B330","status":"pending"},{"id":"K451","status":"pending"},{"id":"U434","status":"closed"}]Checked answer{"ids": ["F639"]}wrong_schema_or_value
{"ids": ["B330", "F639", "K451", "U434", "G866"]}35 tokensCorrect
{"ids":["F639"]}11 tokensCorrect
{"ids":["F639"]}11 tokensCorrect
{"ids":["F639"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"I494","units":6,"price":8},{"id":"X950","units":17,"price":7},{"id":"C626","units":16,"price":5},{"id":"Z161","units":0,"price":12},{"id":"V823","units":3,"price":16}]Checked answer{"total": 295}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Z669","priority":5},{"id":"L807","priority":6},{"id":"Y814","priority":9},{"id":"T130","priority":2},{"id":"F245","priority":3}]Checked answer{"ids": ["Y814", "L807"]}wrong_schema_or_value
{"ids": ["Y814", "T130", "F245", "L807", "Z669"]}35 tokensCorrect
{"ids":["Y814","L807"]}16 tokensCorrect
{"ids":["Y814","L807"]}16 tokensCorrect
{"ids":["Y814","L807"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id R645 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"P635","city":"Perth","units":0},{"id":"C818","city":"Riga","units":8},{"id":"K964","city":"Riga","units":7},{"id":"R645","city":"Oslo","units":15},{"id":"T234","city":"Bern","units":16}]Checked answer{"city": "Oslo", "units": 15}wrong_schema_or_value
{"city":"Perth","units":0}11 tokensCorrect
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Oslo","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"R607","status":"pending"},{"id":"S993","status":"pending"},{"id":"J367","status":"ready"},{"id":"Z504","status":"pending"},{"id":"I587","status":"closed"}]Checked answer{"ids": ["J367"]}wrong_schema_or_value
{
"ids": ["J367", "I587", "Z504"]
}26 tokensCorrect
{"ids":["J367"]}11 tokensCorrect
{"ids":["J367"]}11 tokensCorrect
{"ids":["J367"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"L278","units":10,"price":10},{"id":"O685","units":3,"price":1},{"id":"A943","units":7,"price":15},{"id":"I211","units":12,"price":6},{"id":"D399","units":12,"price":18}]Checked answer{"total": 496}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":408}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"J883","priority":2},{"id":"Z935","priority":9},{"id":"L587","priority":2},{"id":"B464","priority":5},{"id":"S671","priority":2}]Checked answer{"ids": ["Z935", "B464"]}invalid_json_object
[{"id":"J883","priority":2},{"id":"L587","priority":2},{"id":"B464","priority":5},{"id":"S671","priority":2}]50 tokensCorrect
{"ids":["Z935","B464"]}16 tokenswrong_schema_or_value
{"ids":["Z935","J883"]}16 tokenswrong_schema_or_value
{"ids":["Z935","J883"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to U618? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"U618","city":"Lima","units":8},{"id":"Z978","city":"Lima","units":1},{"id":"V466","city":"Riga","units":4},{"id":"J547","city":"Accra","units":14},{"id":"A846","city":"Riga","units":19}]Checked answer{"city": "Lima", "units": 8}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Lima","units":8}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Lima","units":8}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"O866","status":"ready"},{"id":"S494","status":"pending"},{"id":"O416","status":"pending"},{"id":"M512","status":"ready"},{"id":"C148","status":"closed"}]Checked answer{"ids": ["M512", "O866"]}invalid_json_object
[{"id":"O866","status":"ready"},{"id":"M512","status":"ready"},{"id":"O416","status":"pending"},{"id":"C148","status":"closed"},{"id":"S494","status":"pending"}]63 tokensCorrect
{"ids":["M512","O866"]}16 tokensCorrect
{"ids":["M512","O866"]}16 tokensinvalid_json_object
["ids":["O416","M512"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"V949","units":3,"price":11},{"id":"W558","units":19,"price":9},{"id":"M875","units":7,"price":18},{"id":"R336","units":8,"price":15},{"id":"D678","units":14,"price":10}]Checked answer{"total": 590}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"J210","priority":4},{"id":"C846","priority":1},{"id":"P743","priority":8},{"id":"Q105","priority":8},{"id":"C413","priority":6}]Checked answer{"ids": ["P743", "Q105"]}wrong_schema_or_value
{"ids": ["C846", "P743"]}17 tokensCorrect
{"ids":["P743","Q105"]}16 tokensCorrect
{"ids":["P743","Q105"]}16 tokenswrong_schema_or_value
{"ids":["Q105","J210"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"W934","city":"Riga","units":13},{"id":"C677","city":"Perth","units":11},{"id":"V237","city":"Perth","units":7},{"id":"U298","city":"Oslo","units":15},{"id":"E139","city":"Lima","units":17}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Riga","units":13}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"T693","status":"closed"},{"id":"K290","status":"closed"},{"id":"D761","status":"closed"},{"id":"M641","status":"closed"},{"id":"Q297","status":"ready"}]Checked answer{"ids": ["Q297"]}wrong_schema_or_value
{"ids": ["D761", "M641", "Q297", "K290", "T693"]}35 tokensCorrect
{"ids":["Q297"]}11 tokensCorrect
{"ids":["Q297"]}11 tokensCorrect
{"ids":["Q297"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"E244","units":7,"price":5},{"id":"F901","units":17,"price":12},{"id":"W547","units":19,"price":2},{"id":"I703","units":1,"price":2},{"id":"S573","units":19,"price":16}]Checked answer{"total": 583}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"K577","priority":6},{"id":"T863","priority":3},{"id":"Y233","priority":4},{"id":"M216","priority":5},{"id":"B297","priority":8}]Checked answer{"ids": ["B297", "K577"]}invalid_json_object
[{"id":"K577","priority":6},{"id":"T863","priority":3},{"id":"Y233","priority":4},{"id":"M216","priority":5},{"id":"B297","priority":8}]62 tokensCorrect
{"ids":["B297","K577"]}16 tokensCorrect
{"ids":["B297","K577"]}16 tokensCorrect
{"ids":["B297","K577"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record N797. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"D304","city":"Oslo","units":4},{"id":"B448","city":"Perth","units":4},{"id":"N797","city":"Lima","units":18},{"id":"L305","city":"Accra","units":14},{"id":"Z536","city":"Kyoto","units":11}]Checked answer{"city": "Lima", "units": 18}Correct
{
"city": "Lima",
"units": 18
}19 tokensCorrect
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"P687","status":"pending"},{"id":"Y234","status":"closed"},{"id":"D730","status":"closed"},{"id":"A876","status":"ready"},{"id":"G343","status":"ready"}]Checked answer{"ids": ["A876", "G343"]}wrong_schema_or_value
{"ids": ["A876", "G343", "P687", "Y234", "D730"]}35 tokensCorrect
{"ids":["A876","G343"]}16 tokensCorrect
{"ids":["A876","G343"]}16 tokensCorrect
{"ids":["A876","G343"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"C884","units":11,"price":18},{"id":"O984","units":1,"price":17},{"id":"G817","units":14,"price":10},{"id":"V868","units":7,"price":9},{"id":"T203","units":15,"price":3}]Checked answer{"total": 463}invalid_json_object
{"total": 11*18 + 1*17 + 14*10 + 7*9 + 15*3 = 205}40 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"P714","priority":3},{"id":"D308","priority":2},{"id":"F792","priority":7},{"id":"A742","priority":1},{"id":"R439","priority":2}]Checked answer{"ids": ["F792", "P714"]}wrong_schema_or_value
{"ids": ["A742", "D308", "F792", "P714", "R439"]}35 tokensCorrect
{"ids":["F792","P714"]}16 tokensCorrect
{"ids":["F792","P714"]}16 tokenswrong_schema_or_value
{"ids":["F792","D308"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id H848 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"X769","city":"Bern","units":6},{"id":"U126","city":"Riga","units":16},{"id":"Z702","city":"Riga","units":6},{"id":"H848","city":"Accra","units":12},{"id":"M985","city":"Oslo","units":15}]Checked answer{"city": "Accra", "units": 12}Correct
{"city":"Accra","units":12}12 tokensCorrect
{"city":"Accra","units":12}12 tokensCorrect
{"city":"Accra","units":12}12 tokensCorrect
{"city":"Accra","units":12}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"U701","status":"ready"},{"id":"X827","status":"ready"},{"id":"J562","status":"closed"},{"id":"K187","status":"closed"},{"id":"V924","status":"closed"}]Checked answer{"ids": ["U701", "X827"]}wrong_schema_or_value
{
"ids": ["U701", "X827", "J562", "K187", "V924"]
}38 tokenswrong_schema_or_value
{"ids":["X827","U701"]}16 tokenswrong_schema_or_value
{"ids":["X827","U701"]}16 tokensCorrect
{"ids":["U701","X827"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"S119","units":8,"price":4},{"id":"G428","units":10,"price":12},{"id":"D831","units":8,"price":16},{"id":"P487","units":6,"price":9},{"id":"I161","units":13,"price":4}]Checked answer{"total": 386}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"C329","priority":7},{"id":"W937","priority":8},{"id":"R583","priority":2},{"id":"X979","priority":6},{"id":"W693","priority":8}]Checked answer{"ids": ["W693", "W937"]}invalid_json_object
[{"id":"C329","priority":7},{"id":"W937","priority":8},{"id":"R583","priority":2},{"id":"X979","priority":6},{"id":"W693","priority":8}]62 tokenswrong_schema_or_value
{"ids":["C329","W693"]}16 tokenswrong_schema_or_value
{"ids":["W937","C329"]}16 tokenswrong_schema_or_value
{"ids":["C329","W693"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to M543? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"G116","city":"Perth","units":10},{"id":"M543","city":"Riga","units":10},{"id":"C871","city":"Kyoto","units":6},{"id":"V746","city":"Oslo","units":2},{"id":"B577","city":"Suva","units":15}]Checked answer{"city": "Riga", "units": 10}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Riga","units":10}12 tokensCorrect
{"city":"Riga","units":10}12 tokensCorrect
{"city":"Riga","units":10}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"U836","status":"closed"},{"id":"F268","status":"pending"},{"id":"U538","status":"pending"},{"id":"L469","status":"ready"},{"id":"M598","status":"pending"}]Checked answer{"ids": ["L469"]}invalid_json_object
[{"id":"L469","status":"ready"},{"id":"M598","status":"pending"}]27 tokensinvalid_json_object
["L469"]7 tokens
Correct
{"ids":["L469"]}11 tokensinvalid_json_object
["ids":["L469"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"W256","units":0,"price":8},{"id":"C508","units":10,"price":4},{"id":"V180","units":16,"price":19},{"id":"H717","units":17,"price":2},{"id":"O946","units":2,"price":8}]Checked answer{"total": 394}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"J905","priority":9},{"id":"K897","priority":6},{"id":"O537","priority":3},{"id":"W885","priority":4},{"id":"Y587","priority":5}]Checked answer{"ids": ["J905", "K897"]}invalid_json_object
{"ids": ["J905", "O537", "W885", "Y587", "K897", "O537", "W885", "Y587", "K897", "O537", "W885", "Y587", "K897", "O537", "W885", "Y596 tokensCorrect
{"ids":["J905","K897"]}16 tokensCorrect
{"ids":["J905","K897"]}16 tokenswrong_schema_or_value
{"ids":["J905","O537"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=F848. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"O560","city":"Bern","units":1},{"id":"J595","city":"Oslo","units":7},{"id":"F848","city":"Kyoto","units":6},{"id":"V257","city":"Kyoto","units":10},{"id":"D573","city":"Riga","units":16}]Checked answer{"city": "Kyoto", "units": 6}Correct
{"city":"Kyoto","units":6}12 tokensCorrect
{"city":"Kyoto","units":6}12 tokensCorrect
{"city":"Kyoto","units":6}12 tokensCorrect
{"city":"Kyoto","units":6}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"L292","status":"ready"},{"id":"S729","status":"ready"},{"id":"T428","status":"ready"},{"id":"B325","status":"ready"},{"id":"X263","status":"ready"}]Checked answer{"ids": ["B325", "L292", "S729", "T428", "X263"]}Correct
{"ids": ["B325", "L292", "S729", "T428", "X263"]}35 tokenswrong_schema_or_value
{"ids":["B325","C263","L292","S729","T428"]}31 tokenswrong_schema_or_value
{"ids":["B325","C263","D428","E292","L292","S729","T428"]}41 tokenswrong_schema_or_value
{"ids":["L292","S729","B325","X263"]}26 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"A840","units":6,"price":15},{"id":"F750","units":18,"price":17},{"id":"A341","units":7,"price":17},{"id":"U867","units":6,"price":7},{"id":"I725","units":3,"price":12}]Checked answer{"total": 593}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"O398","priority":6},{"id":"B692","priority":1},{"id":"P642","priority":4},{"id":"B697","priority":6},{"id":"U173","priority":2}]Checked answer{"ids": ["B697", "O398"]}invalid_json_object
[{"id":"B697","priority":6},{"id":"U173","priority":2},{"id":"B692","priority":1},{"id":"O398","priority":6},{"id":"P642","priority":4}]62 tokenswrong_schema_or_value
{"ids":["O398","B697"]}16 tokenswrong_schema_or_value
{"ids":["O398","B697"]}16 tokenswrong_schema_or_value
{"ids":["B692","O398"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"B609","city":"Suva","units":8},{"id":"U982","city":"Riga","units":11},{"id":"R450","city":"Kyoto","units":14},{"id":"L439","city":"Perth","units":1},{"id":"N388","city":"Lima","units":7}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Suva","units":8},{"city":"Riga","units":11},{"city":"Kyoto","units":14},{"city":"Perth","units":1},{"city":"Lima","units":7}]55 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"L340","status":"pending"},{"id":"M545","status":"ready"},{"id":"N735","status":"closed"},{"id":"J766","status":"closed"},{"id":"X754","status":"pending"}]Checked answer{"ids": ["M545"]}wrong_schema_or_value
{"ids": ["M545", "J766", "N735", "L340"]}29 tokensCorrect
{"ids":["M545"]}11 tokensCorrect
{"ids":["M545"]}11 tokensCorrect
{"ids":["M545"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Q852","units":15,"price":12},{"id":"F422","units":11,"price":4},{"id":"Y359","units":8,"price":4},{"id":"M276","units":4,"price":10},{"id":"Q567","units":5,"price":17}]Checked answer{"total": 381}wrong_schema_or_value
{"total": 120}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"W542","priority":7},{"id":"F206","priority":1},{"id":"Q111","priority":1},{"id":"A217","priority":2},{"id":"M477","priority":7}]Checked answer{"ids": ["M477", "W542"]}wrong_schema_or_value
{"ids": ["F206", "W542", "M477", "A217", "Q111"]}35 tokensCorrect
{"ids":["M477","W542"]}16 tokensCorrect
{"ids":["M477","W542"]}16 tokensCorrect
{"ids":["M477","W542"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id I848 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"U430","city":"Riga","units":11},{"id":"B813","city":"Suva","units":13},{"id":"I759","city":"Oslo","units":4},{"id":"I848","city":"Kyoto","units":9},{"id":"F575","city":"Suva","units":0}]Checked answer{"city": "Kyoto", "units": 9}wrong_schema_or_value
{"city":"Riga","units":11}12 tokensCorrect
{"city":"Kyoto","units":9}12 tokensCorrect
{"city":"Kyoto","units":9}12 tokensCorrect
{"city":"Kyoto","units":9}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"M673","status":"ready"},{"id":"P305","status":"closed"},{"id":"G899","status":"closed"},{"id":"C675","status":"pending"},{"id":"R532","status":"ready"}]Checked answer{"ids": ["M673", "R532"]}wrong_schema_or_value
{
"ids": ["C675", "G899", "M673", "P305", "R532"]
}38 tokensCorrect
{"ids":["M673","R532"]}16 tokensCorrect
{"ids":["M673","R532"]}16 tokensCorrect
{"ids":["M673","R532"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"U417","units":9,"price":18},{"id":"W239","units":14,"price":1},{"id":"W308","units":3,"price":12},{"id":"L755","units":1,"price":7},{"id":"E761","units":0,"price":1}]Checked answer{"total": 219}wrong_schema_or_value
{"total": 33}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"L725","priority":9},{"id":"C490","priority":6},{"id":"M510","priority":6},{"id":"K673","priority":9},{"id":"U930","priority":2}]Checked answer{"ids": ["K673", "L725"]}invalid_json_object
[{"id":"L725","priority":9},{"id":"U930","priority":2}]26 tokenswrong_schema_or_value
{"ids":["L725","K673"]}16 tokenswrong_schema_or_value
{"ids":["L725","K673"]}16 tokensCorrect
{"ids":["K673","L725"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to B637? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"L920","city":"Suva","units":16},{"id":"B637","city":"Suva","units":8},{"id":"L960","city":"Kyoto","units":17},{"id":"D554","city":"Oslo","units":18},{"id":"W820","city":"Perth","units":19}]Checked answer{"city": "Suva", "units": 8}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Suva","units":8}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Suva","units":8}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"U175","status":"ready"},{"id":"Y918","status":"closed"},{"id":"H722","status":"closed"},{"id":"L989","status":"closed"},{"id":"M395","status":"closed"}]Checked answer{"ids": ["U175"]}invalid_json_object
[{"id":"U175"},{"id":"Y918"},{"id":"H722"},{"id":"L989"},{"id":"M395"}]43 tokensCorrect
{"ids":["U175"]}11 tokensCorrect
{"ids":["U175"]}11 tokensinvalid_json_object
["ids":["U175"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"C506","units":0,"price":5},{"id":"E333","units":3,"price":2},{"id":"X365","units":11,"price":8},{"id":"K581","units":16,"price":11},{"id":"Y505","units":11,"price":14}]Checked answer{"total": 424}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":388}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Q359","priority":6},{"id":"R927","priority":5},{"id":"T501","priority":4},{"id":"G244","priority":6},{"id":"U908","priority":6}]Checked answer{"ids": ["G244", "Q359"]}wrong_schema_or_value
{"ids": ["U908", "G244", "T501", "R927", "Q359"]}35 tokensCorrect
{"ids":["G244","Q359"]}16 tokensCorrect
{"ids":["G244","Q359"]}16 tokenswrong_schema_or_value
{"ids":["Q359","U908"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=E555. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"Z602","city":"Oslo","units":17},{"id":"O913","city":"Perth","units":1},{"id":"H735","city":"Suva","units":17},{"id":"L304","city":"Suva","units":14},{"id":"E555","city":"Lima","units":5}]Checked answer{"city": "Lima", "units": 5}Correct
{"city":"Lima","units":5}11 tokensCorrect
{"city":"Lima","units":5}11 tokensCorrect
{"city":"Lima","units":5}11 tokensCorrect
{"city":"Lima","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"B833","status":"pending"},{"id":"S237","status":"pending"},{"id":"P767","status":"ready"},{"id":"N199","status":"closed"},{"id":"V732","status":"closed"}]Checked answer{"ids": ["P767"]}wrong_schema_or_value
{"ids": ["B833", "S237", "P767", "N199", "V732"]}35 tokensCorrect
{"ids":["P767"]}11 tokensCorrect
{"ids":["P767"]}11 tokensCorrect
{"ids":["P767"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"O278","units":18,"price":11},{"id":"Y100","units":18,"price":3},{"id":"M532","units":18,"price":11},{"id":"A617","units":6,"price":2},{"id":"W817","units":18,"price":5}]Checked answer{"total": 552}invalid_json_object
{
"total": 11 + 3 + 11 + 2 + 5 = 32
}28 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"J601","priority":1},{"id":"D727","priority":9},{"id":"C433","priority":2},{"id":"V546","priority":6},{"id":"P266","priority":8}]Checked answer{"ids": ["D727", "P266"]}invalid_json_object
[{"id":"J601"},{"id":"D727"},{"id":"C433"},{"id":"V546"},{"id":"P266"}]43 tokensCorrect
{"ids":["D727","P266"]}16 tokensCorrect
{"ids":["D727","P266"]}16 tokenswrong_schema_or_value
{"ids":["D727","J601"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record I868. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"N957","city":"Kyoto","units":14},{"id":"L926","city":"Accra","units":19},{"id":"V322","city":"Kyoto","units":6},{"id":"X278","city":"Oslo","units":8},{"id":"I868","city":"Perth","units":15}]Checked answer{"city": "Perth", "units": 15}invalid_json_object
[{"city":"Perth","units":15}]13 tokensCorrect
{"city":"Perth","units":15}12 tokensCorrect
{"city":"Perth","units":15}12 tokensCorrect
{"city":"Perth","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Z489","status":"closed"},{"id":"W356","status":"closed"},{"id":"U440","status":"pending"},{"id":"F246","status":"pending"},{"id":"C160","status":"ready"}]Checked answer{"ids": ["C160"]}wrong_schema_or_value
{"ids": ["C160", "F246", "U440", "Z489", "W356"]}35 tokensCorrect
{"ids":["C160"]}11 tokensCorrect
{"ids":["C160"]}11 tokensCorrect
{"ids":["C160"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"L570","units":1,"price":18},{"id":"G999","units":17,"price":17},{"id":"D963","units":5,"price":6},{"id":"Q722","units":19,"price":9},{"id":"H495","units":19,"price":18}]Checked answer{"total": 850}invalid_json_object
{"total": 18 + 17 + 6 + 9 + 18}22 tokenswrong_schema_or_value
{"total":548}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"F443","priority":8},{"id":"T191","priority":1},{"id":"H712","priority":6},{"id":"H131","priority":6},{"id":"Z714","priority":2}]Checked answer{"ids": ["F443", "H131"]}wrong_schema_or_value
{"ids": ["F443", "T191", "H712", "H131", "Z714"]}35 tokensCorrect
{"ids":["F443","H131"]}16 tokenswrong_schema_or_value
{"ids":["F443","H712"]}16 tokenswrong_schema_or_value
{"ids":["F443","H712"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"X483","city":"Bern","units":5},{"id":"O307","city":"Bern","units":6},{"id":"N626","city":"Bern","units":6},{"id":"P818","city":"Perth","units":6},{"id":"E440","city":"Suva","units":17}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Bern","units":5}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"T233","status":"pending"},{"id":"K306","status":"ready"},{"id":"N661","status":"ready"},{"id":"H891","status":"closed"},{"id":"C637","status":"ready"}]Checked answer{"ids": ["C637", "K306", "N661"]}wrong_schema_or_value
{
"ids": ["C637", "H891", "K306", "N661", "T233"]
}38 tokensCorrect
{"ids":["C637","K306","N661"]}21 tokensCorrect
{"ids":["C637","K306","N661"]}21 tokenswrong_schema_or_value
{"ids":["K306","C637","N661"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"L907","units":1,"price":7},{"id":"L157","units":17,"price":18},{"id":"B441","units":7,"price":13},{"id":"G593","units":14,"price":15},{"id":"A613","units":17,"price":2}]Checked answer{"total": 648}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"U779","priority":3},{"id":"O557","priority":3},{"id":"Q376","priority":2},{"id":"B625","priority":6},{"id":"O843","priority":2}]Checked answer{"ids": ["B625", "O557"]}invalid_json_object
[{"id":"O843","priority":2},{"id":"O557","priority":3},{"id":"Q376","priority":2},{"id":"U779","priority":3},{"id":"B625","priority":6}]62 tokensCorrect
{"ids":["B625","O557"]}16 tokensCorrect
{"ids":["B625","O557"]}16 tokensCorrect
{"ids":["B625","O557"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Y105? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"S355","city":"Accra","units":6},{"id":"F861","city":"Bern","units":3},{"id":"G445","city":"Accra","units":14},{"id":"Y105","city":"Suva","units":4},{"id":"D521","city":"Accra","units":13}]Checked answer{"city": "Suva", "units": 4}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Suva","units":4}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Suva","units":4}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"P862","status":"ready"},{"id":"C536","status":"closed"},{"id":"A526","status":"ready"},{"id":"O723","status":"ready"},{"id":"X748","status":"ready"}]Checked answer{"ids": ["A526", "O723", "P862", "X748"]}invalid_json_object
[{"id":"P862","status":"ready"},{"id":"C536","status":"closed"},{"id":"A526","status":"ready"},{"id":"O723","status":"ready"},{"id":"X748","status":"ready"}]63 tokenswrong_schema_or_value
{"ids":["A526","O723","X748"]}21 tokensCorrect
{"ids":["A526","O723","P862","X748"]}26 tokensinvalid_json_object
["ids":["A526","O723","X748"]]21 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Z342","units":4,"price":5},{"id":"G555","units":4,"price":19},{"id":"L209","units":4,"price":16},{"id":"E979","units":7,"price":16},{"id":"N295","units":11,"price":2}]Checked answer{"total": 294}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"T400","priority":3},{"id":"X436","priority":2},{"id":"Q756","priority":5},{"id":"M666","priority":5},{"id":"M885","priority":3}]Checked answer{"ids": ["M666", "Q756"]}wrong_schema_or_value
{"ids": ["T400", "X436", "Q756", "M666", "M885"]}35 tokensCorrect
{"ids":["M666","Q756"]}16 tokensCorrect
{"ids":["M666","Q756"]}16 tokenswrong_schema_or_value
{"ids":["Q756","M666"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=A289. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"L344","city":"Perth","units":6},{"id":"B391","city":"Accra","units":16},{"id":"J804","city":"Riga","units":9},{"id":"H444","city":"Lima","units":4},{"id":"A289","city":"Riga","units":13}]Checked answer{"city": "Riga", "units": 13}Correct
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"R473","status":"pending"},{"id":"V480","status":"ready"},{"id":"L757","status":"closed"},{"id":"L778","status":"closed"},{"id":"Z316","status":"ready"}]Checked answer{"ids": ["V480", "Z316"]}wrong_schema_or_value
{"ids": ["L757", "L778", "R473", "V480", "Z316"]}35 tokenswrong_schema_or_value
{"ids":["Z316","V480"]}16 tokenswrong_schema_or_value
{"ids":["Z316","V480"]}16 tokenswrong_schema_or_value
{"ids":["Z316","V480"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"V957","units":13,"price":17},{"id":"H804","units":18,"price":17},{"id":"A733","units":5,"price":19},{"id":"G656","units":19,"price":7},{"id":"N873","units":4,"price":8}]Checked answer{"total": 787}invalid_json_object
{
"total": 17 + 17 + 19 + 7 + 8 = 69
}29 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"M997","priority":2},{"id":"M788","priority":2},{"id":"K757","priority":2},{"id":"W612","priority":5},{"id":"A738","priority":5}]Checked answer{"ids": ["A738", "W612"]}invalid_json_object
[{"id":"M997"},{"id":"M788"},{"id":"K757"},{"id":"W612"},{"id":"A738"}]43 tokensCorrect
{"ids":["A738","W612"]}16 tokensCorrect
{"ids":["A738","W612"]}16 tokensCorrect
{"ids":["A738","W612"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Y827. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"M253","city":"Oslo","units":8},{"id":"X440","city":"Suva","units":12},{"id":"Y827","city":"Perth","units":5},{"id":"E131","city":"Kyoto","units":4},{"id":"A605","city":"Bern","units":8}]Checked answer{"city": "Perth", "units": 5}invalid_json_object
[{"city":"Perth","units":5}]12 tokensCorrect
{"city":"Perth","units":5}11 tokensCorrect
{"city":"Perth","units":5}11 tokensCorrect
{"city":"Perth","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"R771","status":"ready"},{"id":"N536","status":"pending"},{"id":"Z420","status":"ready"},{"id":"Z484","status":"closed"},{"id":"Q279","status":"pending"}]Checked answer{"ids": ["R771", "Z420"]}wrong_schema_or_value
{"ids": ["R771", "Z420", "Z484", "Q279"]}29 tokensCorrect
{"ids":["R771","Z420"]}16 tokensCorrect
{"ids":["R771","Z420"]}16 tokensCorrect
{"ids":["R771","Z420"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"B169","units":16,"price":3},{"id":"Z173","units":6,"price":9},{"id":"Z884","units":10,"price":11},{"id":"I422","units":6,"price":14},{"id":"Y798","units":0,"price":14}]Checked answer{"total": 296}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"M244","priority":4},{"id":"K197","priority":6},{"id":"S745","priority":2},{"id":"I929","priority":8},{"id":"R999","priority":2}]Checked answer{"ids": ["I929", "K197"]}wrong_schema_or_value
{"ids": ["M244", "S745"]}17 tokensCorrect
{"ids":["I929","K197"]}16 tokensCorrect
{"ids":["I929","K197"]}16 tokensCorrect
{"ids":["I929","K197"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id V454 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"S150","city":"Bern","units":11},{"id":"R801","city":"Suva","units":15},{"id":"V454","city":"Suva","units":11},{"id":"C216","city":"Perth","units":0},{"id":"T333","city":"Riga","units":17}]Checked answer{"city": "Suva", "units": 11}Correct
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Suva","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"V206","status":"ready"},{"id":"W301","status":"closed"},{"id":"J272","status":"ready"},{"id":"M102","status":"pending"},{"id":"J699","status":"ready"}]Checked answer{"ids": ["J272", "J699", "V206"]}Correct
{
"ids": ["J272", "J699", "V206"]
}26 tokensCorrect
{"ids":["J272","J699","V206"]}21 tokensCorrect
{"ids":["J272","J699","V206"]}21 tokenswrong_schema_or_value
{"ids":["V206","J699","J272"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"E152","units":2,"price":1},{"id":"B848","units":1,"price":6},{"id":"T666","units":14,"price":10},{"id":"D610","units":8,"price":15},{"id":"U524","units":8,"price":12}]Checked answer{"total": 364}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"E181","priority":2},{"id":"R345","priority":1},{"id":"F913","priority":1},{"id":"W368","priority":3},{"id":"C328","priority":4}]Checked answer{"ids": ["C328", "W368"]}invalid_json_object
[{"id":"E181","priority":2},{"id":"R345","priority":1},{"id":"F913","priority":1},{"id":"W368","priority":3},{"id":"C328","priority":4}]62 tokensCorrect
{"ids":["C328","W368"]}16 tokensCorrect
{"ids":["C328","W368"]}16 tokensCorrect
{"ids":["C328","W368"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"Y128","city":"Riga","units":15},{"id":"G389","city":"Perth","units":4},{"id":"A862","city":"Oslo","units":10},{"id":"Z463","city":"Lima","units":6},{"id":"K464","city":"Lima","units":1}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"R920","status":"pending"},{"id":"X995","status":"closed"},{"id":"P739","status":"ready"},{"id":"I394","status":"closed"},{"id":"F124","status":"pending"}]Checked answer{"ids": ["P739"]}invalid_json_object
[{"id":"P739"}]11 tokensinvalid_json_object
["P739"]7 tokens
Correct
{"ids":["P739"]}11 tokensinvalid_json_object
["ids":["P739"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"M738","units":0,"price":9},{"id":"E494","units":0,"price":18},{"id":"Z990","units":6,"price":5},{"id":"Y155","units":19,"price":18},{"id":"A414","units":2,"price":3}]Checked answer{"total": 378}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"O880","priority":3},{"id":"Q931","priority":5},{"id":"U200","priority":2},{"id":"C115","priority":5},{"id":"I146","priority":3}]Checked answer{"ids": ["C115", "Q931"]}wrong_schema_or_value
{"ids": ["U200", "C115", "I146", "O880", "Q931"]}35 tokensCorrect
{"ids":["C115","Q931"]}16 tokensCorrect
{"ids":["C115","Q931"]}16 tokenswrong_schema_or_value
{"ids":["Q931","C115"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=B808. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"E984","city":"Bern","units":17},{"id":"B808","city":"Suva","units":9},{"id":"M979","city":"Accra","units":14},{"id":"S842","city":"Accra","units":19},{"id":"T168","city":"Perth","units":17}]Checked answer{"city": "Suva", "units": 9}Correct
{"city": "Suva", "units": 9}14 tokensCorrect
{"city":"Suva","units":9}11 tokensCorrect
{"city":"Suva","units":9}11 tokensCorrect
{"city":"Suva","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"D767","status":"ready"},{"id":"B564","status":"closed"},{"id":"J141","status":"pending"},{"id":"X985","status":"closed"},{"id":"P737","status":"pending"}]Checked answer{"ids": ["D767"]}wrong_schema_or_value
{"ids": ["B564", "D767", "J141", "P737", "X985"]}35 tokensCorrect
{"ids":["D767"]}11 tokensCorrect
{"ids":["D767"]}11 tokensCorrect
{"ids":["D767"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"X736","units":8,"price":14},{"id":"N214","units":12,"price":11},{"id":"H295","units":1,"price":5},{"id":"K870","units":14,"price":10},{"id":"N519","units":17,"price":14}]Checked answer{"total": 627}invalid_json_object
{
"total": 14 + 11 + 5 + 10 + 14 + 14 + 17 + 17 = 89
}42 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"W682","priority":4},{"id":"M261","priority":4},{"id":"D575","priority":2},{"id":"H783","priority":7},{"id":"J236","priority":2}]Checked answer{"ids": ["H783", "M261"]}invalid_json_object
[{"id":"W682"},{"id":"M261"},{"id":"D575"},{"id":"H783"},{"id":"J236"}]43 tokenswrong_schema_or_value
{"ids":["H783","J236"]}16 tokenswrong_schema_or_value
{"ids":["H783","W682"]}16 tokensCorrect
{"ids":["H783","M261"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record J273. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"O400","city":"Suva","units":13},{"id":"C335","city":"Kyoto","units":13},{"id":"C272","city":"Kyoto","units":14},{"id":"J273","city":"Perth","units":10},{"id":"M898","city":"Bern","units":7}]Checked answer{"city": "Perth", "units": 10}Correct
{
"city": "Perth",
"units": 10
}19 tokensCorrect
{"city":"Perth","units":10}12 tokensCorrect
{"city":"Perth","units":10}12 tokensCorrect
{"city":"Perth","units":10}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"A312","status":"closed"},{"id":"O807","status":"ready"},{"id":"J260","status":"pending"},{"id":"D356","status":"closed"},{"id":"U330","status":"ready"}]Checked answer{"ids": ["O807", "U330"]}wrong_schema_or_value
{"ids": ["A312", "O807", "J260", "D356", "U330"]}35 tokenswrong_schema_or_value
{"ids":["A312","U330","O807"]}21 tokenswrong_schema_or_value
{"ids":["D356","U330"]}16 tokenswrong_schema_or_value
{"ids":["O807","A312"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Y196","units":8,"price":19},{"id":"C160","units":1,"price":4},{"id":"S222","units":9,"price":9},{"id":"D580","units":4,"price":13},{"id":"W468","units":0,"price":17}]Checked answer{"total": 289}invalid_json_object
{"total": 19 + 4 + 9 + 13 + 17 = 64}26 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Z300","priority":2},{"id":"G690","priority":6},{"id":"I934","priority":8},{"id":"N438","priority":9},{"id":"L129","priority":6}]Checked answer{"ids": ["N438", "I934"]}wrong_schema_or_value
{"ids": ["G690", "I934", "L129", "Z300"]}29 tokenswrong_schema_or_value
{"ids":["I934","N438"]}16 tokenswrong_schema_or_value
{"ids":["I934","N438"]}16 tokensCorrect
{"ids":["N438","I934"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z638 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"W432","city":"Suva","units":2},{"id":"J458","city":"Kyoto","units":17},{"id":"P335","city":"Lima","units":9},{"id":"Z638","city":"Oslo","units":4},{"id":"S293","city":"Bern","units":0}]Checked answer{"city": "Oslo", "units": 4}Correct
{"city":"Oslo","units":4}11 tokensCorrect
{"city":"Oslo","units":4}11 tokensCorrect
{"city":"Oslo","units":4}11 tokensCorrect
{"city":"Oslo","units":4}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"T118","status":"ready"},{"id":"Z654","status":"closed"},{"id":"R303","status":"pending"},{"id":"X700","status":"ready"},{"id":"E107","status":"pending"}]Checked answer{"ids": ["T118", "X700"]}wrong_schema_or_value
{
"ids": ["R303", "X700", "T118", "Z654"]
}32 tokenswrong_schema_or_value
{"ids":["X700","T118"]}16 tokenswrong_schema_or_value
{"ids":["E107","T118","X700"]}21 tokensCorrect
{"ids":["T118","X700"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"S737","units":9,"price":6},{"id":"L560","units":13,"price":11},{"id":"Q832","units":15,"price":18},{"id":"I256","units":19,"price":4},{"id":"H727","units":17,"price":6}]Checked answer{"total": 645}wrong_schema_or_value
{"total": 230}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"M329","priority":7},{"id":"Y570","priority":4},{"id":"U530","priority":3},{"id":"G294","priority":4},{"id":"S971","priority":6}]Checked answer{"ids": ["M329", "S971"]}invalid_json_object
[{"id":"M329","priority":7},{"id":"U530","priority":3},{"id":"G294","priority":4},{"id":"S971","priority":6}]50 tokensCorrect
{"ids":["M329","S971"]}16 tokensCorrect
{"ids":["M329","S971"]}16 tokensCorrect
{"ids":["M329","S971"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to P402? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"P402","city":"Suva","units":5},{"id":"B180","city":"Suva","units":5},{"id":"Q489","city":"Lima","units":5},{"id":"W976","city":"Perth","units":11},{"id":"C783","city":"Kyoto","units":1}]Checked answer{"city": "Suva", "units": 5}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Suva","units":5}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Suva","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"U948","status":"closed"},{"id":"H579","status":"pending"},{"id":"S910","status":"pending"},{"id":"K479","status":"pending"},{"id":"E118","status":"ready"}]Checked answer{"ids": ["E118"]}invalid_json_object
[{"id":"E118","status":"ready"}]15 tokensinvalid_json_object
["E118"]7 tokens
Correct
{"ids":["E118"]}11 tokensinvalid_json_object
["ids":["E118"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Z697","units":0,"price":12},{"id":"Q291","units":15,"price":4},{"id":"O946","units":5,"price":17},{"id":"K476","units":5,"price":5},{"id":"X458","units":0,"price":13}]Checked answer{"total": 170}wrong_schema_or_value
{"total": 120}9 tokenswrong_schema_or_value
{"total":378}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"U282","priority":4},{"id":"W225","priority":6},{"id":"U463","priority":8},{"id":"G220","priority":9},{"id":"E208","priority":6}]Checked answer{"ids": ["G220", "U463"]}wrong_schema_or_value
{"ids": ["U282", "W225", "U463", "G220", "E208"]}35 tokenswrong_schema_or_value
{"ids":["G220","E208"]}16 tokenswrong_schema_or_value
{"ids":["G220","E208"]}16 tokenswrong_schema_or_value
{"ids":["G220","E208"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"S265","city":"Oslo","units":14},{"id":"M669","city":"Lima","units":12},{"id":"R856","city":"Lima","units":5},{"id":"J563","city":"Lima","units":19},{"id":"N529","city":"Perth","units":10}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Oslo","units":14}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"L364","status":"closed"},{"id":"D604","status":"ready"},{"id":"Q478","status":"closed"},{"id":"P506","status":"ready"},{"id":"D480","status":"ready"}]Checked answer{"ids": ["D480", "D604", "P506"]}invalid_json_object
{"ids": ["D480", "D604", "D480", "D604", "D604", "D604", "D604", "D604", "D604", "D604", "D604", "D604", "D604", "D604", "D604", "D696 tokenswrong_schema_or_value
{"ids":["D480","P506","D604"]}21 tokenswrong_schema_or_value
{"ids":["D480","P506","D604"]}21 tokenswrong_schema_or_value
{"ids":["D480","P506","D604"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"R166","units":6,"price":15},{"id":"F662","units":7,"price":6},{"id":"G797","units":11,"price":16},{"id":"J540","units":11,"price":15},{"id":"R239","units":10,"price":10}]Checked answer{"total": 573}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":368}8 tokenswrong_schema_or_value
{"total":368}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"V605","priority":2},{"id":"N395","priority":2},{"id":"V603","priority":3},{"id":"M529","priority":4},{"id":"Z339","priority":1}]Checked answer{"ids": ["M529", "V603"]}invalid_json_object
[{"id":"V603","priority":3},{"id":"M529","priority":4},{"id":"V605","priority":2},{"id":"N395","priority":2},{"id":"Z339","priority":1}]62 tokensCorrect
{"ids":["M529","V603"]}16 tokensCorrect
{"ids":["M529","V603"]}16 tokensCorrect
{"ids":["M529","V603"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record U883. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"I449","city":"Oslo","units":12},{"id":"U883","city":"Lima","units":16},{"id":"F982","city":"Accra","units":16},{"id":"H896","city":"Lima","units":9},{"id":"B654","city":"Accra","units":15}]Checked answer{"city": "Lima", "units": 16}Correct
{
"city": "Lima",
"units": 16
}19 tokensCorrect
{"city":"Lima","units":16}12 tokensCorrect
{"city":"Lima","units":16}12 tokensCorrect
{"city":"Lima","units":16}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"K810","status":"pending"},{"id":"E638","status":"pending"},{"id":"X157","status":"ready"},{"id":"C422","status":"closed"},{"id":"J570","status":"pending"}]Checked answer{"ids": ["X157"]}wrong_schema_or_value
{"ids": ["X157", "J570", "K810", "C422"]}29 tokensCorrect
{"ids":["X157"]}11 tokensCorrect
{"ids":["X157"]}11 tokensCorrect
{"ids":["X157"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"W710","units":13,"price":11},{"id":"T330","units":13,"price":4},{"id":"E606","units":17,"price":1},{"id":"E246","units":13,"price":4},{"id":"X374","units":13,"price":16}]Checked answer{"total": 472}invalid_json_object
{"total": 11 + 4 + 1 + 4 + 16 = 36}25 tokenswrong_schema_or_value
{"total":568}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"C348","priority":6},{"id":"H962","priority":6},{"id":"L146","priority":9},{"id":"J620","priority":7},{"id":"B904","priority":6}]Checked answer{"ids": ["L146", "J620"]}wrong_schema_or_value
{"ids": ["C348", "H962", "L146", "J620", "B904"]}35 tokensCorrect
{"ids":["L146","J620"]}16 tokensCorrect
{"ids":["L146","J620"]}16 tokenswrong_schema_or_value
{"ids":["L146","H962"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id E290 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"E290","city":"Kyoto","units":9},{"id":"R439","city":"Oslo","units":16},{"id":"N981","city":"Lima","units":1},{"id":"M412","city":"Suva","units":1},{"id":"B685","city":"Kyoto","units":0}]Checked answer{"city": "Kyoto", "units": 9}Correct
{"city":"Kyoto","units":9}12 tokensCorrect
{"city":"Kyoto","units":9}12 tokensCorrect
{"city":"Kyoto","units":9}12 tokensCorrect
{"city":"Kyoto","units":9}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"Q291","status":"pending"},{"id":"M682","status":"closed"},{"id":"L455","status":"closed"},{"id":"P131","status":"ready"},{"id":"O562","status":"closed"}]Checked answer{"ids": ["P131"]}wrong_schema_or_value
{
"ids": ["P131", "O562", "L455", "M682", "Q291"]
}38 tokensCorrect
{"ids":["P131"]}11 tokensCorrect
{"ids":["P131"]}11 tokensCorrect
{"ids":["P131"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"D635","units":14,"price":14},{"id":"R560","units":6,"price":5},{"id":"T874","units":5,"price":15},{"id":"M688","units":8,"price":3},{"id":"E988","units":12,"price":13}]Checked answer{"total": 481}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"I766","priority":6},{"id":"S305","priority":5},{"id":"C566","priority":5},{"id":"P861","priority":9},{"id":"J954","priority":3}]Checked answer{"ids": ["P861", "I766"]}invalid_json_object
[{"id":"I766","priority":6},{"id":"S305","priority":5},{"id":"C566","priority":5},{"id":"P861","priority":9},{"id":"J954","priority":3}]62 tokensCorrect
{"ids":["P861","I766"]}16 tokensCorrect
{"ids":["P861","I766"]}16 tokensCorrect
{"ids":["P861","I766"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to A821? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"A821","city":"Bern","units":1},{"id":"P946","city":"Perth","units":10},{"id":"B835","city":"Lima","units":8},{"id":"G233","city":"Bern","units":7},{"id":"K918","city":"Suva","units":16}]Checked answer{"city": "Bern", "units": 1}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Bern","units":1}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Bern","units":1}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"R939","status":"pending"},{"id":"Z370","status":"pending"},{"id":"I168","status":"pending"},{"id":"Y964","status":"closed"},{"id":"L890","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"Z370","status":"pending"},{"id":"I168","status":"pending"},{"id":"Y964","status":"closed"},{"id":"L890","status":"closed"}]51 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensinvalid_json_object
["ids":[]}6 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"V434","units":0,"price":17},{"id":"K768","units":9,"price":12},{"id":"L889","units":12,"price":15},{"id":"S828","units":15,"price":4},{"id":"Y688","units":8,"price":10}]Checked answer{"total": 428}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"K296","priority":5},{"id":"U737","priority":9},{"id":"F820","priority":6},{"id":"N162","priority":9},{"id":"H332","priority":3}]Checked answer{"ids": ["N162", "U737"]}wrong_schema_or_value
{"ids": ["U737", "F820", "N162", "H332", "K296"]}35 tokensCorrect
{"ids":["N162","U737"]}16 tokenswrong_schema_or_value
{"ids":["N162","K296"]}16 tokensCorrect
{"ids":["N162","U737"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=H582. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"F320","city":"Oslo","units":15},{"id":"D808","city":"Suva","units":19},{"id":"H582","city":"Lima","units":7},{"id":"N307","city":"Accra","units":5},{"id":"O875","city":"Oslo","units":1}]Checked answer{"city": "Lima", "units": 7}wrong_schema_or_value
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Lima","units":7}11 tokensCorrect
{"city":"Lima","units":7}11 tokensCorrect
{"city":"Lima","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Z274","status":"pending"},{"id":"N463","status":"ready"},{"id":"G482","status":"ready"},{"id":"A426","status":"pending"},{"id":"Z294","status":"pending"}]Checked answer{"ids": ["G482", "N463"]}wrong_schema_or_value
{"ids": ["A426", "G482", "N463", "Z274", "Z294"]}35 tokensCorrect
{"ids":["G482","N463"]}16 tokensCorrect
{"ids":["G482","N463"]}16 tokenswrong_schema_or_value
{"ids":["N463","G482"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"J232","units":3,"price":13},{"id":"Q854","units":10,"price":13},{"id":"O313","units":10,"price":9},{"id":"T321","units":18,"price":3},{"id":"O143","units":5,"price":15}]Checked answer{"total": 388}invalid_json_object
{
"total": 13 + 13 + 13 + 13 + 15 + 9 + 3 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 96 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"J252","priority":4},{"id":"G687","priority":7},{"id":"W819","priority":3},{"id":"E283","priority":3},{"id":"T573","priority":6}]Checked answer{"ids": ["G687", "T573"]}invalid_json_object
[{"id":"J252"},{"id":"G687"},{"id":"W819"},{"id":"E283"},{"id":"T573"}]43 tokensCorrect
{"ids":["G687","T573"]}16 tokensCorrect
{"ids":["G687","T573"]}16 tokenswrong_schema_or_value
{"ids":["G687","J252"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"M910","city":"Bern","units":5},{"id":"W658","city":"Perth","units":2},{"id":"D785","city":"Oslo","units":16},{"id":"E972","city":"Lima","units":16},{"id":"G348","city":"Kyoto","units":1}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Bern","units":5},{"city":"Perth","units":2},{"city":"Oslo","units":16},{"city":"Lima","units":16},{"city":"Kyoto","units":1}]54 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"I777","status":"ready"},{"id":"N471","status":"ready"},{"id":"R358","status":"pending"},{"id":"N972","status":"ready"},{"id":"E654","status":"ready"}]Checked answer{"ids": ["E654", "I777", "N471", "N972"]}wrong_schema_or_value
{"ids": ["I777", "N471", "N972", "E654"]}29 tokenswrong_schema_or_value
{"ids":["I777","N471","N972","E654"]}26 tokensCorrect
{"ids":["E654","I777","N471","N972"]}26 tokensCorrect
{"ids":["E654","I777","N471","N972"]}26 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"J811","units":10,"price":3},{"id":"D795","units":2,"price":9},{"id":"C647","units":8,"price":1},{"id":"B366","units":3,"price":13},{"id":"F149","units":16,"price":2}]Checked answer{"total": 127}invalid_json_object
{"total": 10 * 3 + 2 * 9 + 8 * 1 + 3 * 13 + 16 * 2} = 30 + 18 + 8 + 39 + 32 = 12761 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Q507","priority":5},{"id":"J383","priority":2},{"id":"H732","priority":9},{"id":"I345","priority":3},{"id":"J915","priority":2}]Checked answer{"ids": ["H732", "Q507"]}wrong_schema_or_value
{"ids": ["J383", "J915"]}17 tokenswrong_schema_or_value
{"ids":["H732","I345"]}16 tokensCorrect
{"ids":["H732","Q507"]}16 tokensCorrect
{"ids":["H732","Q507"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Y373 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"X786","city":"Bern","units":1},{"id":"Y373","city":"Kyoto","units":19},{"id":"J737","city":"Oslo","units":1},{"id":"B175","city":"Riga","units":16},{"id":"P411","city":"Perth","units":9}]Checked answer{"city": "Kyoto", "units": 19}wrong_schema_or_value
{"city":"Bern","units":1}10 tokensCorrect
{"city":"Kyoto","units":19}13 tokensCorrect
{"city":"Kyoto","units":19}13 tokensCorrect
{"city":"Kyoto","units":19}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"O615","status":"pending"},{"id":"W884","status":"ready"},{"id":"Q742","status":"pending"},{"id":"P916","status":"pending"},{"id":"P666","status":"pending"}]Checked answer{"ids": ["W884"]}wrong_schema_or_value
{
"ids": ["O615", "W884", "Q742", "P666", "P916"]
}38 tokensCorrect
{"ids":["W884"]}11 tokensCorrect
{"ids":["W884"]}11 tokensCorrect
{"ids":["W884"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"O437","units":19,"price":13},{"id":"S406","units":5,"price":11},{"id":"Q508","units":4,"price":6},{"id":"M860","units":9,"price":9},{"id":"V205","units":2,"price":17}]Checked answer{"total": 441}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"V178","priority":2},{"id":"Y846","priority":1},{"id":"L904","priority":9},{"id":"V663","priority":4},{"id":"F773","priority":3}]Checked answer{"ids": ["L904", "V663"]}invalid_json_object
[{"id":"V178","priority":2},{"id":"Y846","priority":1},{"id":"L904","priority":9},{"id":"V663","priority":4},{"id":"F773","priority":3}]62 tokenswrong_schema_or_value
{"ids":["L904","F773"]}16 tokensCorrect
{"ids":["L904","V663"]}16 tokensCorrect
{"ids":["L904","V663"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to C311? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"M626","city":"Kyoto","units":15},{"id":"C311","city":"Kyoto","units":12},{"id":"B630","city":"Perth","units":19},{"id":"U812","city":"Accra","units":17},{"id":"Z832","city":"Accra","units":15}]Checked answer{"city": "Kyoto", "units": 12}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Kyoto","units":12}13 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Kyoto","units":12}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"P295","status":"closed"},{"id":"A242","status":"closed"},{"id":"M472","status":"pending"},{"id":"Q124","status":"pending"},{"id":"P923","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"A242","status":"closed"},{"id":"M472","status":"pending"},{"id":"P295","status":"closed"},{"id":"P923","status":"closed"},{"id":"Q124","status":"pending"}]63 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"C857","units":18,"price":14},{"id":"S255","units":11,"price":7},{"id":"O126","units":15,"price":6},{"id":"Z595","units":2,"price":13},{"id":"A885","units":2,"price":8}]Checked answer{"total": 461}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"V206","priority":9},{"id":"U705","priority":7},{"id":"J534","priority":3},{"id":"L673","priority":4},{"id":"C764","priority":7}]Checked answer{"ids": ["V206", "C764"]}wrong_schema_or_value
{"ids": ["U705", "J534", "L673", "C764", "V206"]}35 tokensCorrect
{"ids":["V206","C764"]}16 tokensCorrect
{"ids":["V206","C764"]}16 tokensCorrect
{"ids":["V206","C764"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=U945. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"P741","city":"Perth","units":19},{"id":"U945","city":"Riga","units":16},{"id":"E420","city":"Perth","units":4},{"id":"D730","city":"Oslo","units":8},{"id":"H289","city":"Bern","units":10}]Checked answer{"city": "Riga", "units": 16}Correct
{"city":"Riga","units":16}12 tokensCorrect
{"city":"Riga","units":16}12 tokensCorrect
{"city":"Riga","units":16}12 tokensCorrect
{"city":"Riga","units":16}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"D608","status":"ready"},{"id":"S951","status":"closed"},{"id":"W237","status":"ready"},{"id":"D333","status":"pending"},{"id":"W195","status":"ready"}]Checked answer{"ids": ["D608", "W195", "W237"]}wrong_schema_or_value
{"ids": ["D333", "D608", "D333", "D608", "S951", "W237", "W195", "W237"]}53 tokensCorrect
{"ids":["D608","W195","W237"]}21 tokensCorrect
{"ids":["D608","W195","W237"]}21 tokensCorrect
{"ids":["D608","W195","W237"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"H805","units":4,"price":3},{"id":"L342","units":16,"price":12},{"id":"O276","units":18,"price":14},{"id":"H358","units":18,"price":9},{"id":"B822","units":7,"price":15}]Checked answer{"total": 723}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"Z639","priority":9},{"id":"S275","priority":9},{"id":"Q284","priority":8},{"id":"Q902","priority":5},{"id":"X591","priority":6}]Checked answer{"ids": ["S275", "Z639"]}invalid_json_object
[{"id":"S275","priority":9},{"id":"Q284","priority":8},{"id":"Q902","priority":5},{"id":"Z639","priority":9},{"id":"X591","priority":6}]62 tokensCorrect
{"ids":["S275","Z639"]}16 tokensCorrect
{"ids":["S275","Z639"]}16 tokensCorrect
{"ids":["S275","Z639"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record K189. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"M255","city":"Bern","units":4},{"id":"K189","city":"Perth","units":7},{"id":"W994","city":"Lima","units":11},{"id":"J669","city":"Accra","units":10},{"id":"L279","city":"Accra","units":9}]Checked answer{"city": "Perth", "units": 7}Correct
{
"city": "Perth",
"units": 7
}18 tokensCorrect
{"city":"Perth","units":7}11 tokensCorrect
{"city":"Perth","units":7}11 tokensCorrect
{"city":"Perth","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"C586","status":"ready"},{"id":"W381","status":"pending"},{"id":"A554","status":"closed"},{"id":"B905","status":"closed"},{"id":"G658","status":"ready"}]Checked answer{"ids": ["C586", "G658"]}wrong_schema_or_value
{"ids": ["A554", "B905", "C586", "G658"]}29 tokensCorrect
{"ids":["C586","G658"]}16 tokensCorrect
{"ids":["C586","G658"]}16 tokensCorrect
{"ids":["C586","G658"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"C333","units":17,"price":13},{"id":"Z232","units":18,"price":19},{"id":"I944","units":13,"price":3},{"id":"G936","units":7,"price":4},{"id":"D466","units":17,"price":18}]Checked answer{"total": 936}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"T570","priority":7},{"id":"D775","priority":6},{"id":"N986","priority":4},{"id":"Y573","priority":2},{"id":"R671","priority":5}]Checked answer{"ids": ["T570", "D775"]}wrong_schema_or_value
{"ids": ["Y573", "R671", "N986", "T570", "D775"]}35 tokensCorrect
{"ids":["T570","D775"]}16 tokensCorrect
{"ids":["T570","D775"]}16 tokensCorrect
{"ids":["T570","D775"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"P110","city":"Bern","units":11},{"id":"H150","city":"Lima","units":13},{"id":"X185","city":"Riga","units":10},{"id":"I755","city":"Kyoto","units":0},{"id":"C389","city":"Riga","units":11}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Bern","units":11}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"T737","status":"pending"},{"id":"X418","status":"ready"},{"id":"J470","status":"ready"},{"id":"A673","status":"pending"},{"id":"Y713","status":"closed"}]Checked answer{"ids": ["J470", "X418"]}wrong_schema_or_value
{
"ids": ["A673", "J470", "X418", "T737", "Y713"]
}38 tokensCorrect
{"ids":["J470","X418"]}16 tokensCorrect
{"ids":["J470","X418"]}16 tokensCorrect
{"ids":["J470","X418"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"L569","units":16,"price":12},{"id":"T526","units":10,"price":18},{"id":"P663","units":1,"price":1},{"id":"A233","units":12,"price":7},{"id":"P158","units":14,"price":10}]Checked answer{"total": 597}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"X897","priority":7},{"id":"V423","priority":8},{"id":"S917","priority":6},{"id":"M768","priority":7},{"id":"R297","priority":2}]Checked answer{"ids": ["V423", "M768"]}invalid_json_object
[{"id":"X897","priority":7},{"id":"V423","priority":8},{"id":"S917","priority":6},{"id":"M768","priority":7},{"id":"R297","priority":2}]62 tokenswrong_schema_or_value
{"ids":["V423","X897"]}16 tokenswrong_schema_or_value
{"ids":["V423","X897"]}16 tokenswrong_schema_or_value
{"ids":["V423","X897"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to N677? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"E509","city":"Lima","units":17},{"id":"N677","city":"Lima","units":19},{"id":"G148","city":"Lima","units":16},{"id":"S350","city":"Riga","units":19},{"id":"B268","city":"Accra","units":8}]Checked answer{"city": "Lima", "units": 19}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Lima","units":19}12 tokensCorrect
{"city":"Lima","units":19}12 tokensCorrect
{"city":"Lima","units":19}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"F284","status":"pending"},{"id":"V975","status":"ready"},{"id":"Z294","status":"pending"},{"id":"I166","status":"pending"},{"id":"V516","status":"pending"}]Checked answer{"ids": ["V975"]}invalid_json_object
[{"id":"V516","status":"ready"},{"id":"V975","status":"ready"},{"id":"F284","status":"pending"},{"id":"I166","status":"pending"},{"id":"Z294","status":"pending"}]63 tokensCorrect
{"ids":["V975"]}11 tokensCorrect
{"ids":["V975"]}11 tokensinvalid_json_object
["ids":["V516"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"P312","units":2,"price":4},{"id":"P349","units":6,"price":9},{"id":"E650","units":10,"price":19},{"id":"C341","units":4,"price":12},{"id":"D621","units":6,"price":14}]Checked answer{"total": 384}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"E417","priority":1},{"id":"Z934","priority":9},{"id":"W678","priority":5},{"id":"F375","priority":1},{"id":"Q580","priority":3}]Checked answer{"ids": ["Z934", "W678"]}wrong_schema_or_value
{"ids": ["E417", "Q580"]}17 tokenswrong_schema_or_value
{"ids":["Z934","E417"]}16 tokenswrong_schema_or_value
{"ids":["Z934","E417"]}16 tokenswrong_schema_or_value
{"ids":["Z934","E417"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Y437. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"Z908","city":"Bern","units":9},{"id":"J146","city":"Lima","units":3},{"id":"C660","city":"Riga","units":2},{"id":"Q268","city":"Lima","units":6},{"id":"Y437","city":"Perth","units":9}]Checked answer{"city": "Perth", "units": 9}Correct
{"city": "Perth", "units": 9}14 tokensCorrect
{"city":"Perth","units":9}11 tokensCorrect
{"city":"Perth","units":9}11 tokensCorrect
{"city":"Perth","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"X750","status":"ready"},{"id":"K382","status":"ready"},{"id":"C321","status":"pending"},{"id":"S333","status":"pending"},{"id":"E655","status":"closed"}]Checked answer{"ids": ["K382", "X750"]}wrong_schema_or_value
{"ids": ["C321", "E655", "K382", "X750", "S333"]}35 tokensCorrect
{"ids":["K382","X750"]}16 tokensCorrect
{"ids":["K382","X750"]}16 tokensCorrect
{"ids":["K382","X750"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"C379","units":17,"price":5},{"id":"T454","units":6,"price":4},{"id":"T401","units":19,"price":4},{"id":"A691","units":2,"price":10},{"id":"V139","units":10,"price":4}]Checked answer{"total": 245}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"H485","priority":2},{"id":"G564","priority":4},{"id":"D230","priority":2},{"id":"P817","priority":1},{"id":"M723","priority":3}]Checked answer{"ids": ["G564", "M723"]}invalid_json_object
[{"id":"H485","priority":2},{"id":"G564","priority":4},{"id":"D230","priority":2},{"id":"P817","priority":1},{"id":"M723","priority":3}]62 tokensCorrect
{"ids":["G564","M723"]}16 tokensCorrect
{"ids":["G564","M723"]}16 tokenswrong_schema_or_value
{"ids":["G564","D230"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z205. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Z205","city":"Perth","units":0},{"id":"X925","city":"Kyoto","units":4},{"id":"R737","city":"Kyoto","units":17},{"id":"S703","city":"Bern","units":14},{"id":"X471","city":"Lima","units":8}]Checked answer{"city": "Perth", "units": 0}wrong_schema_or_value
{
"city": "Perth",
"units": null
}17 tokensCorrect
{"city":"Perth","units":0}11 tokensCorrect
{"city":"Perth","units":0}11 tokensCorrect
{"city":"Perth","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Z157","status":"closed"},{"id":"Z509","status":"pending"},{"id":"M260","status":"ready"},{"id":"H898","status":"pending"},{"id":"Y576","status":"pending"}]Checked answer{"ids": ["M260"]}wrong_schema_or_value
{"ids": ["M260", "Y576", "Z157", "Z509", "H898"]}35 tokensCorrect
{"ids":["M260"]}11 tokensCorrect
{"ids":["M260"]}11 tokensCorrect
{"ids":["M260"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"E499","units":13,"price":3},{"id":"D636","units":16,"price":15},{"id":"K195","units":12,"price":17},{"id":"D904","units":14,"price":16},{"id":"J157","units":16,"price":1}]Checked answer{"total": 723}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Q656","priority":8},{"id":"S960","priority":9},{"id":"B515","priority":9},{"id":"P452","priority":2},{"id":"F730","priority":7}]Checked answer{"ids": ["B515", "S960"]}wrong_schema_or_value
{"ids": ["S960", "B515", "P452", "F730", "Q656"]}35 tokensCorrect
{"ids":["B515","S960"]}16 tokenswrong_schema_or_value
{"ids":["S960","B515"]}16 tokenswrong_schema_or_value
{"ids":["S960","B515"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id F479 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"W722","city":"Accra","units":16},{"id":"H284","city":"Kyoto","units":17},{"id":"F479","city":"Bern","units":12},{"id":"S697","city":"Kyoto","units":10},{"id":"N215","city":"Bern","units":19}]Checked answer{"city": "Bern", "units": 12}wrong_schema_or_value
{"city":"Accra","units":16}12 tokensCorrect
{"city":"Bern","units":12}11 tokensCorrect
{"city":"Bern","units":12}11 tokensCorrect
{"city":"Bern","units":12}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"E194","status":"pending"},{"id":"K335","status":"pending"},{"id":"X999","status":"closed"},{"id":"A308","status":"closed"},{"id":"Q640","status":"ready"}]Checked answer{"ids": ["Q640"]}wrong_schema_or_value
{
"ids": ["A308", "E194", "K335", "Q640"]
}32 tokensCorrect
{"ids":["Q640"]}11 tokensCorrect
{"ids":["Q640"]}11 tokensCorrect
{"ids":["Q640"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"W535","units":17,"price":5},{"id":"V323","units":9,"price":4},{"id":"C644","units":1,"price":15},{"id":"H274","units":7,"price":8},{"id":"U428","units":15,"price":14}]Checked answer{"total": 402}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"J342","priority":5},{"id":"J523","priority":2},{"id":"H661","priority":3},{"id":"M799","priority":4},{"id":"G584","priority":2}]Checked answer{"ids": ["J342", "M799"]}invalid_json_object
[{"id":"J523","priority":2},{"id":"J342","priority":5},{"id":"H661","priority":3},{"id":"M799","priority":4},{"id":"G584","priority":2}]62 tokensCorrect
{"ids":["J342","M799"]}16 tokensCorrect
{"ids":["J342","M799"]}16 tokensCorrect
{"ids":["J342","M799"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"Q891","city":"Riga","units":11},{"id":"J561","city":"Oslo","units":1},{"id":"L315","city":"Accra","units":18},{"id":"G966","city":"Perth","units":4},{"id":"K194","city":"Riga","units":12}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"D885","status":"closed"},{"id":"O119","status":"closed"},{"id":"J909","status":"pending"},{"id":"K491","status":"ready"},{"id":"R115","status":"ready"}]Checked answer{"ids": ["K491", "R115"]}invalid_json_object
[{"id":"K491"},{"id":"R115"}]19 tokensinvalid_json_object
["K491","R115"]12 tokens
Correct
{"ids":["K491","R115"]}16 tokensinvalid_json_object
["ids":["K491","R115"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"H852","units":17,"price":3},{"id":"Q851","units":14,"price":15},{"id":"O810","units":8,"price":5},{"id":"S557","units":2,"price":1},{"id":"H840","units":1,"price":9}]Checked answer{"total": 312}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"U181","priority":7},{"id":"O337","priority":4},{"id":"P597","priority":4},{"id":"U946","priority":8},{"id":"Q528","priority":1}]Checked answer{"ids": ["U946", "U181"]}wrong_schema_or_value
{"ids": ["U181", "O337", "P597", "U946", "Q528"]}35 tokenswrong_schema_or_value
{"ids":["U181","U946"]}16 tokenswrong_schema_or_value
{"ids":["U181","U946"]}16 tokenswrong_schema_or_value
{"ids":["U181","O337"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=X245. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"A885","city":"Riga","units":10},{"id":"G712","city":"Perth","units":17},{"id":"X245","city":"Accra","units":7},{"id":"R669","city":"Lima","units":6},{"id":"T894","city":"Suva","units":18}]Checked answer{"city": "Accra", "units": 7}wrong_schema_or_value
{"city": "Riga", "units": 10}15 tokensCorrect
{"city":"Accra","units":7}11 tokensCorrect
{"city":"Accra","units":7}11 tokensCorrect
{"city":"Accra","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"E160","status":"closed"},{"id":"A448","status":"ready"},{"id":"M847","status":"closed"},{"id":"Q274","status":"closed"},{"id":"Q531","status":"pending"}]Checked answer{"ids": ["A448"]}wrong_schema_or_value
{"ids": ["A448", "E160", "M847", "Q274", "Q531"]}35 tokensCorrect
{"ids":["A448"]}11 tokensCorrect
{"ids":["A448"]}11 tokensCorrect
{"ids":["A448"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"X109","units":14,"price":9},{"id":"D113","units":8,"price":16},{"id":"N846","units":15,"price":2},{"id":"T160","units":5,"price":9},{"id":"S140","units":15,"price":6}]Checked answer{"total": 419}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"A711","priority":4},{"id":"Z227","priority":1},{"id":"P160","priority":6},{"id":"U704","priority":2},{"id":"U982","priority":6}]Checked answer{"ids": ["P160", "U982"]}invalid_json_object
[{"id":"A711","priority":4},{"id":"U704","priority":2},{"id":"U982","priority":6},{"id":"P160","priority":6},{"id":"Z227","priority":1}]62 tokensCorrect
{"ids":["P160","U982"]}16 tokenswrong_schema_or_value
{"ids":["P160","A711"]}16 tokenswrong_schema_or_value
{"ids":["A711","P160"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record F652. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"F652","city":"Kyoto","units":2},{"id":"W207","city":"Bern","units":1},{"id":"I174","city":"Bern","units":15},{"id":"G303","city":"Accra","units":2},{"id":"K830","city":"Oslo","units":8}]Checked answer{"city": "Kyoto", "units": 2}Correct
{
"city": "Kyoto",
"units": 2
}19 tokensCorrect
{"city":"Kyoto","units":2}12 tokensCorrect
{"city":"Kyoto","units":2}12 tokensCorrect
{"city":"Kyoto","units":2}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"L956","status":"ready"},{"id":"P130","status":"closed"},{"id":"C659","status":"closed"},{"id":"F799","status":"pending"},{"id":"S953","status":"ready"}]Checked answer{"ids": ["L956", "S953"]}wrong_schema_or_value
{"ids": ["C659", "F799", "L956", "S953", "P130"]}35 tokensCorrect
{"ids":["L956","S953"]}16 tokensCorrect
{"ids":["L956","S953"]}16 tokensCorrect
{"ids":["L956","S953"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"O988","units":6,"price":17},{"id":"K134","units":18,"price":17},{"id":"A302","units":9,"price":13},{"id":"M528","units":14,"price":10},{"id":"S100","units":16,"price":8}]Checked answer{"total": 793}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"M971","priority":2},{"id":"H722","priority":3},{"id":"U841","priority":2},{"id":"P609","priority":4},{"id":"O929","priority":2}]Checked answer{"ids": ["P609", "H722"]}wrong_schema_or_value
{"ids": ["M971", "U841", "P609", "H722", "O929"]}35 tokensCorrect
{"ids":["P609","H722"]}16 tokenswrong_schema_or_value
{"ids":["H722","P609"]}16 tokensCorrect
{"ids":["P609","H722"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id R510 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"I590","city":"Perth","units":3},{"id":"U653","city":"Lima","units":11},{"id":"R510","city":"Accra","units":0},{"id":"E159","city":"Perth","units":4},{"id":"O357","city":"Kyoto","units":19}]Checked answer{"city": "Accra", "units": 0}wrong_schema_or_value
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Accra","units":0}11 tokensCorrect
{"city":"Accra","units":0}11 tokensCorrect
{"city":"Accra","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"G783","status":"pending"},{"id":"G706","status":"pending"},{"id":"G638","status":"ready"},{"id":"C309","status":"pending"},{"id":"K503","status":"closed"}]Checked answer{"ids": ["G638"]}wrong_schema_or_value
{
"ids": ["G638", "G706", "G783", "K503"]
}32 tokensCorrect
{"ids":["G638"]}11 tokensCorrect
{"ids":["G638"]}11 tokensCorrect
{"ids":["G638"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"R836","units":1,"price":16},{"id":"K414","units":14,"price":3},{"id":"C956","units":14,"price":19},{"id":"Y307","units":5,"price":5},{"id":"K756","units":14,"price":13}]Checked answer{"total": 531}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"S407","priority":2},{"id":"U102","priority":8},{"id":"Q613","priority":6},{"id":"K522","priority":6},{"id":"B769","priority":3}]Checked answer{"ids": ["U102", "K522"]}invalid_json_object
[{"id":"U102","priority":8},{"id":"Q613","priority":6},{"id":"S407","priority":2},{"id":"K522","priority":6},{"id":"B769","priority":3}]62 tokenswrong_schema_or_value
{"ids":["U102","Q613"]}16 tokensCorrect
{"ids":["U102","K522"]}16 tokenswrong_schema_or_value
{"ids":["U102","Q613"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to S192? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"S192","city":"Perth","units":6},{"id":"O819","city":"Lima","units":4},{"id":"Y816","city":"Oslo","units":16},{"id":"L782","city":"Oslo","units":2},{"id":"G701","city":"Perth","units":9}]Checked answer{"city": "Perth", "units": 6}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Perth","units":6}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"A237","status":"ready"},{"id":"C924","status":"ready"},{"id":"T655","status":"closed"},{"id":"J454","status":"pending"},{"id":"E969","status":"closed"}]Checked answer{"ids": ["A237", "C924"]}invalid_json_object
[{"id":"A237"},{"id":"C924"},{"id":"E969"},{"id":"J454"},{"id":"T655"}]43 tokensCorrect
{"ids":["A237","C924"]}16 tokensCorrect
{"ids":["A237","C924"]}16 tokensinvalid_json_object
["ids":["C924","A237"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"L655","units":13,"price":10},{"id":"F563","units":6,"price":4},{"id":"Y478","units":19,"price":18},{"id":"C738","units":9,"price":3},{"id":"F113","units":4,"price":1}]Checked answer{"total": 527}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"B260","priority":2},{"id":"A300","priority":4},{"id":"Q546","priority":6},{"id":"J237","priority":1},{"id":"F177","priority":3}]Checked answer{"ids": ["Q546", "A300"]}wrong_schema_or_value
{"ids": ["A300", "B260", "F177", "J237", "Q546"]}35 tokensCorrect
{"ids":["Q546","A300"]}16 tokenswrong_schema_or_value
{"ids":["A300","Q546"]}16 tokensCorrect
{"ids":["Q546","A300"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"X290","city":"Perth","units":18},{"id":"T948","city":"Lima","units":18},{"id":"U694","city":"Suva","units":12},{"id":"G524","city":"Lima","units":5},{"id":"G910","city":"Bern","units":5}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Perth","units":18}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"M138","status":"ready"},{"id":"C398","status":"ready"},{"id":"W281","status":"pending"},{"id":"D751","status":"pending"},{"id":"G918","status":"ready"}]Checked answer{"ids": ["C398", "G918", "M138"]}wrong_schema_or_value
{"ids": ["C398", "D751", "G918", "M138", "W281"]}35 tokensCorrect
{"ids":["C398","G918","M138"]}21 tokensCorrect
{"ids":["C398","G918","M138"]}21 tokenswrong_schema_or_value
{"ids":["C398","M138","G918"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"M924","units":16,"price":12},{"id":"M710","units":14,"price":13},{"id":"T700","units":2,"price":11},{"id":"S482","units":5,"price":6},{"id":"Y444","units":4,"price":19}]Checked answer{"total": 502}wrong_schema_or_value
{
"total": 34
}11 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":408}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"L675","priority":2},{"id":"U736","priority":3},{"id":"G268","priority":5},{"id":"O963","priority":3},{"id":"K213","priority":5}]Checked answer{"ids": ["G268", "K213"]}invalid_json_object
[{"id":"U736","priority":3},{"id":"G268","priority":5},{"id":"L675","priority":2},{"id":"K213","priority":5}]50 tokensCorrect
{"ids":["G268","K213"]}16 tokensCorrect
{"ids":["G268","K213"]}16 tokensCorrect
{"ids":["G268","K213"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record D576. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Y350","city":"Perth","units":10},{"id":"I570","city":"Oslo","units":10},{"id":"H490","city":"Kyoto","units":11},{"id":"N146","city":"Riga","units":19},{"id":"D576","city":"Lima","units":17}]Checked answer{"city": "Lima", "units": 17}invalid_json_object
[{"city":"Lima","units":17}]13 tokensCorrect
{"city":"Lima","units":17}12 tokensCorrect
{"city":"Lima","units":17}12 tokensCorrect
{"city":"Lima","units":17}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"U140","status":"pending"},{"id":"J474","status":"closed"},{"id":"Q377","status":"pending"},{"id":"Z169","status":"pending"},{"id":"P351","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["J474", "P351", "Q377", "U140", "Z169"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"P818","units":0,"price":8},{"id":"U704","units":10,"price":9},{"id":"G387","units":12,"price":7},{"id":"P380","units":3,"price":1},{"id":"Z922","units":13,"price":16}]Checked answer{"total": 385}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":368}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"G645","priority":4},{"id":"H438","priority":6},{"id":"J570","priority":4},{"id":"F871","priority":3},{"id":"Y328","priority":1}]Checked answer{"ids": ["H438", "G645"]}wrong_schema_or_value
{"ids": ["G645", "H438", "J570", "F871", "Y328"]}35 tokenswrong_schema_or_value
{"ids":["H438","J570"]}16 tokensCorrect
{"ids":["H438","G645"]}16 tokensCorrect
{"ids":["H438","G645"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id I580 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"G885","city":"Riga","units":13},{"id":"M173","city":"Kyoto","units":6},{"id":"Z738","city":"Suva","units":1},{"id":"R354","city":"Kyoto","units":13},{"id":"I580","city":"Bern","units":6}]Checked answer{"city": "Bern", "units": 6}wrong_schema_or_value
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Bern","units":6}10 tokensCorrect
{"city":"Bern","units":6}10 tokensCorrect
{"city":"Bern","units":6}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"G781","status":"pending"},{"id":"S524","status":"ready"},{"id":"B592","status":"pending"},{"id":"C989","status":"pending"},{"id":"V448","status":"ready"}]Checked answer{"ids": ["S524", "V448"]}wrong_schema_or_value
{
"ids": ["B592", "C989", "G781", "S524", "V448"]
}38 tokenswrong_schema_or_value
{"ids":["V448","S524"]}16 tokenswrong_schema_or_value
{"ids":["V448","S524"]}16 tokensCorrect
{"ids":["S524","V448"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"I429","units":3,"price":1},{"id":"H697","units":19,"price":18},{"id":"X815","units":18,"price":14},{"id":"E892","units":13,"price":3},{"id":"T418","units":13,"price":15}]Checked answer{"total": 831}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Q895","priority":9},{"id":"I690","priority":9},{"id":"Q823","priority":4},{"id":"T860","priority":1},{"id":"Z869","priority":8}]Checked answer{"ids": ["I690", "Q895"]}invalid_json_object
[{"id":"Q895","priority":9},{"id":"I690","priority":9},{"id":"T860","priority":1},{"id":"Z869","priority":8}]50 tokensCorrect
{"ids":["I690","Q895"]}16 tokensCorrect
{"ids":["I690","Q895"]}16 tokensCorrect
{"ids":["I690","Q895"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to R910? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"B938","city":"Accra","units":5},{"id":"U782","city":"Suva","units":16},{"id":"R910","city":"Suva","units":13},{"id":"C605","city":"Suva","units":12},{"id":"L170","city":"Perth","units":4}]Checked answer{"city": "Suva", "units": 13}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Suva","units":13}12 tokensCorrect
{"city":"Suva","units":13}12 tokensCorrect
{"city":"Suva","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"B753","status":"ready"},{"id":"S178","status":"closed"},{"id":"I684","status":"pending"},{"id":"Y747","status":"ready"},{"id":"Y953","status":"ready"}]Checked answer{"ids": ["B753", "Y747", "Y953"]}invalid_json_object
[{"id":"B753","status":"ready"},{"id":"Y747","status":"ready"},{"id":"Y953","status":"ready"}]39 tokensCorrect
{"ids":["B753","Y747","Y953"]}21 tokensCorrect
{"ids":["B753","Y747","Y953"]}21 tokensinvalid_json_object
["ids":["B753","Y747","Y953"]]21 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"U781","units":14,"price":1},{"id":"T544","units":14,"price":6},{"id":"U967","units":16,"price":2},{"id":"Z827","units":19,"price":9},{"id":"X986","units":16,"price":4}]Checked answer{"total": 365}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"K151","priority":5},{"id":"Z342","priority":3},{"id":"B444","priority":8},{"id":"G606","priority":7},{"id":"S853","priority":5}]Checked answer{"ids": ["B444", "G606"]}wrong_schema_or_value
{"ids": ["K151", "Z342", "B444", "G606", "S853"]}35 tokensCorrect
{"ids":["B444","G606"]}16 tokensCorrect
{"ids":["B444","G606"]}16 tokensCorrect
{"ids":["B444","G606"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=J117. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"T200","city":"Lima","units":2},{"id":"N361","city":"Oslo","units":1},{"id":"Z373","city":"Accra","units":8},{"id":"J117","city":"Kyoto","units":5},{"id":"B579","city":"Riga","units":7}]Checked answer{"city": "Kyoto", "units": 5}Correct
{"city":"Kyoto","units":5}12 tokensCorrect
{"city":"Kyoto","units":5}12 tokensCorrect
{"city":"Kyoto","units":5}12 tokensCorrect
{"city":"Kyoto","units":5}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"G217","status":"ready"},{"id":"A711","status":"pending"},{"id":"I780","status":"ready"},{"id":"M998","status":"closed"},{"id":"X248","status":"ready"}]Checked answer{"ids": ["G217", "I780", "X248"]}wrong_schema_or_value
{"ids": ["A711", "G217", "I780", "M998", "X248"]}35 tokensCorrect
{"ids":["G217","I780","X248"]}21 tokenswrong_schema_or_value
{"ids":["A711","G217","I780","X248"]}26 tokensCorrect
{"ids":["G217","I780","X248"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"W142","units":5,"price":12},{"id":"U963","units":5,"price":19},{"id":"F245","units":7,"price":17},{"id":"R247","units":16,"price":10},{"id":"Y168","units":16,"price":7}]Checked answer{"total": 546}invalid_json_object
{
"total": 12 + 19 + 17 + 10 + 7 = 67
}30 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"T836","priority":6},{"id":"Z670","priority":8},{"id":"W476","priority":6},{"id":"Z434","priority":9},{"id":"Q549","priority":8}]Checked answer{"ids": ["Z434", "Q549"]}invalid_json_object
[{"id":"Z670","priority":8},{"id":"W476","priority":6},{"id":"Z434","priority":9},{"id":"T836","priority":6}]50 tokensCorrect
{"ids":["Z434","Q549"]}16 tokenswrong_schema_or_value
{"ids":["Q549","Z434"]}16 tokensCorrect
{"ids":["Z434","Q549"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"L565","city":"Lima","units":16},{"id":"O897","city":"Suva","units":13},{"id":"U654","city":"Perth","units":13},{"id":"I174","city":"Kyoto","units":4},{"id":"D613","city":"Lima","units":18}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Lima","units":16},{"city":"Suva","units":13},{"city":"Perth","units":13},{"city":"Kyoto","units":4},{"city":"Lima","units":18}]57 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"N306","status":"closed"},{"id":"K689","status":"pending"},{"id":"N980","status":"closed"},{"id":"C840","status":"closed"},{"id":"J412","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["C840", "J412", "N306", "N980", "K689"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"P442","units":19,"price":7},{"id":"Q977","units":10,"price":9},{"id":"X773","units":7,"price":4},{"id":"J935","units":2,"price":5},{"id":"W311","units":4,"price":10}]Checked answer{"total": 301}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"X187","priority":1},{"id":"X635","priority":6},{"id":"B342","priority":5},{"id":"A659","priority":5},{"id":"O297","priority":4}]Checked answer{"ids": ["X635", "A659"]}wrong_schema_or_value
{"ids": ["X187", "B342"]}17 tokenswrong_schema_or_value
{"ids":["X635","B342"]}16 tokenswrong_schema_or_value
{"ids":["X635","B342"]}16 tokenswrong_schema_or_value
{"ids":["X635","B342"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z340 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"H647","city":"Accra","units":0},{"id":"A151","city":"Bern","units":11},{"id":"F766","city":"Oslo","units":1},{"id":"Z340","city":"Perth","units":15},{"id":"C260","city":"Oslo","units":10}]Checked answer{"city": "Perth", "units": 15}Correct
{"city":"Perth","units":15}12 tokensCorrect
{"city":"Perth","units":15}12 tokensCorrect
{"city":"Perth","units":15}12 tokensCorrect
{"city":"Perth","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"F341","status":"closed"},{"id":"L150","status":"pending"},{"id":"S426","status":"pending"},{"id":"C206","status":"ready"},{"id":"J396","status":"ready"}]Checked answer{"ids": ["C206", "J396"]}wrong_schema_or_value
{
"ids": ["C206", "F341", "J396", "L150", "S426"]
}38 tokensCorrect
{"ids":["C206","J396"]}16 tokensCorrect
{"ids":["C206","J396"]}16 tokensCorrect
{"ids":["C206","J396"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"G873","units":6,"price":14},{"id":"K725","units":18,"price":5},{"id":"G299","units":14,"price":12},{"id":"V568","units":5,"price":8},{"id":"B310","units":19,"price":18}]Checked answer{"total": 724}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"O538","priority":3},{"id":"W661","priority":7},{"id":"H479","priority":3},{"id":"C420","priority":5},{"id":"C455","priority":5}]Checked answer{"ids": ["W661", "C420"]}invalid_json_object
[{"id":"O538","priority":3},{"id":"H479","priority":3},{"id":"C420","priority":5},{"id":"C455","priority":5}]50 tokensCorrect
{"ids":["W661","C420"]}16 tokensCorrect
{"ids":["W661","C420"]}16 tokensCorrect
{"ids":["W661","C420"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to M313? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"P390","city":"Bern","units":9},{"id":"Z797","city":"Perth","units":10},{"id":"L396","city":"Lima","units":5},{"id":"K305","city":"Kyoto","units":8},{"id":"M313","city":"Bern","units":6}]Checked answer{"city": "Bern", "units": 6}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Bern","units":6}10 tokenswrong_schema_or_value
{"city":null,"units":6}10 tokensCorrect
{"city":"Bern","units":6}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"C147","status":"ready"},{"id":"S669","status":"pending"},{"id":"B992","status":"closed"},{"id":"T822","status":"pending"},{"id":"N544","status":"ready"}]Checked answer{"ids": ["C147", "N544"]}invalid_json_object
[{"id":"C147"},{"id":"N544"},{"id":"S669"},{"id":"T822"},{"id":"B992"}]43 tokensCorrect
{"ids":["C147","N544"]}16 tokensCorrect
{"ids":["C147","N544"]}16 tokensinvalid_json_object
["ids":["C147","N544"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"J992","units":17,"price":13},{"id":"U696","units":17,"price":2},{"id":"C353","units":4,"price":9},{"id":"E813","units":18,"price":8},{"id":"G409","units":15,"price":8}]Checked answer{"total": 555}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"V442","priority":7},{"id":"J863","priority":1},{"id":"W572","priority":6},{"id":"R251","priority":8},{"id":"A646","priority":7}]Checked answer{"ids": ["R251", "A646"]}wrong_schema_or_value
{"ids": ["V442", "J863", "W572", "R251", "A646"]}35 tokensCorrect
{"ids":["R251","A646"]}16 tokenswrong_schema_or_value
{"ids":["R251","V442"]}16 tokenswrong_schema_or_value
{"ids":["R251","V442"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=T204. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"Y524","city":"Kyoto","units":15},{"id":"W903","city":"Perth","units":5},{"id":"J804","city":"Kyoto","units":16},{"id":"T204","city":"Riga","units":16},{"id":"F370","city":"Kyoto","units":0}]Checked answer{"city": "Riga", "units": 16}wrong_schema_or_value
{"city":"Kyoto","units":15}13 tokensCorrect
{"city":"Riga","units":16}12 tokensCorrect
{"city":"Riga","units":16}12 tokensCorrect
{"city":"Riga","units":16}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"B289","status":"ready"},{"id":"P240","status":"closed"},{"id":"F334","status":"closed"},{"id":"I746","status":"pending"},{"id":"Q124","status":"pending"}]Checked answer{"ids": ["B289"]}wrong_schema_or_value
{"ids": ["B289", "P240", "F334", "I746", "Q124"]}35 tokensCorrect
{"ids":["B289"]}11 tokensCorrect
{"ids":["B289"]}11 tokensCorrect
{"ids":["B289"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"O752","units":5,"price":3},{"id":"S200","units":8,"price":1},{"id":"A571","units":15,"price":9},{"id":"F803","units":1,"price":12},{"id":"G990","units":9,"price":12}]Checked answer{"total": 278}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"B952","priority":5},{"id":"X238","priority":6},{"id":"I383","priority":7},{"id":"C965","priority":7},{"id":"Y256","priority":6}]Checked answer{"ids": ["C965", "I383"]}invalid_json_object
[{"id":"B952","priority":5},{"id":"Y256","priority":6},{"id":"C965","priority":7},{"id":"I383","priority":7}]50 tokenswrong_schema_or_value
{"ids":["I383","C965"]}16 tokenswrong_schema_or_value
{"ids":["I383","C965"]}16 tokenswrong_schema_or_value
{"ids":["I383","C965"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record P160. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Q815","city":"Suva","units":11},{"id":"P160","city":"Bern","units":3},{"id":"F178","city":"Lima","units":7},{"id":"O385","city":"Perth","units":7},{"id":"V641","city":"Oslo","units":12}]Checked answer{"city": "Bern", "units": 3}wrong_schema_or_value
{
"city": "Suva",
"units": 11
}19 tokensCorrect
{"city":"Bern","units":3}10 tokensCorrect
{"city":"Bern","units":3}10 tokensCorrect
{"city":"Bern","units":3}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"I312","status":"pending"},{"id":"D544","status":"ready"},{"id":"S508","status":"pending"},{"id":"B303","status":"pending"},{"id":"L531","status":"ready"}]Checked answer{"ids": ["D544", "L531"]}wrong_schema_or_value
{"ids": ["B303", "D544", "I312", "L531", "S508"]}35 tokensCorrect
{"ids":["D544","L531"]}16 tokenswrong_schema_or_value
{"ids":["L531","D544"]}16 tokensCorrect
{"ids":["D544","L531"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"U656","units":2,"price":7},{"id":"C807","units":4,"price":8},{"id":"B118","units":2,"price":15},{"id":"M345","units":5,"price":7},{"id":"H391","units":13,"price":19}]Checked answer{"total": 358}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"N268","priority":9},{"id":"Z154","priority":7},{"id":"D537","priority":2},{"id":"M942","priority":9},{"id":"E863","priority":2}]Checked answer{"ids": ["M942", "N268"]}wrong_schema_or_value
{"ids": ["M942", "D537", "N268", "Z154", "E863"]}35 tokensCorrect
{"ids":["M942","N268"]}16 tokensCorrect
{"ids":["M942","N268"]}16 tokenswrong_schema_or_value
{"ids":["N268","M942"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"K619","city":"Bern","units":11},{"id":"Q484","city":"Kyoto","units":2},{"id":"N336","city":"Lima","units":17},{"id":"E502","city":"Bern","units":14},{"id":"Z860","city":"Riga","units":6}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Bern","units":11}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"Y767","status":"closed"},{"id":"Q423","status":"closed"},{"id":"J328","status":"pending"},{"id":"H330","status":"pending"},{"id":"R950","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{
"ids": ["J328", "H330", "R950"]
}26 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"U789","units":16,"price":17},{"id":"J535","units":16,"price":9},{"id":"L648","units":4,"price":17},{"id":"W766","units":7,"price":11},{"id":"G449","units":16,"price":14}]Checked answer{"total": 785}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Q752","priority":8},{"id":"J226","priority":4},{"id":"K925","priority":4},{"id":"E941","priority":4},{"id":"O758","priority":2}]Checked answer{"ids": ["Q752", "E941"]}invalid_json_object
[{"id":"Q752","priority":8},{"id":"J226","priority":4},{"id":"K925","priority":4},{"id":"E941","priority":4},{"id":"O758","priority":2}]62 tokenswrong_schema_or_value
{"ids":["Q752","J226"]}16 tokenswrong_schema_or_value
{"ids":["Q752","J226"]}16 tokenswrong_schema_or_value
{"ids":["Q752","J226"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to L273? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"N614","city":"Lima","units":0},{"id":"E714","city":"Oslo","units":1},{"id":"U315","city":"Riga","units":7},{"id":"N742","city":"Perth","units":3},{"id":"L273","city":"Suva","units":5}]Checked answer{"city": "Suva", "units": 5}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Suva","units":5}11 tokensCorrect
{"city":"Suva","units":5}11 tokensCorrect
{"city":"Suva","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"H360","status":"pending"},{"id":"G842","status":"closed"},{"id":"T210","status":"pending"},{"id":"O859","status":"closed"},{"id":"E876","status":"pending"}]Checked answer{"ids": []}invalid_json_object
[{"id":"H360"},{"id":"G842"},{"id":"T210"},{"id":"O859"},{"id":"E876"}]43 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensinvalid_json_object
["ids":[]}6 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Q978","units":14,"price":19},{"id":"O917","units":14,"price":19},{"id":"P126","units":6,"price":16},{"id":"F599","units":12,"price":12},{"id":"M857","units":13,"price":10}]Checked answer{"total": 902}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"S193","priority":4},{"id":"P550","priority":9},{"id":"C986","priority":8},{"id":"Y855","priority":7},{"id":"Z207","priority":2}]Checked answer{"ids": ["P550", "C986"]}wrong_schema_or_value
{"ids": ["Y855", "Z207", "S193", "P550", "C986"]}35 tokensCorrect
{"ids":["P550","C986"]}16 tokensCorrect
{"ids":["P550","C986"]}16 tokenswrong_schema_or_value
{"ids":["P550","S193"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=P476. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"A553","city":"Accra","units":17},{"id":"A991","city":"Kyoto","units":16},{"id":"R386","city":"Oslo","units":16},{"id":"P476","city":"Kyoto","units":16},{"id":"S444","city":"Kyoto","units":9}]Checked answer{"city": "Kyoto", "units": 16}Correct
{"city": "Kyoto", "units": 16}16 tokensCorrect
{"city":"Kyoto","units":16}13 tokensCorrect
{"city":"Kyoto","units":16}13 tokensCorrect
{"city":"Kyoto","units":16}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"D809","status":"pending"},{"id":"T751","status":"pending"},{"id":"D769","status":"pending"},{"id":"T254","status":"ready"},{"id":"N536","status":"pending"}]Checked answer{"ids": ["T254"]}wrong_schema_or_value
{"ids": ["D769", "D809", "T254", "T751", "N536"]}35 tokensCorrect
{"ids":["T254"]}11 tokensCorrect
{"ids":["T254"]}11 tokensCorrect
{"ids":["T254"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"D773","units":14,"price":2},{"id":"U803","units":16,"price":11},{"id":"H679","units":18,"price":16},{"id":"C139","units":17,"price":13},{"id":"E265","units":5,"price":6}]Checked answer{"total": 743}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"L930","priority":3},{"id":"X410","priority":7},{"id":"X846","priority":3},{"id":"P230","priority":8},{"id":"R829","priority":7}]Checked answer{"ids": ["P230", "R829"]}invalid_json_object
[{"id":"L930","priority":3},{"id":"X846","priority":3},{"id":"X410","priority":7},{"id":"P230","priority":8}]50 tokensCorrect
{"ids":["P230","R829"]}16 tokensCorrect
{"ids":["P230","R829"]}16 tokensCorrect
{"ids":["P230","R829"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record G347. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"G347","city":"Lima","units":2},{"id":"E248","city":"Oslo","units":0},{"id":"Z390","city":"Perth","units":5},{"id":"S811","city":"Riga","units":2},{"id":"M101","city":"Lima","units":4}]Checked answer{"city": "Lima", "units": 2}invalid_json_object
[{"city":"Lima","units":2}]12 tokensCorrect
{"city":"Lima","units":2}11 tokensCorrect
{"city":"Lima","units":2}11 tokensCorrect
{"city":"Lima","units":2}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"U215","status":"pending"},{"id":"F598","status":"pending"},{"id":"K966","status":"closed"},{"id":"B688","status":"pending"},{"id":"M852","status":"ready"}]Checked answer{"ids": ["M852"]}wrong_schema_or_value
{"ids": ["B688", "F598", "K966", "M852", "U215"]}35 tokensCorrect
{"ids":["M852"]}11 tokensCorrect
{"ids":["M852"]}11 tokensCorrect
{"ids":["M852"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"X364","units":19,"price":17},{"id":"F459","units":0,"price":15},{"id":"F541","units":16,"price":15},{"id":"E457","units":16,"price":7},{"id":"Q468","units":11,"price":15}]Checked answer{"total": 840}invalid_json_object
{"total": 17 * 19 + 15 * 0 + 15 * 16 + 7 * 16 + 15 * 11}42 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"X881","priority":1},{"id":"U772","priority":7},{"id":"E876","priority":5},{"id":"U209","priority":5},{"id":"Z595","priority":3}]Checked answer{"ids": ["U772", "E876"]}wrong_schema_or_value
{"ids": ["U772", "U209", "X881", "Z595", "E876"]}35 tokensCorrect
{"ids":["U772","E876"]}16 tokenswrong_schema_or_value
{"ids":["E876","U209"]}16 tokenswrong_schema_or_value
{"ids":["E772","U209"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Y893 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"C243","city":"Kyoto","units":15},{"id":"I896","city":"Suva","units":0},{"id":"W101","city":"Suva","units":7},{"id":"Y893","city":"Perth","units":12},{"id":"L328","city":"Lima","units":5}]Checked answer{"city": "Perth", "units": 12}Correct
{"city":"Perth","units":12}12 tokensCorrect
{"city":"Perth","units":12}12 tokensCorrect
{"city":"Perth","units":12}12 tokensCorrect
{"city":"Perth","units":12}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"V178","status":"pending"},{"id":"M682","status":"pending"},{"id":"E570","status":"ready"},{"id":"I740","status":"ready"},{"id":"S576","status":"ready"}]Checked answer{"ids": ["E570", "I740", "S576"]}wrong_schema_or_value
{
"ids": ["I740", "E570", "S576", "V178", "M682"]
}38 tokenswrong_schema_or_value
{"ids":["I740","S576","E570"]}21 tokensCorrect
{"ids":["E570","I740","S576"]}21 tokenswrong_schema_or_value
{"ids":["E570","S576","I740"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"H157","units":15,"price":19},{"id":"X416","units":7,"price":17},{"id":"X824","units":11,"price":5},{"id":"V129","units":15,"price":19},{"id":"C709","units":16,"price":7}]Checked answer{"total": 856}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"V778","priority":5},{"id":"Y851","priority":3},{"id":"Q616","priority":9},{"id":"M808","priority":9},{"id":"G370","priority":2}]Checked answer{"ids": ["M808", "Q616"]}invalid_json_object
[{"id":"V778","priority":5},{"id":"Y851","priority":3},{"id":"Q616","priority":9},{"id":"M808","priority":9},{"id":"G370","priority":2}]62 tokensCorrect
{"ids":["M808","Q616"]}16 tokensCorrect
{"ids":["M808","Q616"]}16 tokenswrong_schema_or_value
{"ids":["Q616","M808"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"K545","city":"Perth","units":18},{"id":"N564","city":"Riga","units":15},{"id":"Q959","city":"Oslo","units":1},{"id":"Z916","city":"Lima","units":6},{"id":"W746","city":"Lima","units":18}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"B193","status":"closed"},{"id":"A299","status":"closed"},{"id":"V659","status":"ready"},{"id":"R880","status":"ready"},{"id":"L731","status":"pending"}]Checked answer{"ids": ["R880", "V659"]}invalid_json_object
[{"id":"V659"}]11 tokensCorrect
{"ids":["R880","V659"]}16 tokensCorrect
{"ids":["R880","V659"]}16 tokensinvalid_json_object
["ids":["R880","V659"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Q341","units":17,"price":8},{"id":"P882","units":10,"price":18},{"id":"O124","units":13,"price":17},{"id":"V143","units":3,"price":14},{"id":"Q335","units":2,"price":5}]Checked answer{"total": 589}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"V694","priority":2},{"id":"S869","priority":5},{"id":"M166","priority":7},{"id":"W428","priority":4},{"id":"N578","priority":2}]Checked answer{"ids": ["M166", "S869"]}wrong_schema_or_value
{"ids": ["V694", "S869", "M166", "W428", "N578"]}35 tokensCorrect
{"ids":["M166","S869"]}16 tokensCorrect
{"ids":["M166","S869"]}16 tokensCorrect
{"ids":["M166","S869"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=O121. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"J401","city":"Kyoto","units":5},{"id":"X253","city":"Lima","units":6},{"id":"O905","city":"Perth","units":19},{"id":"O121","city":"Kyoto","units":11},{"id":"I832","city":"Oslo","units":7}]Checked answer{"city": "Kyoto", "units": 11}Correct
{"city":"Kyoto","units":11}13 tokensCorrect
{"city":"Kyoto","units":11}13 tokensCorrect
{"city":"Kyoto","units":11}13 tokensCorrect
{"city":"Kyoto","units":11}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"F278","status":"closed"},{"id":"B734","status":"closed"},{"id":"M443","status":"pending"},{"id":"F991","status":"closed"},{"id":"N947","status":"ready"}]Checked answer{"ids": ["N947"]}wrong_schema_or_value
{"ids": ["F278", "B734", "M443", "F991", "N947"]}35 tokensCorrect
{"ids":["N947"]}11 tokensCorrect
{"ids":["N947"]}11 tokensCorrect
{"ids":["N947"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"C323","units":0,"price":5},{"id":"T414","units":11,"price":19},{"id":"R547","units":11,"price":18},{"id":"N192","units":19,"price":17},{"id":"I645","units":16,"price":12}]Checked answer{"total": 922}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"S187","priority":5},{"id":"D752","priority":1},{"id":"C335","priority":3},{"id":"H973","priority":5},{"id":"W921","priority":1}]Checked answer{"ids": ["H973", "S187"]}invalid_json_object
[{"id":"S187"},{"id":"D752"},{"id":"C335"},{"id":"H973"},{"id":"W921"}]43 tokensCorrect
{"ids":["H973","S187"]}16 tokensCorrect
{"ids":["H973","S187"]}16 tokensCorrect
{"ids":["H973","S187"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record A871. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"U778","city":"Oslo","units":7},{"id":"N909","city":"Perth","units":15},{"id":"A871","city":"Oslo","units":15},{"id":"Q929","city":"Riga","units":5},{"id":"S938","city":"Lima","units":11}]Checked answer{"city": "Oslo", "units": 15}Correct
{
"city": "Oslo",
"units": 15
}19 tokensCorrect
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Oslo","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"T628","status":"closed"},{"id":"O782","status":"closed"},{"id":"S281","status":"closed"},{"id":"B348","status":"closed"},{"id":"O475","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["O475", "O782", "S281", "T628", "B348"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"T485","units":0,"price":18},{"id":"X388","units":15,"price":16},{"id":"F418","units":13,"price":12},{"id":"F470","units":8,"price":6},{"id":"G364","units":12,"price":9}]Checked answer{"total": 552}wrong_schema_or_value
{"total": 120}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Z698","priority":8},{"id":"G956","priority":3},{"id":"N334","priority":3},{"id":"S434","priority":1},{"id":"C515","priority":7}]Checked answer{"ids": ["Z698", "C515"]}wrong_schema_or_value
{"ids": ["G956", "N334", "S434", "C515", "Z698"]}35 tokensCorrect
{"ids":["Z698","C515"]}16 tokensCorrect
{"ids":["Z698","C515"]}16 tokensCorrect
{"ids":["Z698","C515"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id M962 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"L979","city":"Lima","units":13},{"id":"D611","city":"Perth","units":8},{"id":"G632","city":"Kyoto","units":6},{"id":"H350","city":"Lima","units":10},{"id":"M962","city":"Perth","units":3}]Checked answer{"city": "Perth", "units": 3}wrong_schema_or_value
{"city":"Lima","units":13}12 tokensCorrect
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Perth","units":3}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"A791","status":"pending"},{"id":"X884","status":"ready"},{"id":"Z283","status":"closed"},{"id":"N126","status":"ready"},{"id":"P864","status":"pending"}]Checked answer{"ids": ["N126", "X884"]}wrong_schema_or_value
{
"ids": ["A791", "X884", "Z283", "N126", "P864"]
}38 tokensCorrect
{"ids":["N126","X884"]}16 tokenswrong_schema_or_value
{"ids":["A791","N126","N126"]}21 tokensCorrect
{"ids":["N126","X884"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"D710","units":0,"price":8},{"id":"C263","units":9,"price":11},{"id":"L506","units":10,"price":9},{"id":"P766","units":7,"price":11},{"id":"O554","units":3,"price":8}]Checked answer{"total": 290}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"N960","priority":8},{"id":"F306","priority":2},{"id":"W444","priority":1},{"id":"J965","priority":8},{"id":"Y147","priority":5}]Checked answer{"ids": ["J965", "N960"]}invalid_json_object
[{"id":"N960","priority":8},{"id":"F306","priority":2},{"id":"W444","priority":1},{"id":"J965","priority":8},{"id":"Y147","priority":5}]62 tokenswrong_schema_or_value
{"ids":["N960","J965"]}16 tokenswrong_schema_or_value
{"ids":["N960","J965"]}16 tokensCorrect
{"ids":["J965","N960"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to D638? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"F499","city":"Perth","units":1},{"id":"H881","city":"Oslo","units":7},{"id":"R144","city":"Accra","units":12},{"id":"U906","city":"Riga","units":9},{"id":"D638","city":"Lima","units":15}]Checked answer{"city": "Lima", "units": 15}wrong_schema_or_value
{
"city": null,
"units": 15
}17 tokensCorrect
{"city":"Lima","units":15}12 tokenswrong_schema_or_value
{"city":null,"units":15}11 tokensCorrect
{"city":"Lima","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"D536","status":"ready"},{"id":"P407","status":"pending"},{"id":"W196","status":"pending"},{"id":"D146","status":"pending"},{"id":"O307","status":"ready"}]Checked answer{"ids": ["D536", "O307"]}invalid_json_object
[{"id":"D536"},{"id":"O307"},{"id":"P407"},{"id":"W196"},{"id":"D146"}]43 tokensCorrect
{"ids":["D536","O307"]}16 tokensCorrect
{"ids":["D536","O307"]}16 tokensinvalid_json_object
["ids":["D536","O307"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"S915","units":18,"price":11},{"id":"V193","units":6,"price":5},{"id":"R380","units":18,"price":11},{"id":"D466","units":13,"price":4},{"id":"N366","units":0,"price":4}]Checked answer{"total": 478}wrong_schema_or_value
{"total": 110}9 tokenswrong_schema_or_value
{"total":588}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"X588","priority":7},{"id":"R766","priority":5},{"id":"P941","priority":4},{"id":"J337","priority":5},{"id":"N160","priority":5}]Checked answer{"ids": ["X588", "J337"]}wrong_schema_or_value
{"ids": ["X588", "R766", "P941", "J337", "N160"]}35 tokensCorrect
{"ids":["X588","J337"]}16 tokensCorrect
{"ids":["X588","J337"]}16 tokenswrong_schema_or_value
{"ids":["X588","R766"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"R600","city":"Suva","units":19},{"id":"C880","city":"Lima","units":17},{"id":"W228","city":"Suva","units":18},{"id":"B269","city":"Perth","units":13},{"id":"O598","city":"Bern","units":3}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Suva","units":19}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"S394","status":"ready"},{"id":"M936","status":"closed"},{"id":"L704","status":"closed"},{"id":"C562","status":"pending"},{"id":"P679","status":"closed"}]Checked answer{"ids": ["S394"]}wrong_schema_or_value
{"ids": ["C562", "L704", "M936", "S394", "P679"]}35 tokensCorrect
{"ids":["S394"]}11 tokensCorrect
{"ids":["S394"]}11 tokensCorrect
{"ids":["S394"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"F737","units":11,"price":13},{"id":"W625","units":9,"price":2},{"id":"T573","units":17,"price":3},{"id":"H894","units":11,"price":11},{"id":"I942","units":14,"price":8}]Checked answer{"total": 445}invalid_json_object
{
"total": 13 + 2 + 3 + 11 + 8 = 31
}28 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"Z872","priority":7},{"id":"M596","priority":5},{"id":"A963","priority":9},{"id":"G378","priority":6},{"id":"P687","priority":8}]Checked answer{"ids": ["A963", "P687"]}invalid_json_object
[{"id":"Z872","priority":7},{"id":"M596","priority":5},{"id":"A963","priority":9},{"id":"G378","priority":6},{"id":"P687","priority":8}]62 tokensCorrect
{"ids":["A963","P687"]}16 tokensCorrect
{"ids":["A963","P687"]}16 tokenswrong_schema_or_value
{"ids":["A963","Z872"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record F114. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"B104","city":"Perth","units":1},{"id":"L711","city":"Perth","units":7},{"id":"W928","city":"Accra","units":6},{"id":"F114","city":"Kyoto","units":4},{"id":"F263","city":"Riga","units":12}]Checked answer{"city": "Kyoto", "units": 4}Correct
{
"city": "Kyoto",
"units": 4
}19 tokensCorrect
{"city":"Kyoto","units":4}12 tokensCorrect
{"city":"Kyoto","units":4}12 tokensCorrect
{"city":"Kyoto","units":4}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"L238","status":"ready"},{"id":"E257","status":"pending"},{"id":"F501","status":"closed"},{"id":"D522","status":"pending"},{"id":"B746","status":"pending"}]Checked answer{"ids": ["L238"]}wrong_schema_or_value
{"ids": ["E257", "F501", "D522", "L238"]}29 tokensCorrect
{"ids":["L238"]}11 tokensCorrect
{"ids":["L238"]}11 tokensCorrect
{"ids":["L238"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"C279","units":18,"price":15},{"id":"Q998","units":15,"price":5},{"id":"R775","units":5,"price":17},{"id":"W714","units":3,"price":17},{"id":"O259","units":5,"price":3}]Checked answer{"total": 496}wrong_schema_or_value
{"total": 150}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"E142","priority":9},{"id":"I869","priority":8},{"id":"I282","priority":6},{"id":"I720","priority":7},{"id":"B842","priority":2}]Checked answer{"ids": ["E142", "I869"]}wrong_schema_or_value
{"ids": ["I869", "I282", "I720", "E142"]}29 tokensCorrect
{"ids":["E142","I869"]}16 tokensCorrect
{"ids":["E142","I869"]}16 tokensCorrect
{"ids":["E142","I869"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id E880 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"M896","city":"Suva","units":3},{"id":"Q807","city":"Lima","units":1},{"id":"N929","city":"Riga","units":4},{"id":"E880","city":"Suva","units":2},{"id":"I973","city":"Bern","units":0}]Checked answer{"city": "Suva", "units": 2}Correct
{"city":"Suva","units":2}11 tokensCorrect
{"city":"Suva","units":2}11 tokensCorrect
{"city":"Suva","units":2}11 tokensCorrect
{"city":"Suva","units":2}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"U411","status":"closed"},{"id":"E626","status":"closed"},{"id":"Y475","status":"pending"},{"id":"L652","status":"ready"},{"id":"A220","status":"ready"}]Checked answer{"ids": ["A220", "L652"]}wrong_schema_or_value
{
"ids": ["A220", "E626", "L652", "U411"]
}32 tokensCorrect
{"ids":["A220","L652"]}16 tokensCorrect
{"ids":["A220","L652"]}16 tokensCorrect
{"ids":["A220","L652"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"H581","units":11,"price":11},{"id":"A916","units":1,"price":1},{"id":"Z638","units":11,"price":15},{"id":"C998","units":2,"price":9},{"id":"B164","units":19,"price":17}]Checked answer{"total": 628}wrong_schema_or_value
{"total": 120}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":468}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"J443","priority":6},{"id":"L164","priority":9},{"id":"A347","priority":3},{"id":"A894","priority":7},{"id":"D868","priority":9}]Checked answer{"ids": ["D868", "L164"]}invalid_json_object
[{"id":"L164","priority":9},{"id":"A347","priority":3},{"id":"A894","priority":7},{"id":"J443","priority":6}]50 tokensCorrect
{"ids":["D868","L164"]}16 tokensCorrect
{"ids":["D868","L164"]}16 tokensCorrect
{"ids":["D868","L164"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to S838? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"B491","city":"Bern","units":14},{"id":"S838","city":"Suva","units":19},{"id":"Z543","city":"Lima","units":11},{"id":"Y103","city":"Lima","units":0},{"id":"V110","city":"Suva","units":6}]Checked answer{"city": "Suva", "units": 19}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Suva","units":19}12 tokensCorrect
{"city":"Suva","units":19}12 tokensCorrect
{"city":"Suva","units":19}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"J590","status":"closed"},{"id":"L759","status":"pending"},{"id":"O860","status":"ready"},{"id":"G182","status":"pending"},{"id":"N407","status":"ready"}]Checked answer{"ids": ["N407", "O860"]}invalid_json_object
[{"id":"O860","status":"ready"},{"id":"G182","status":"pending"},{"id":"N407","status":"ready"},{"id":"J590","status":"closed"},{"id":"L759","status":"pending"}]63 tokensCorrect
{"ids":["N407","O860"]}16 tokensCorrect
{"ids":["N407","O860"]}16 tokensinvalid_json_object
["ids":["O860","N407"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Y278","units":17,"price":11},{"id":"S560","units":14,"price":14},{"id":"D659","units":12,"price":13},{"id":"I391","units":9,"price":8},{"id":"F534","units":13,"price":5}]Checked answer{"total": 676}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Z798","priority":7},{"id":"S388","priority":8},{"id":"K517","priority":1},{"id":"O855","priority":2},{"id":"O578","priority":7}]Checked answer{"ids": ["S388", "O578"]}wrong_schema_or_value
{"ids": ["O855", "O578", "K517", "S388", "Z798"]}35 tokenswrong_schema_or_value
{"ids":["S388","Z798"]}16 tokenswrong_schema_or_value
{"ids":["S388","Z798"]}16 tokenswrong_schema_or_value
{"ids":["S388","Z798"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=T638. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"T554","city":"Suva","units":10},{"id":"T638","city":"Perth","units":3},{"id":"X343","city":"Suva","units":6},{"id":"V141","city":"Bern","units":12},{"id":"O222","city":"Suva","units":8}]Checked answer{"city": "Perth", "units": 3}wrong_schema_or_value
{"city":"Suva","units":10}12 tokensCorrect
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Perth","units":3}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"G384","status":"ready"},{"id":"C471","status":"closed"},{"id":"D204","status":"ready"},{"id":"U425","status":"closed"},{"id":"C956","status":"closed"}]Checked answer{"ids": ["D204", "G384"]}wrong_schema_or_value
{"ids": ["C471", "C956", "D204", "G384", "U425"]}35 tokensCorrect
{"ids":["D204","G384"]}16 tokensCorrect
{"ids":["D204","G384"]}16 tokensCorrect
{"ids":["D204","G384"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"E826","units":5,"price":10},{"id":"S842","units":4,"price":7},{"id":"Z329","units":10,"price":2},{"id":"X150","units":1,"price":8},{"id":"W141","units":16,"price":5}]Checked answer{"total": 186}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":288}8 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"C933","priority":3},{"id":"G488","priority":5},{"id":"D602","priority":7},{"id":"Z877","priority":6},{"id":"U289","priority":4}]Checked answer{"ids": ["D602", "Z877"]}invalid_json_object
[{"id":"G488","priority":5},{"id":"U289","priority":4},{"id":"C933","priority":3},{"id":"D602","priority":7},{"id":"Z877","priority":6}]62 tokensCorrect
{"ids":["D602","Z877"]}16 tokenswrong_schema_or_value
{"ids":["D602","G488"]}16 tokenswrong_schema_or_value
{"ids":["D602","C933"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Y434","city":"Oslo","units":2},{"id":"S806","city":"Kyoto","units":14},{"id":"I427","city":"Suva","units":15},{"id":"D177","city":"Riga","units":7},{"id":"K914","city":"Bern","units":12}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{
"city": "Oslo",
"units": 2
}18 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Y478","status":"closed"},{"id":"P760","status":"pending"},{"id":"P323","status":"pending"},{"id":"G864","status":"pending"},{"id":"P830","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["P323", "P760", "P830", "Y478"]}29 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"U287","units":17,"price":9},{"id":"X699","units":0,"price":5},{"id":"F543","units":17,"price":6},{"id":"D965","units":15,"price":19},{"id":"H133","units":4,"price":3}]Checked answer{"total": 552}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":548}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"U653","priority":6},{"id":"A629","priority":1},{"id":"U170","priority":1},{"id":"L630","priority":3},{"id":"U581","priority":4}]Checked answer{"ids": ["U653", "U581"]}wrong_schema_or_value
{"ids": ["U653", "A629", "U170", "L630", "U581"]}35 tokenswrong_schema_or_value
{"ids":["U653","L630"]}16 tokenswrong_schema_or_value
{"ids":["U653","L630"]}16 tokenswrong_schema_or_value
{"ids":["U653","A629"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id K216 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"U601","city":"Kyoto","units":4},{"id":"Z984","city":"Riga","units":5},{"id":"B496","city":"Lima","units":12},{"id":"W403","city":"Riga","units":14},{"id":"K216","city":"Riga","units":14}]Checked answer{"city": "Riga", "units": 14}wrong_schema_or_value
{"city":"Kyoto","units":4}12 tokensCorrect
{"city":"Riga","units":14}12 tokensCorrect
{"city":"Riga","units":14}12 tokensCorrect
{"city":"Riga","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"O961","status":"ready"},{"id":"K352","status":"pending"},{"id":"R482","status":"pending"},{"id":"T878","status":"pending"},{"id":"V128","status":"closed"}]Checked answer{"ids": ["O961"]}wrong_schema_or_value
{
"ids": ["O961", "K352", "R482", "T878", "V128"]
}38 tokensCorrect
{"ids":["O961"]}11 tokensCorrect
{"ids":["O961"]}11 tokensCorrect
{"ids":["O961"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"X589","units":8,"price":8},{"id":"S345","units":18,"price":4},{"id":"Y771","units":15,"price":10},{"id":"Z446","units":13,"price":17},{"id":"V956","units":4,"price":1}]Checked answer{"total": 511}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"W538","priority":3},{"id":"S553","priority":1},{"id":"G580","priority":8},{"id":"I866","priority":2},{"id":"D557","priority":4}]Checked answer{"ids": ["G580", "D557"]}invalid_json_object
[{"id":"W538","priority":3},{"id":"S553","priority":1},{"id":"G580","priority":8},{"id":"I866","priority":2},{"id":"D557","priority":4}]62 tokensCorrect
{"ids":["G580","D557"]}16 tokenswrong_schema_or_value
{"ids":["G580","W538"]}16 tokensCorrect
{"ids":["G580","D557"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to K883? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"W814","city":"Riga","units":15},{"id":"O602","city":"Bern","units":4},{"id":"F680","city":"Suva","units":10},{"id":"K883","city":"Kyoto","units":13},{"id":"P838","city":"Riga","units":16}]Checked answer{"city": "Kyoto", "units": 13}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Kyoto","units":13}13 tokensCorrect
{"city":"Kyoto","units":13}13 tokensCorrect
{"city":"Kyoto","units":13}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"L366","status":"closed"},{"id":"E286","status":"pending"},{"id":"G796","status":"closed"},{"id":"M651","status":"pending"},{"id":"X935","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"E286"},{"id":"G796"},{"id":"M651"},{"id":"X935"}]35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"M282","units":19,"price":3},{"id":"I232","units":9,"price":18},{"id":"H540","units":6,"price":10},{"id":"U130","units":18,"price":16},{"id":"P228","units":2,"price":17}]Checked answer{"total": 601}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":678}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Q224","priority":8},{"id":"L907","priority":6},{"id":"O734","priority":6},{"id":"O484","priority":9},{"id":"J685","priority":1}]Checked answer{"ids": ["O484", "Q224"]}wrong_schema_or_value
{"ids": ["O484", "O734", "J685", "Q224", "L907"]}35 tokensCorrect
{"ids":["O484","Q224"]}16 tokensCorrect
{"ids":["O484","Q224"]}16 tokensCorrect
{"ids":["O484","Q224"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=X306. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"W459","city":"Perth","units":11},{"id":"H626","city":"Accra","units":17},{"id":"X306","city":"Accra","units":15},{"id":"G955","city":"Oslo","units":8},{"id":"A700","city":"Lima","units":10}]Checked answer{"city": "Accra", "units": 15}wrong_schema_or_value
{"city":"Perth","units":11}12 tokensCorrect
{"city":"Accra","units":15}12 tokensCorrect
{"city":"Accra","units":15}12 tokensCorrect
{"city":"Accra","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"C209","status":"pending"},{"id":"N969","status":"pending"},{"id":"Z255","status":"closed"},{"id":"T920","status":"ready"},{"id":"P559","status":"ready"}]Checked answer{"ids": ["P559", "T920"]}wrong_schema_or_value
{"ids": ["C209", "N969", "Z255", "T920", "P559"]}35 tokensCorrect
{"ids":["P559","T920"]}16 tokensCorrect
{"ids":["P559","T920"]}16 tokensCorrect
{"ids":["P559","T920"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"V148","units":11,"price":7},{"id":"Z615","units":13,"price":11},{"id":"L654","units":14,"price":17},{"id":"D840","units":7,"price":19},{"id":"N293","units":13,"price":18}]Checked answer{"total": 825}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"R329","priority":5},{"id":"D655","priority":3},{"id":"M522","priority":1},{"id":"R476","priority":9},{"id":"T222","priority":9}]Checked answer{"ids": ["R476", "T222"]}invalid_json_object
[{"id":"R329","priority":5},{"id":"D655","priority":3},{"id":"M522","priority":1},{"id":"R476","priority":9},{"id":"T222","priority":9}]62 tokenswrong_schema_or_value
{"ids":["T222","R476"]}16 tokensCorrect
{"ids":["R476","T222"]}16 tokensCorrect
{"ids":["R476","T222"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record X754. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Y644","city":"Perth","units":17},{"id":"I689","city":"Accra","units":2},{"id":"L153","city":"Lima","units":3},{"id":"W662","city":"Bern","units":10},{"id":"X754","city":"Perth","units":14}]Checked answer{"city": "Perth", "units": 14}invalid_json_object
[{"city":"Perth","units":17}]13 tokensCorrect
{"city":"Perth","units":14}12 tokensCorrect
{"city":"Perth","units":14}12 tokensCorrect
{"city":"Perth","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"F401","status":"pending"},{"id":"M668","status":"pending"},{"id":"B303","status":"ready"},{"id":"Z470","status":"pending"},{"id":"L188","status":"pending"}]Checked answer{"ids": ["B303"]}wrong_schema_or_value
{"ids": ["B303", "L188", "M668", "F401", "Z470"]}35 tokensCorrect
{"ids":["B303"]}11 tokensCorrect
{"ids":["B303"]}11 tokensCorrect
{"ids":["B303"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"L125","units":2,"price":17},{"id":"R204","units":8,"price":9},{"id":"Q585","units":2,"price":18},{"id":"S566","units":15,"price":8},{"id":"O718","units":10,"price":2}]Checked answer{"total": 282}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"S164","priority":9},{"id":"J255","priority":3},{"id":"F394","priority":4},{"id":"Q367","priority":9},{"id":"J634","priority":4}]Checked answer{"ids": ["Q367", "S164"]}wrong_schema_or_value
{"ids": ["J255", "S164", "J634", "Q367"]}29 tokenswrong_schema_or_value
{"ids":["S164","Q367"]}16 tokenswrong_schema_or_value
{"ids":["S164","Q367"]}16 tokenswrong_schema_or_value
{"ids":["S164","Q367"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"T178","city":"Oslo","units":12},{"id":"D872","city":"Bern","units":16},{"id":"P565","city":"Kyoto","units":11},{"id":"Q808","city":"Accra","units":5},{"id":"X446","city":"Oslo","units":1}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Oslo","units":12}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"N298","status":"ready"},{"id":"C576","status":"closed"},{"id":"S378","status":"pending"},{"id":"W331","status":"ready"},{"id":"Z932","status":"ready"}]Checked answer{"ids": ["N298", "W331", "Z932"]}wrong_schema_or_value
{
"ids": ["N298", "S378", "W331", "Z932"]
}32 tokensCorrect
{"ids":["N298","W331","Z932"]}21 tokenswrong_schema_or_value
{"ids":["C576","N298","W331","Z932"]}26 tokensCorrect
{"ids":["N298","W331","Z932"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"E189","units":3,"price":6},{"id":"F454","units":3,"price":7},{"id":"Z247","units":5,"price":2},{"id":"I758","units":11,"price":16},{"id":"W764","units":0,"price":8}]Checked answer{"total": 225}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":27}7 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"I908","priority":3},{"id":"C825","priority":2},{"id":"U667","priority":6},{"id":"J198","priority":8},{"id":"S933","priority":4}]Checked answer{"ids": ["J198", "U667"]}invalid_json_object
[{"id":"I908","priority":3},{"id":"C825","priority":2},{"id":"J198","priority":8},{"id":"S933","priority":4}]50 tokensCorrect
{"ids":["J198","U667"]}16 tokensCorrect
{"ids":["J198","U667"]}16 tokenswrong_schema_or_value
{"ids":["J198","I908"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to M554? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"M200","city":"Kyoto","units":18},{"id":"M554","city":"Kyoto","units":0},{"id":"H183","city":"Perth","units":0},{"id":"W770","city":"Bern","units":6},{"id":"A369","city":"Bern","units":3}]Checked answer{"city": "Kyoto", "units": 0}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Kyoto","units":0}12 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Kyoto","units":0}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"S249","status":"closed"},{"id":"Z679","status":"closed"},{"id":"N519","status":"ready"},{"id":"D903","status":"ready"},{"id":"I263","status":"ready"}]Checked answer{"ids": ["D903", "I263", "N519"]}invalid_json_object
[{"id":"N519"},{"id":"I263"},{"id":"D903"},{"id":"S249"},{"id":"Z679"}]43 tokensCorrect
{"ids":["D903","I263","N519"]}21 tokensCorrect
{"ids":["D903","I263","N519"]}21 tokensinvalid_json_object
["ids":["I263","N519","D903"]]21 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"K511","units":8,"price":4},{"id":"K865","units":3,"price":18},{"id":"P914","units":17,"price":10},{"id":"Q909","units":2,"price":12},{"id":"K647","units":8,"price":18}]Checked answer{"total": 424}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"I552","priority":4},{"id":"R504","priority":9},{"id":"S676","priority":5},{"id":"V624","priority":9},{"id":"Z417","priority":1}]Checked answer{"ids": ["R504", "V624"]}invalid_json_object
{"ids": ["I552", "V624", "S676", "Z417", "R504", "I552", "V624", "S676", "Z417", "R504", "I552", "V624", "S676", "Z417", "R504", "I596 tokensCorrect
{"ids":["R504","V624"]}16 tokenswrong_schema_or_value
{"ids":["V624","R504"]}16 tokensCorrect
{"ids":["R504","V624"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=N850. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"X702","city":"Riga","units":14},{"id":"C556","city":"Oslo","units":10},{"id":"N850","city":"Perth","units":13},{"id":"X333","city":"Riga","units":17},{"id":"Z123","city":"Oslo","units":6}]Checked answer{"city": "Perth", "units": 13}wrong_schema_or_value
{"city":"Riga","units":14}12 tokensCorrect
{"city":"Perth","units":13}12 tokensCorrect
{"city":"Perth","units":13}12 tokensCorrect
{"city":"Perth","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Z720","status":"pending"},{"id":"S780","status":"closed"},{"id":"L719","status":"pending"},{"id":"A990","status":"ready"},{"id":"U714","status":"ready"}]Checked answer{"ids": ["A990", "U714"]}wrong_schema_or_value
{"ids": ["A990", "L719", "S780", "U714", "Z720"]}35 tokensCorrect
{"ids":["A990","U714"]}16 tokensCorrect
{"ids":["A990","U714"]}16 tokensCorrect
{"ids":["A990","U714"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"A863","units":15,"price":6},{"id":"X847","units":12,"price":17},{"id":"H813","units":5,"price":1},{"id":"I231","units":5,"price":5},{"id":"G444","units":17,"price":19}]Checked answer{"total": 647}wrong_schema_or_value
{
"total": 103
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"R356","priority":7},{"id":"S947","priority":5},{"id":"R131","priority":4},{"id":"G228","priority":2},{"id":"M834","priority":7}]Checked answer{"ids": ["M834", "R356"]}invalid_json_object
[{"id":"R356"},{"id":"S947"},{"id":"R131"},{"id":"G228"},{"id":"M834"}]43 tokensCorrect
{"ids":["M834","R356"]}16 tokensCorrect
{"ids":["M834","R356"]}16 tokenswrong_schema_or_value
{"ids":["R356","M834"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record O150. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"C340","city":"Bern","units":0},{"id":"O150","city":"Lima","units":14},{"id":"W741","city":"Suva","units":12},{"id":"M679","city":"Oslo","units":17},{"id":"Z480","city":"Lima","units":4}]Checked answer{"city": "Lima", "units": 14}Correct
{
"city": "Lima",
"units": 14
}19 tokensCorrect
{"city":"Lima","units":14}12 tokensCorrect
{"city":"Lima","units":14}12 tokensCorrect
{"city":"Lima","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"H573","status":"closed"},{"id":"C679","status":"closed"},{"id":"J494","status":"ready"},{"id":"V782","status":"ready"},{"id":"M373","status":"ready"}]Checked answer{"ids": ["J494", "M373", "V782"]}wrong_schema_or_value
{"ids": ["J494", "M373", "V782", "H573", "C679"]}35 tokensCorrect
{"ids":["J494","M373","V782"]}21 tokensCorrect
{"ids":["J494","M373","V782"]}21 tokensCorrect
{"ids":["J494","M373","V782"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"M738","units":19,"price":13},{"id":"H777","units":4,"price":14},{"id":"L373","units":3,"price":11},{"id":"M348","units":4,"price":12},{"id":"X470","units":6,"price":7}]Checked answer{"total": 426}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"U296","priority":3},{"id":"E694","priority":7},{"id":"M576","priority":7},{"id":"V508","priority":6},{"id":"H419","priority":8}]Checked answer{"ids": ["H419", "E694"]}wrong_schema_or_value
{"ids": ["U296", "E694", "M576", "V508", "H419"]}35 tokenswrong_schema_or_value
{"ids":["H419","M576"]}16 tokenswrong_schema_or_value
{"ids":["H419","M576"]}16 tokensCorrect
{"ids":["H419","E694"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id X240 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"W894","city":"Suva","units":11},{"id":"I861","city":"Suva","units":4},{"id":"Q784","city":"Lima","units":4},{"id":"X240","city":"Oslo","units":9},{"id":"L371","city":"Accra","units":1}]Checked answer{"city": "Oslo", "units": 9}wrong_schema_or_value
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Oslo","units":9}11 tokensCorrect
{"city":"Oslo","units":9}11 tokensCorrect
{"city":"Oslo","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"G903","status":"ready"},{"id":"U576","status":"closed"},{"id":"O132","status":"ready"},{"id":"O925","status":"pending"},{"id":"X173","status":"pending"}]Checked answer{"ids": ["G903", "O132"]}wrong_schema_or_value
{
"ids": ["G903", "O132", "O925", "X173"]
}32 tokensCorrect
{"ids":["G903","O132"]}16 tokensCorrect
{"ids":["G903","O132"]}16 tokensCorrect
{"ids":["G903","O132"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"D148","units":19,"price":13},{"id":"H264","units":1,"price":13},{"id":"J261","units":16,"price":7},{"id":"D968","units":9,"price":14},{"id":"K649","units":18,"price":19}]Checked answer{"total": 840}wrong_schema_or_value
{"total": 102}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"M897","priority":6},{"id":"W634","priority":4},{"id":"C619","priority":6},{"id":"L851","priority":7},{"id":"F655","priority":6}]Checked answer{"ids": ["L851", "C619"]}invalid_json_object
[{"id":"M897","priority":6},{"id":"L851","priority":7}]26 tokensCorrect
{"ids":["L851","C619"]}16 tokenswrong_schema_or_value
{"ids":["L851","M897"]}16 tokensCorrect
{"ids":["L851","C619"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"B822","city":"Suva","units":8},{"id":"O133","city":"Perth","units":11},{"id":"H520","city":"Accra","units":2},{"id":"X790","city":"Bern","units":13},{"id":"G130","city":"Perth","units":6}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"Z133","status":"pending"},{"id":"Q460","status":"closed"},{"id":"O351","status":"ready"},{"id":"P287","status":"ready"},{"id":"D859","status":"closed"}]Checked answer{"ids": ["O351", "P287"]}invalid_json_object
[{"id":"O351","status":"ready"},{"id":"P287","status":"ready"},{"id":"Z133","status":"pending"},{"id":"Q460","status":"closed"},{"id":"D859","status":"closed"}]63 tokenswrong_schema_or_value
{"ids":["P287","O351"]}16 tokensCorrect
{"ids":["O351","P287"]}16 tokensinvalid_json_object
["ids":["O351","P287"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"B774","units":6,"price":11},{"id":"S814","units":4,"price":16},{"id":"P735","units":11,"price":8},{"id":"E122","units":3,"price":18},{"id":"K171","units":13,"price":4}]Checked answer{"total": 324}wrong_schema_or_value
{"total": 120}9 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"E815","priority":8},{"id":"Y181","priority":9},{"id":"J987","priority":4},{"id":"P131","priority":9},{"id":"K646","priority":6}]Checked answer{"ids": ["P131", "Y181"]}wrong_schema_or_value
{"ids": ["E815", "Y181", "J987", "P131", "K646"]}35 tokenswrong_schema_or_value
{"ids":["E815","P131"]}16 tokenswrong_schema_or_value
{"ids":["E815","P131"]}16 tokenswrong_schema_or_value
{"ids":["E815","P131"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Y587. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"X443","city":"Oslo","units":8},{"id":"Y587","city":"Riga","units":1},{"id":"R888","city":"Riga","units":5},{"id":"D593","city":"Perth","units":5},{"id":"U942","city":"Kyoto","units":19}]Checked answer{"city": "Riga", "units": 1}Correct
{"city":"Riga","units":1}11 tokensCorrect
{"city":"Riga","units":1}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Riga","units":1}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"B705","status":"pending"},{"id":"A714","status":"closed"},{"id":"Y205","status":"pending"},{"id":"T222","status":"closed"},{"id":"L145","status":"ready"}]Checked answer{"ids": ["L145"]}wrong_schema_or_value
{"ids": ["A714", "B705", "L145", "T222", "Y205"]}35 tokensCorrect
{"ids":["L145"]}11 tokensCorrect
{"ids":["L145"]}11 tokensCorrect
{"ids":["L145"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"F340","units":11,"price":13},{"id":"A439","units":13,"price":18},{"id":"F646","units":15,"price":14},{"id":"A603","units":8,"price":8},{"id":"O712","units":17,"price":5}]Checked answer{"total": 736}wrong_schema_or_value
{
"total": 42
}11 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"J894","priority":2},{"id":"S965","priority":6},{"id":"J763","priority":3},{"id":"V629","priority":8},{"id":"I751","priority":1}]Checked answer{"ids": ["V629", "S965"]}invalid_json_object
[{"id":"J894"},{"id":"S965"},{"id":"J763"},{"id":"V629"},{"id":"I751"}]43 tokensCorrect
{"ids":["V629","S965"]}16 tokensCorrect
{"ids":["V629","S965"]}16 tokenswrong_schema_or_value
{"ids":["S965","V629"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record H727. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"H727","city":"Bern","units":10},{"id":"B492","city":"Bern","units":9},{"id":"V785","city":"Suva","units":3},{"id":"T608","city":"Oslo","units":15},{"id":"Z444","city":"Oslo","units":12}]Checked answer{"city": "Bern", "units": 10}Correct
{
"city": "Bern",
"units": 10
}18 tokensCorrect
{"city":"Bern","units":10}11 tokensCorrect
{"city":"Bern","units":10}11 tokensCorrect
{"city":"Bern","units":10}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Z803","status":"ready"},{"id":"C998","status":"closed"},{"id":"L706","status":"ready"},{"id":"P334","status":"pending"},{"id":"S679","status":"pending"}]Checked answer{"ids": ["L706", "Z803"]}wrong_schema_or_value
{"ids": ["C998", "L706", "P334", "Z803"]}29 tokensCorrect
{"ids":["L706","Z803"]}16 tokensCorrect
{"ids":["L706","Z803"]}16 tokensCorrect
{"ids":["L706","Z803"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"O291","units":19,"price":4},{"id":"G588","units":4,"price":3},{"id":"L539","units":17,"price":17},{"id":"U660","units":12,"price":9},{"id":"A565","units":7,"price":15}]Checked answer{"total": 590}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"O492","priority":3},{"id":"M351","priority":4},{"id":"E674","priority":9},{"id":"R127","priority":2},{"id":"Y922","priority":9}]Checked answer{"ids": ["E674", "Y922"]}wrong_schema_or_value
{"ids": ["O492", "M351", "R127", "Y922", "E674"]}35 tokensCorrect
{"ids":["E674","Y922"]}16 tokensCorrect
{"ids":["E674","Y922"]}16 tokensCorrect
{"ids":["E674","Y922"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id T280 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"D298","city":"Riga","units":15},{"id":"Y687","city":"Perth","units":11},{"id":"T280","city":"Suva","units":12},{"id":"F488","city":"Kyoto","units":13},{"id":"S826","city":"Accra","units":10}]Checked answer{"city": "Suva", "units": 12}Correct
{"city":"Suva","units":12}12 tokensCorrect
{"city":"Suva","units":12}12 tokensCorrect
{"city":"Suva","units":12}12 tokensCorrect
{"city":"Suva","units":12}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"A716","status":"ready"},{"id":"S991","status":"ready"},{"id":"U866","status":"ready"},{"id":"C837","status":"pending"},{"id":"Q407","status":"ready"}]Checked answer{"ids": ["A716", "Q407", "S991", "U866"]}wrong_schema_or_value
{
"ids": ["A716", "S991", "U866", "C837", "Q407"]
}38 tokenswrong_schema_or_value
{"ids":["A716","S991","Q407"]}21 tokenswrong_schema_or_value
{"ids":["A716","S991","Q407","U866"]}26 tokenswrong_schema_or_value
{"ids":["A716","S991","Q407"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"L298","units":19,"price":1},{"id":"N730","units":6,"price":3},{"id":"P149","units":16,"price":2},{"id":"Z361","units":5,"price":6},{"id":"O735","units":16,"price":16}]Checked answer{"total": 355}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"A315","priority":1},{"id":"P758","priority":3},{"id":"Y137","priority":7},{"id":"M883","priority":9},{"id":"Y312","priority":1}]Checked answer{"ids": ["M883", "Y137"]}invalid_json_object
[{"id":"A315","priority":1},{"id":"P758","priority":3},{"id":"Y137","priority":7},{"id":"M883","priority":9},{"id":"Y312","priority":1}]62 tokensCorrect
{"ids":["M883","Y137"]}16 tokensCorrect
{"ids":["M883","Y137"]}16 tokensCorrect
{"ids":["M883","Y137"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to C572? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"C572","city":"Oslo","units":18},{"id":"R228","city":"Suva","units":11},{"id":"I870","city":"Bern","units":7},{"id":"C914","city":"Suva","units":9},{"id":"I788","city":"Lima","units":12}]Checked answer{"city": "Oslo", "units": 18}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Oslo","units":18}12 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Oslo","units":18}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"K503","status":"ready"},{"id":"S810","status":"pending"},{"id":"B319","status":"pending"},{"id":"J940","status":"pending"},{"id":"X213","status":"pending"}]Checked answer{"ids": ["K503"]}invalid_json_object
[{"id":"K503"},{"id":"S810"},{"id":"B319"},{"id":"J940"},{"id":"X213"}]43 tokensCorrect
{"ids":["K503"]}11 tokensCorrect
{"ids":["K503"]}11 tokensinvalid_json_object
["ids":["K503"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"D878","units":0,"price":3},{"id":"I154","units":9,"price":16},{"id":"Q186","units":17,"price":14},{"id":"X373","units":2,"price":17},{"id":"D758","units":14,"price":3}]Checked answer{"total": 458}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"B233","priority":2},{"id":"Q107","priority":5},{"id":"P854","priority":2},{"id":"D759","priority":6},{"id":"R782","priority":8}]Checked answer{"ids": ["R782", "D759"]}wrong_schema_or_value
{"ids": ["B233", "P854", "R782", "D759", "Q107", "P854"]}41 tokensCorrect
{"ids":["R782","D759"]}16 tokensCorrect
{"ids":["R782","D759"]}16 tokensCorrect
{"ids":["R782","D759"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"U758","city":"Kyoto","units":19},{"id":"A902","city":"Kyoto","units":13},{"id":"E344","city":"Oslo","units":2},{"id":"S130","city":"Kyoto","units":13},{"id":"F778","city":"Lima","units":11}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Kyoto","units":19}13 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"H312","status":"closed"},{"id":"E304","status":"ready"},{"id":"A954","status":"closed"},{"id":"R285","status":"closed"},{"id":"T455","status":"ready"}]Checked answer{"ids": ["E304", "T455"]}wrong_schema_or_value
{"ids": ["A954", "E304", "H312", "R285", "T455"]}35 tokensCorrect
{"ids":["E304","T455"]}16 tokensCorrect
{"ids":["E304","T455"]}16 tokensCorrect
{"ids":["E304","T455"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"M450","units":15,"price":11},{"id":"J327","units":9,"price":14},{"id":"U532","units":4,"price":13},{"id":"A731","units":6,"price":11},{"id":"Q499","units":9,"price":6}]Checked answer{"total": 463}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"I230","priority":6},{"id":"Y409","priority":9},{"id":"I395","priority":5},{"id":"N273","priority":1},{"id":"J238","priority":9}]Checked answer{"ids": ["J238", "Y409"]}invalid_json_object
[{"id":"I230","priority":6},{"id":"Y409","priority":9},{"id":"I395","priority":5},{"id":"N273","priority":1},{"id":"J238","priority":9}]62 tokensCorrect
{"ids":["J238","Y409"]}16 tokensCorrect
{"ids":["J238","Y409"]}16 tokensCorrect
{"ids":["J238","Y409"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record X915. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"W472","city":"Bern","units":11},{"id":"N505","city":"Perth","units":11},{"id":"M366","city":"Accra","units":11},{"id":"X915","city":"Kyoto","units":10},{"id":"T854","city":"Accra","units":3}]Checked answer{"city": "Kyoto", "units": 10}Correct
{
"city": "Kyoto",
"units": 10
}20 tokensCorrect
{"city":"Kyoto","units":10}13 tokensCorrect
{"city":"Kyoto","units":10}13 tokensCorrect
{"city":"Kyoto","units":10}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"D563","status":"pending"},{"id":"F812","status":"pending"},{"id":"G793","status":"closed"},{"id":"S237","status":"closed"},{"id":"L717","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["D563", "F812", "G793", "S237", "L717"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"L823","units":16,"price":12},{"id":"V115","units":6,"price":17},{"id":"T802","units":1,"price":16},{"id":"U184","units":4,"price":18},{"id":"S720","units":5,"price":7}]Checked answer{"total": 417}wrong_schema_or_value
{"total": 120}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"F809","priority":7},{"id":"F174","priority":5},{"id":"I438","priority":1},{"id":"Z858","priority":7},{"id":"L232","priority":5}]Checked answer{"ids": ["F809", "Z858"]}wrong_schema_or_value
{"ids": ["F809", "I438", "Z858", "L232"]}29 tokensCorrect
{"ids":["F809","Z858"]}16 tokensCorrect
{"ids":["F809","Z858"]}16 tokensCorrect
{"ids":["F809","Z858"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id N110 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"D499","city":"Kyoto","units":13},{"id":"G593","city":"Kyoto","units":0},{"id":"N110","city":"Kyoto","units":10},{"id":"F459","city":"Lima","units":18},{"id":"E150","city":"Suva","units":14}]Checked answer{"city": "Kyoto", "units": 10}wrong_schema_or_value
{"city":"Kyoto","units":13}13 tokensCorrect
{"city":"Kyoto","units":10}13 tokensCorrect
{"city":"Kyoto","units":10}13 tokensCorrect
{"city":"Kyoto","units":10}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"P666","status":"closed"},{"id":"K751","status":"ready"},{"id":"A995","status":"pending"},{"id":"W190","status":"closed"},{"id":"Q852","status":"pending"}]Checked answer{"ids": ["K751"]}wrong_schema_or_value
{
"ids": ["A995", "K751", "Q852", "P666", "W190"]
}38 tokensCorrect
{"ids":["K751"]}11 tokensCorrect
{"ids":["K751"]}11 tokensCorrect
{"ids":["K751"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"S977","units":0,"price":16},{"id":"A382","units":6,"price":5},{"id":"E696","units":2,"price":7},{"id":"N651","units":1,"price":1},{"id":"N528","units":4,"price":6}]Checked answer{"total": 69}wrong_schema_or_value
{"total": 23}8 tokenswrong_schema_or_value
{"total":17}7 tokenswrong_schema_or_value
{"total":17}7 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Y527","priority":1},{"id":"G989","priority":7},{"id":"Y689","priority":4},{"id":"Z587","priority":6},{"id":"C780","priority":4}]Checked answer{"ids": ["G989", "Z587"]}invalid_json_object
[{"id":"Y527","priority":1},{"id":"G989","priority":7},{"id":"Y689","priority":4},{"id":"Z587","priority":6},{"id":"C780","priority":4}]62 tokensCorrect
{"ids":["G989","Z587"]}16 tokenswrong_schema_or_value
{"ids":["G989","C780"]}16 tokenswrong_schema_or_value
{"ids":["G989","C780"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to B857? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"J909","city":"Riga","units":5},{"id":"S177","city":"Riga","units":12},{"id":"P917","city":"Accra","units":10},{"id":"J573","city":"Kyoto","units":2},{"id":"B857","city":"Kyoto","units":14}]Checked answer{"city": "Kyoto", "units": 14}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Kyoto","units":14}13 tokenswrong_schema_or_value
{"city":null,"units":14}11 tokensCorrect
{"city":"Kyoto","units":14}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"E480","status":"pending"},{"id":"K747","status":"ready"},{"id":"G552","status":"closed"},{"id":"L582","status":"pending"},{"id":"I383","status":"pending"}]Checked answer{"ids": ["K747"]}invalid_json_object
[{"id":"K747","status":"ready"},{"id":"L582","status":"pending"},{"id":"I383","status":"pending"},{"id":"E480","status":"pending"},{"id":"G552","status":"closed"}]63 tokensCorrect
{"ids":["K747"]}11 tokensCorrect
{"ids":["K747"]}11 tokensinvalid_json_object
["ids":["K747"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"V228","units":5,"price":5},{"id":"A468","units":16,"price":7},{"id":"L181","units":17,"price":8},{"id":"Y501","units":16,"price":19},{"id":"R450","units":12,"price":14}]Checked answer{"total": 745}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"W554","priority":8},{"id":"H979","priority":5},{"id":"H123","priority":9},{"id":"V946","priority":8},{"id":"U819","priority":7}]Checked answer{"ids": ["H123", "V946"]}wrong_schema_or_value
{"ids": ["H123", "V946", "U819", "W554", "H979"]}35 tokenswrong_schema_or_value
{"ids":["H123","W554"]}16 tokenswrong_schema_or_value
{"ids":["H123","W554"]}16 tokensCorrect
{"ids":["H123","V946"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=P652. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"M833","city":"Bern","units":6},{"id":"A556","city":"Bern","units":18},{"id":"I202","city":"Riga","units":6},{"id":"U592","city":"Riga","units":11},{"id":"P652","city":"Kyoto","units":18}]Checked answer{"city": "Kyoto", "units": 18}Correct
{"city":"Kyoto","units":18}13 tokensCorrect
{"city":"Kyoto","units":18}13 tokensCorrect
{"city":"Kyoto","units":18}13 tokensCorrect
{"city":"Kyoto","units":18}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"J487","status":"closed"},{"id":"C965","status":"pending"},{"id":"V130","status":"ready"},{"id":"I772","status":"pending"},{"id":"C448","status":"closed"}]Checked answer{"ids": ["V130"]}wrong_schema_or_value
{"ids": ["C448", "C965", "J487", "I772", "V130"]}35 tokensCorrect
{"ids":["V130"]}11 tokensCorrect
{"ids":["V130"]}11 tokensCorrect
{"ids":["V130"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"K568","units":11,"price":8},{"id":"L565","units":8,"price":13},{"id":"E524","units":18,"price":11},{"id":"L657","units":6,"price":13},{"id":"C442","units":2,"price":2}]Checked answer{"total": 472}wrong_schema_or_value
{
"total": 120
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"N603","priority":5},{"id":"U619","priority":9},{"id":"H292","priority":6},{"id":"V566","priority":2},{"id":"Y701","priority":9}]Checked answer{"ids": ["U619", "Y701"]}invalid_json_object
[{"id":"N603","priority":5},{"id":"U619","priority":9},{"id":"H292","priority":6},{"id":"V566","priority":2},{"id":"Y701","priority":9}]62 tokenswrong_schema_or_value
{"ids":["Y701","H292"]}16 tokenswrong_schema_or_value
{"ids":["H292","Y701"]}16 tokenswrong_schema_or_value
{"ids":["H292","N603"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"K966","city":"Perth","units":8},{"id":"A684","city":"Suva","units":6},{"id":"N901","city":"Oslo","units":6},{"id":"I342","city":"Lima","units":6},{"id":"C448","city":"Riga","units":19}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Perth","units":8},{"city":"Suva","units":6},{"city":"Oslo","units":6},{"city":"Lima","units":6},{"city":"Riga","units":19}]53 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"B111","status":"ready"},{"id":"E565","status":"pending"},{"id":"Y860","status":"closed"},{"id":"J886","status":"ready"},{"id":"Q823","status":"closed"}]Checked answer{"ids": ["B111", "J886"]}wrong_schema_or_value
{"ids": ["B111", "E565", "J886", "Q823"]}29 tokensCorrect
{"ids":["B111","J886"]}16 tokensCorrect
{"ids":["B111","J886"]}16 tokensCorrect
{"ids":["B111","J886"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"F302","units":7,"price":9},{"id":"C865","units":0,"price":5},{"id":"A290","units":8,"price":4},{"id":"U303","units":18,"price":4},{"id":"T376","units":8,"price":13}]Checked answer{"total": 271}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":468}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"K636","priority":7},{"id":"V156","priority":7},{"id":"G556","priority":9},{"id":"E687","priority":2},{"id":"K197","priority":8}]Checked answer{"ids": ["G556", "K197"]}wrong_schema_or_value
{"ids": ["K636", "V156", "G556", "K197", "E687"]}35 tokensCorrect
{"ids":["G556","K197"]}16 tokensCorrect
{"ids":["G556","K197"]}16 tokensCorrect
{"ids":["G556","K197"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id W365 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"W365","city":"Perth","units":17},{"id":"F422","city":"Oslo","units":17},{"id":"L213","city":"Oslo","units":11},{"id":"F413","city":"Accra","units":3},{"id":"R959","city":"Perth","units":11}]Checked answer{"city": "Perth", "units": 17}Correct
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"M761","status":"closed"},{"id":"N530","status":"closed"},{"id":"A928","status":"pending"},{"id":"M293","status":"closed"},{"id":"E949","status":"ready"}]Checked answer{"ids": ["E949"]}wrong_schema_or_value
{
"ids": ["A928", "E949", "M293", "M761", "N530"]
}38 tokensCorrect
{"ids":["E949"]}11 tokensCorrect
{"ids":["E949"]}11 tokensCorrect
{"ids":["E949"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"H707","units":5,"price":7},{"id":"V885","units":11,"price":11},{"id":"Z902","units":11,"price":19},{"id":"Q230","units":1,"price":6},{"id":"I955","units":4,"price":2}]Checked answer{"total": 379}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":378}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"D674","priority":8},{"id":"H976","priority":9},{"id":"Y931","priority":7},{"id":"K669","priority":2},{"id":"N886","priority":3}]Checked answer{"ids": ["H976", "D674"]}invalid_json_object
[{"id":"D674","priority":8},{"id":"H976","priority":9},{"id":"Y931","priority":7},{"id":"K669","priority":2},{"id":"N886","priority":3}]62 tokensCorrect
{"ids":["H976","D674"]}16 tokensCorrect
{"ids":["H976","D674"]}16 tokensCorrect
{"ids":["H976","D674"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to B528? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"K805","city":"Suva","units":18},{"id":"B528","city":"Lima","units":1},{"id":"T938","city":"Bern","units":2},{"id":"X990","city":"Kyoto","units":14},{"id":"B920","city":"Oslo","units":9}]Checked answer{"city": "Lima", "units": 1}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Lima","units":1}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Lima","units":1}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"B143","status":"pending"},{"id":"D734","status":"ready"},{"id":"N962","status":"closed"},{"id":"D890","status":"pending"},{"id":"F209","status":"ready"}]Checked answer{"ids": ["D734", "F209"]}invalid_json_object
[{"id":"D890","status":"pending"},{"id":"D734","status":"ready"},{"id":"F209","status":"ready"},{"id":"N962","status":"closed"},{"id":"B143","status":"pending"}]63 tokensinvalid_json_object
["D734","F209"]12 tokens
Correct
{"ids":["D734","F209"]}16 tokensinvalid_json_object
["ids":["D734","F209"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Y722","units":15,"price":19},{"id":"R695","units":16,"price":5},{"id":"F660","units":8,"price":9},{"id":"R654","units":8,"price":9},{"id":"Y265","units":10,"price":15}]Checked answer{"total": 659}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"U622","priority":1},{"id":"F508","priority":2},{"id":"P119","priority":5},{"id":"T875","priority":6},{"id":"B540","priority":1}]Checked answer{"ids": ["T875", "P119"]}wrong_schema_or_value
{"ids": ["U622", "F508", "P119", "T875", "B540"]}35 tokenswrong_schema_or_value
{"ids":["P119","T875"]}16 tokenswrong_schema_or_value
{"ids":["P119","T875"]}16 tokensCorrect
{"ids":["T875","P119"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Y316. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"H153","city":"Bern","units":19},{"id":"D604","city":"Accra","units":16},{"id":"Y316","city":"Accra","units":18},{"id":"Q261","city":"Bern","units":4},{"id":"Z387","city":"Bern","units":1}]Checked answer{"city": "Accra", "units": 18}wrong_schema_or_value
{"city":"Bern","units":19}11 tokensCorrect
{"city":"Accra","units":18}12 tokensCorrect
{"city":"Accra","units":18}12 tokensCorrect
{"city":"Accra","units":18}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"R573","status":"closed"},{"id":"D891","status":"pending"},{"id":"R932","status":"pending"},{"id":"X490","status":"ready"},{"id":"H975","status":"closed"}]Checked answer{"ids": ["X490"]}wrong_schema_or_value
{"ids": ["X490", "H975", "R573", "D891", "R932"]}35 tokensCorrect
{"ids":["X490"]}11 tokensCorrect
{"ids":["X490"]}11 tokensCorrect
{"ids":["X490"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"O498","units":13,"price":15},{"id":"S285","units":19,"price":1},{"id":"K520","units":1,"price":6},{"id":"G443","units":9,"price":4},{"id":"C654","units":15,"price":11}]Checked answer{"total": 421}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"G391","priority":2},{"id":"S497","priority":7},{"id":"I917","priority":4},{"id":"K603","priority":9},{"id":"S806","priority":2}]Checked answer{"ids": ["K603", "S497"]}invalid_json_object
[{"id":"S806","priority":2},{"id":"S497","priority":7},{"id":"G391","priority":2},{"id":"I917","priority":4},{"id":"K603","priority":9}]62 tokensCorrect
{"ids":["K603","S497"]}16 tokensCorrect
{"ids":["K603","S497"]}16 tokensCorrect
{"ids":["K603","S497"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record N376. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"C363","city":"Kyoto","units":2},{"id":"X384","city":"Bern","units":16},{"id":"C580","city":"Suva","units":16},{"id":"L325","city":"Lima","units":19},{"id":"N376","city":"Lima","units":3}]Checked answer{"city": "Lima", "units": 3}Correct
{
"city": "Lima",
"units": 3
}18 tokensCorrect
{"city":"Lima","units":3}11 tokensCorrect
{"city":"Lima","units":3}11 tokensCorrect
{"city":"Lima","units":3}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"F175","status":"closed"},{"id":"B281","status":"ready"},{"id":"C330","status":"closed"},{"id":"O323","status":"ready"},{"id":"P613","status":"closed"}]Checked answer{"ids": ["B281", "O323"]}wrong_schema_or_value
{"ids": ["B281", "C330", "O323", "P613"]}29 tokensCorrect
{"ids":["B281","O323"]}16 tokensCorrect
{"ids":["B281","O323"]}16 tokensCorrect
{"ids":["B281","O323"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"F146","units":0,"price":14},{"id":"D154","units":12,"price":7},{"id":"U213","units":4,"price":17},{"id":"P978","units":4,"price":4},{"id":"A569","units":10,"price":13}]Checked answer{"total": 298}wrong_schema_or_value
{"total": 140}9 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":308}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"T324","priority":7},{"id":"V680","priority":4},{"id":"A667","priority":2},{"id":"Q749","priority":8},{"id":"I825","priority":6}]Checked answer{"ids": ["Q749", "T324"]}wrong_schema_or_value
{"ids": ["V680", "A667", "I825", "Q749", "T324"]}35 tokensCorrect
{"ids":["Q749","T324"]}16 tokensCorrect
{"ids":["Q749","T324"]}16 tokensCorrect
{"ids":["Q749","T324"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"U832","city":"Riga","units":10},{"id":"C630","city":"Perth","units":2},{"id":"Q931","city":"Bern","units":6},{"id":"N434","city":"Oslo","units":8},{"id":"Z422","city":"Perth","units":13}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Riga","units":10}12 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"T492","status":"closed"},{"id":"W570","status":"pending"},{"id":"Y288","status":"ready"},{"id":"Z353","status":"closed"},{"id":"M396","status":"ready"}]Checked answer{"ids": ["M396", "Y288"]}wrong_schema_or_value
{
"ids": ["M396", "Y288", "Z353", "T492"]
}32 tokensCorrect
{"ids":["M396","Y288"]}16 tokensCorrect
{"ids":["M396","Y288"]}16 tokensCorrect
{"ids":["M396","Y288"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"O539","units":19,"price":7},{"id":"T949","units":14,"price":8},{"id":"V150","units":11,"price":1},{"id":"B261","units":6,"price":11},{"id":"F994","units":14,"price":8}]Checked answer{"total": 434}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"O199","priority":2},{"id":"X174","priority":2},{"id":"J151","priority":2},{"id":"C279","priority":9},{"id":"W665","priority":6}]Checked answer{"ids": ["C279", "W665"]}invalid_json_object
[{"id":"O199","priority":2},{"id":"X174","priority":2},{"id":"J151","priority":2},{"id":"C279","priority":9},{"id":"W665","priority":6}]62 tokensCorrect
{"ids":["C279","W665"]}16 tokensCorrect
{"ids":["C279","W665"]}16 tokensCorrect
{"ids":["C279","W665"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to L921? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"L921","city":"Lima","units":8},{"id":"A115","city":"Bern","units":0},{"id":"W919","city":"Lima","units":12},{"id":"K247","city":"Oslo","units":4},{"id":"F745","city":"Accra","units":14}]Checked answer{"city": "Lima", "units": 8}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Lima","units":8}11 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Lima","units":8}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"W602","status":"closed"},{"id":"F932","status":"pending"},{"id":"N941","status":"pending"},{"id":"G365","status":"ready"},{"id":"Q422","status":"pending"}]Checked answer{"ids": ["G365"]}invalid_json_object
[{"id":"G365"}]11 tokensinvalid_json_object
["G365"]7 tokens
Correct
{"ids":["G365"]}11 tokensinvalid_json_object
["ids":["G365"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"F398","units":0,"price":2},{"id":"T657","units":13,"price":12},{"id":"Z855","units":1,"price":11},{"id":"P882","units":11,"price":15},{"id":"W189","units":5,"price":11}]Checked answer{"total": 387}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"I138","priority":1},{"id":"Z640","priority":8},{"id":"W875","priority":2},{"id":"V174","priority":5},{"id":"K455","priority":1}]Checked answer{"ids": ["Z640", "V174"]}wrong_schema_or_value
{"ids": ["I138", "Z640", "W875", "V174", "K455"]}35 tokenswrong_schema_or_value
{"ids":["Z640","W875"]}16 tokenswrong_schema_or_value
{"ids":["Z640","I138"]}16 tokenswrong_schema_or_value
{"ids":["Z640","I138"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=M214. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"D556","city":"Riga","units":9},{"id":"Y261","city":"Kyoto","units":19},{"id":"P769","city":"Kyoto","units":6},{"id":"B321","city":"Kyoto","units":18},{"id":"M214","city":"Riga","units":15}]Checked answer{"city": "Riga", "units": 15}wrong_schema_or_value
{"city":"Riga","units":9}11 tokensCorrect
{"city":"Riga","units":15}12 tokensCorrect
{"city":"Riga","units":15}12 tokensCorrect
{"city":"Riga","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"E431","status":"closed"},{"id":"Z661","status":"closed"},{"id":"K275","status":"closed"},{"id":"K986","status":"ready"},{"id":"Z965","status":"closed"}]Checked answer{"ids": ["K986"]}wrong_schema_or_value
{"ids": ["K275", "K986", "Z661", "Z965", "E431"]}35 tokensCorrect
{"ids":["K986"]}11 tokensCorrect
{"ids":["K986"]}11 tokensCorrect
{"ids":["K986"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"K349","units":12,"price":18},{"id":"B803","units":5,"price":9},{"id":"H687","units":5,"price":19},{"id":"B430","units":14,"price":5},{"id":"L745","units":7,"price":1}]Checked answer{"total": 433}invalid_json_object
{
"total": 12 * 18 + 5 * 9 + 5 * 19 + 14 * 5 + 7 * 1 = 216 + 45 + 95 + 70 + 7 = 522
}66 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"T748","priority":9},{"id":"Q109","priority":9},{"id":"G675","priority":6},{"id":"N759","priority":2},{"id":"C120","priority":8}]Checked answer{"ids": ["Q109", "T748"]}invalid_json_object
[{"id":"T748","priority":9},{"id":"G675","priority":6},{"id":"N759","priority":2},{"id":"C120","priority":8}]50 tokensCorrect
{"ids":["Q109","T748"]}16 tokenswrong_schema_or_value
{"ids":["T748","Q109"]}16 tokensCorrect
{"ids":["Q109","T748"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record J428. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"A726","city":"Lima","units":16},{"id":"J428","city":"Accra","units":19},{"id":"V421","city":"Accra","units":6},{"id":"H883","city":"Oslo","units":3},{"id":"Q488","city":"Oslo","units":13}]Checked answer{"city": "Accra", "units": 19}wrong_schema_or_value
{
"city": "Lima",
"units": 16
}19 tokensCorrect
{"city":"Accra","units":19}12 tokensCorrect
{"city":"Accra","units":19}12 tokensCorrect
{"city":"Accra","units":19}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"G325","status":"ready"},{"id":"S282","status":"ready"},{"id":"Y623","status":"pending"},{"id":"F985","status":"pending"},{"id":"F383","status":"ready"}]Checked answer{"ids": ["F383", "G325", "S282"]}wrong_schema_or_value
{"ids": ["G325", "S282", "F383", "F985", "Y623"]}35 tokenswrong_schema_or_value
{"ids":["G325","F383","S282"]}21 tokenswrong_schema_or_value
{"ids":["G325","F383"]}16 tokenswrong_schema_or_value
{"ids":["G325","F383","S282"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Z561","units":15,"price":12},{"id":"Z376","units":16,"price":17},{"id":"I560","units":1,"price":15},{"id":"K556","units":7,"price":15},{"id":"M590","units":17,"price":13}]Checked answer{"total": 793}wrong_schema_or_value
{"total": 120}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"H139","priority":4},{"id":"U611","priority":3},{"id":"D486","priority":1},{"id":"R368","priority":5},{"id":"O787","priority":9}]Checked answer{"ids": ["O787", "R368"]}wrong_schema_or_value
{"ids": ["U611", "D486", "R368", "H139", "O787"]}35 tokensCorrect
{"ids":["O787","R368"]}16 tokensCorrect
{"ids":["O787","R368"]}16 tokenswrong_schema_or_value
{"ids":["O787","H139"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id W755 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"W755","city":"Suva","units":13},{"id":"V627","city":"Riga","units":2},{"id":"R752","city":"Perth","units":5},{"id":"Z646","city":"Suva","units":12},{"id":"N556","city":"Oslo","units":9}]Checked answer{"city": "Suva", "units": 13}Correct
{"city":"Suva","units":13}12 tokensCorrect
{"city":"Suva","units":13}12 tokensCorrect
{"city":"Suva","units":13}12 tokensCorrect
{"city":"Suva","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"F854","status":"closed"},{"id":"G755","status":"ready"},{"id":"X809","status":"pending"},{"id":"U782","status":"pending"},{"id":"L922","status":"ready"}]Checked answer{"ids": ["G755", "L922"]}wrong_schema_or_value
{
"ids": ["F854", "G755", "U782", "L922"]
}32 tokenswrong_schema_or_value
{"ids":["L922","G755"]}16 tokenswrong_schema_or_value
{"ids":["L922","G755"]}16 tokenswrong_schema_or_value
{"ids":["L922","G755"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"L771","units":10,"price":14},{"id":"E108","units":7,"price":1},{"id":"N306","units":13,"price":4},{"id":"A130","units":13,"price":17},{"id":"P148","units":11,"price":19}]Checked answer{"total": 629}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":468}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"R673","priority":2},{"id":"Z703","priority":5},{"id":"E157","priority":8},{"id":"M255","priority":3},{"id":"R122","priority":3}]Checked answer{"ids": ["E157", "Z703"]}invalid_json_object
[{"id":"R673","priority":2},{"id":"Z703","priority":5},{"id":"E157","priority":8},{"id":"M255","priority":3},{"id":"R122","priority":3}]62 tokensCorrect
{"ids":["E157","Z703"]}16 tokensCorrect
{"ids":["E157","Z703"]}16 tokensCorrect
{"ids":["E157","Z703"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"O454","city":"Riga","units":12},{"id":"R995","city":"Accra","units":8},{"id":"H375","city":"Oslo","units":0},{"id":"M694","city":"Accra","units":3},{"id":"N192","city":"Bern","units":12}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"G633","status":"ready"},{"id":"Y691","status":"ready"},{"id":"B215","status":"ready"},{"id":"T843","status":"ready"},{"id":"T426","status":"pending"}]Checked answer{"ids": ["B215", "G633", "T843", "Y691"]}invalid_json_object
[{"id":"G633","status":"ready"},{"id":"Y691","status":"ready"},{"id":"B215","status":"ready"},{"id":"T843","status":"ready"},{"id":"T426","status":"pending"}]63 tokenswrong_schema_or_value
{"ids":["B215","G633","Y691","T843"]}26 tokenswrong_schema_or_value
{"ids":["B215","G633","Y691","T843"]}26 tokensinvalid_json_object
["ids":["G633","B215","T843"]]21 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"C704","units":7,"price":9},{"id":"W271","units":7,"price":4},{"id":"F628","units":15,"price":19},{"id":"R939","units":13,"price":19},{"id":"A532","units":14,"price":15}]Checked answer{"total": 833}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"E397","priority":4},{"id":"V984","priority":8},{"id":"B307","priority":1},{"id":"V403","priority":5},{"id":"D550","priority":1}]Checked answer{"ids": ["V984", "V403"]}wrong_schema_or_value
{"ids": ["V403", "V984", "B307", "E397", "D550"]}35 tokenswrong_schema_or_value
{"ids":["V984","E397"]}16 tokenswrong_schema_or_value
{"ids":["V984","E397"]}16 tokenswrong_schema_or_value
{"ids":["V403","E397"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=V135. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"N444","city":"Suva","units":11},{"id":"M137","city":"Accra","units":3},{"id":"V135","city":"Perth","units":7},{"id":"L468","city":"Suva","units":13},{"id":"D518","city":"Suva","units":10}]Checked answer{"city": "Perth", "units": 7}wrong_schema_or_value
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Perth","units":7}11 tokensCorrect
{"city":"Perth","units":7}11 tokensCorrect
{"city":"Perth","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"D184","status":"closed"},{"id":"U779","status":"ready"},{"id":"E775","status":"pending"},{"id":"P626","status":"pending"},{"id":"V700","status":"ready"}]Checked answer{"ids": ["U779", "V700"]}wrong_schema_or_value
{"ids": ["U779", "D184", "P626", "E775", "V700"]}35 tokenswrong_schema_or_value
{"ids":["D184","V700","U779"]}21 tokenswrong_schema_or_value
{"ids":["D184","V700","U779"]}21 tokenswrong_schema_or_value
{"ids":["D184","V700","U779"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"W176","units":0,"price":1},{"id":"D428","units":14,"price":14},{"id":"O740","units":1,"price":7},{"id":"H926","units":9,"price":12},{"id":"M742","units":7,"price":19}]Checked answer{"total": 444}wrong_schema_or_value
{
"total": 40
}11 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":408}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"U789","priority":2},{"id":"W577","priority":9},{"id":"G117","priority":3},{"id":"A652","priority":1},{"id":"P673","priority":4}]Checked answer{"ids": ["W577", "P673"]}invalid_json_object
[{"id":"U789"},{"id":"W577"},{"id":"G117"},{"id":"A652"},{"id":"P673"}]43 tokensCorrect
{"ids":["W577","P673"]}16 tokenswrong_schema_or_value
{"ids":["W577","G117"]}16 tokenswrong_schema_or_value
{"ids":["W577","A652"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record J781. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"J781","city":"Accra","units":18},{"id":"M646","city":"Bern","units":13},{"id":"D716","city":"Riga","units":0},{"id":"N486","city":"Riga","units":19},{"id":"W510","city":"Bern","units":17}]Checked answer{"city": "Accra", "units": 18}Correct
{
"city": "Accra",
"units": 18
}19 tokensCorrect
{"city":"Accra","units":18}12 tokensCorrect
{"city":"Accra","units":18}12 tokensCorrect
{"city":"Accra","units":18}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Z398","status":"closed"},{"id":"P481","status":"pending"},{"id":"Z921","status":"closed"},{"id":"I243","status":"closed"},{"id":"X325","status":"ready"}]Checked answer{"ids": ["X325"]}wrong_schema_or_value
{"ids": ["I243", "X325"]}17 tokensCorrect
{"ids":["X325"]}11 tokensCorrect
{"ids":["X325"]}11 tokensCorrect
{"ids":["X325"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"S145","units":6,"price":7},{"id":"I759","units":12,"price":12},{"id":"O172","units":18,"price":13},{"id":"K696","units":18,"price":10},{"id":"V441","units":13,"price":6}]Checked answer{"total": 678}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"P900","priority":8},{"id":"W817","priority":3},{"id":"Q657","priority":7},{"id":"Z633","priority":3},{"id":"Y988","priority":9}]Checked answer{"ids": ["Y988", "P900"]}wrong_schema_or_value
{"ids": ["P900", "W817", "Q657", "Z633", "Y988"]}35 tokensCorrect
{"ids":["Y988","P900"]}16 tokensCorrect
{"ids":["Y988","P900"]}16 tokensCorrect
{"ids":["Y988","P900"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id V994 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"Z770","city":"Accra","units":15},{"id":"V994","city":"Bern","units":0},{"id":"D545","city":"Accra","units":11},{"id":"M535","city":"Accra","units":4},{"id":"T297","city":"Perth","units":18}]Checked answer{"city": "Bern", "units": 0}wrong_schema_or_value
{"city":"Accra","units":15}12 tokensCorrect
{"city":"Bern","units":0}10 tokensCorrect
{"city":"Bern","units":0}10 tokensCorrect
{"city":"Bern","units":0}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"Z734","status":"closed"},{"id":"F654","status":"pending"},{"id":"E636","status":"ready"},{"id":"H115","status":"ready"},{"id":"G824","status":"pending"}]Checked answer{"ids": ["E636", "H115"]}wrong_schema_or_value
{
"ids": ["E636", "F654", "G824", "H115"]
}32 tokensCorrect
{"ids":["E636","H115"]}16 tokensCorrect
{"ids":["E636","H115"]}16 tokensCorrect
{"ids":["E636","H115"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"P155","units":10,"price":3},{"id":"E176","units":17,"price":1},{"id":"M179","units":19,"price":3},{"id":"R566","units":16,"price":8},{"id":"Y813","units":19,"price":14}]Checked answer{"total": 498}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"K501","priority":5},{"id":"X342","priority":1},{"id":"J540","priority":8},{"id":"Y284","priority":7},{"id":"B987","priority":8}]Checked answer{"ids": ["B987", "J540"]}invalid_json_object
[{"id":"K501","priority":5},{"id":"X342","priority":1},{"id":"J540","priority":8},{"id":"Y284","priority":7},{"id":"B987","priority":8}]62 tokenswrong_schema_or_value
{"ids":["J540","B987"]}16 tokenswrong_schema_or_value
{"ids":["J540","B987"]}16 tokenswrong_schema_or_value
{"ids":["J540","B987"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to V703? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"T485","city":"Bern","units":18},{"id":"N582","city":"Suva","units":10},{"id":"V703","city":"Accra","units":4},{"id":"E442","city":"Suva","units":1},{"id":"X554","city":"Accra","units":2}]Checked answer{"city": "Accra", "units": 4}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Accra","units":4}11 tokensCorrect
{"city":"Accra","units":4}11 tokensCorrect
{"city":"Accra","units":4}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"I634","status":"ready"},{"id":"O874","status":"closed"},{"id":"Y419","status":"ready"},{"id":"M252","status":"closed"},{"id":"I854","status":"pending"}]Checked answer{"ids": ["I634", "Y419"]}invalid_json_object
[{"id":"I634"},{"id":"O874"},{"id":"Y419"},{"id":"M252"},{"id":"I854"}]43 tokensCorrect
{"ids":["I634","Y419"]}16 tokensCorrect
{"ids":["I634","Y419"]}16 tokensinvalid_json_object
["ids":["I634","Y419"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"W253","units":3,"price":4},{"id":"Q290","units":5,"price":7},{"id":"R215","units":8,"price":14},{"id":"E163","units":0,"price":19},{"id":"Y411","units":5,"price":11}]Checked answer{"total": 214}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":268}8 tokenswrong_schema_or_value
{"total":268}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Y149","priority":5},{"id":"P789","priority":6},{"id":"G783","priority":3},{"id":"E326","priority":5},{"id":"O778","priority":3}]Checked answer{"ids": ["P789", "E326"]}wrong_schema_or_value
{"ids": ["Y149", "P789", "G783", "E326", "O778"]}35 tokenswrong_schema_or_value
{"ids":["P789","Y149"]}16 tokenswrong_schema_or_value
{"ids":["P789","Y149"]}16 tokensCorrect
{"ids":["P789","E326"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"E806","city":"Oslo","units":2},{"id":"R253","city":"Perth","units":4},{"id":"X600","city":"Lima","units":1},{"id":"O400","city":"Riga","units":15},{"id":"K650","city":"Perth","units":18}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Oslo","units":2}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"J162","status":"closed"},{"id":"Q755","status":"ready"},{"id":"P260","status":"ready"},{"id":"T704","status":"pending"},{"id":"O986","status":"closed"}]Checked answer{"ids": ["P260", "Q755"]}wrong_schema_or_value
{"ids": ["Q755", "P260", "T704", "J162", "O986"]}35 tokensCorrect
{"ids":["P260","Q755"]}16 tokensCorrect
{"ids":["P260","Q755"]}16 tokensCorrect
{"ids":["P260","Q755"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"H860","units":8,"price":1},{"id":"C268","units":6,"price":15},{"id":"L713","units":8,"price":11},{"id":"C233","units":18,"price":8},{"id":"E221","units":15,"price":19}]Checked answer{"total": 615}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"H633","priority":6},{"id":"Z763","priority":1},{"id":"K549","priority":5},{"id":"F157","priority":6},{"id":"N714","priority":3}]Checked answer{"ids": ["F157", "H633"]}invalid_json_object
[{"id":"H633","priority":6},{"id":"Z763","priority":1},{"id":"K549","priority":5},{"id":"F157","priority":6},{"id":"N714","priority":3}]62 tokensCorrect
{"ids":["F157","H633"]}16 tokenswrong_schema_or_value
{"ids":["H633","F157"]}16 tokensCorrect
{"ids":["F157","H633"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record W977. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"W977","city":"Riga","units":1},{"id":"L878","city":"Accra","units":0},{"id":"B613","city":"Suva","units":19},{"id":"J757","city":"Perth","units":14},{"id":"L924","city":"Bern","units":19}]Checked answer{"city": "Riga", "units": 1}Correct
{
"city": "Riga",
"units": 1
}18 tokensCorrect
{"city":"Riga","units":1}11 tokensCorrect
{"city":"Riga","units":1}11 tokensCorrect
{"city":"Riga","units":1}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"L613","status":"pending"},{"id":"D960","status":"ready"},{"id":"L471","status":"pending"},{"id":"A214","status":"closed"},{"id":"P169","status":"closed"}]Checked answer{"ids": ["D960"]}wrong_schema_or_value
{"ids": ["D960", "L471", "L613", "P169"]}29 tokensCorrect
{"ids":["D960"]}11 tokensCorrect
{"ids":["D960"]}11 tokensCorrect
{"ids":["D960"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"M277","units":7,"price":2},{"id":"F765","units":10,"price":11},{"id":"M152","units":5,"price":14},{"id":"C791","units":5,"price":10},{"id":"W383","units":11,"price":3}]Checked answer{"total": 277}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":318}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"O828","priority":7},{"id":"G559","priority":7},{"id":"E998","priority":5},{"id":"B236","priority":8},{"id":"E543","priority":7}]Checked answer{"ids": ["B236", "E543"]}wrong_schema_or_value
{"ids": ["O828", "G559", "E998", "B236", "E543"]}35 tokenswrong_schema_or_value
{"ids":["B236","O828"]}16 tokenswrong_schema_or_value
{"ids":["B236","G559"]}16 tokenswrong_schema_or_value
{"ids":["B236","G559"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id M387 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"O206","city":"Kyoto","units":12},{"id":"K434","city":"Perth","units":18},{"id":"M387","city":"Suva","units":17},{"id":"X835","city":"Perth","units":10},{"id":"K162","city":"Suva","units":11}]Checked answer{"city": "Suva", "units": 17}Correct
{"city":"Suva","units":17}12 tokensCorrect
{"city":"Suva","units":17}12 tokensCorrect
{"city":"Suva","units":17}12 tokensCorrect
{"city":"Suva","units":17}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"K481","status":"closed"},{"id":"T618","status":"ready"},{"id":"V550","status":"closed"},{"id":"T163","status":"pending"},{"id":"F196","status":"pending"}]Checked answer{"ids": ["T618"]}wrong_schema_or_value
{
"ids": ["T163", "T618", "V550", "K481"]
}32 tokensCorrect
{"ids":["T618"]}11 tokensCorrect
{"ids":["T618"]}11 tokenswrong_schema_or_value
{"ids":["T163"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"J744","units":17,"price":16},{"id":"Q713","units":5,"price":16},{"id":"M695","units":16,"price":17},{"id":"G918","units":7,"price":13},{"id":"R442","units":1,"price":13}]Checked answer{"total": 728}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"N418","priority":7},{"id":"Z302","priority":2},{"id":"J827","priority":1},{"id":"E635","priority":6},{"id":"O788","priority":5}]Checked answer{"ids": ["N418", "E635"]}invalid_json_object
[{"id":"N418","priority":7},{"id":"Z302","priority":2},{"id":"J827","priority":1},{"id":"E635","priority":6},{"id":"O788","priority":5}]62 tokensCorrect
{"ids":["N418","E635"]}16 tokensCorrect
{"ids":["N418","E635"]}16 tokensCorrect
{"ids":["N418","E635"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to G928? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"Y697","city":"Bern","units":19},{"id":"A260","city":"Kyoto","units":9},{"id":"A266","city":"Perth","units":19},{"id":"I322","city":"Lima","units":13},{"id":"G928","city":"Oslo","units":15}]Checked answer{"city": "Oslo", "units": 15}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Oslo","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"K747","status":"pending"},{"id":"I385","status":"ready"},{"id":"U972","status":"ready"},{"id":"P296","status":"ready"},{"id":"V834","status":"pending"}]Checked answer{"ids": ["I385", "P296", "U972"]}invalid_json_object
[{"id":"I385","status":"ready"},{"id":"U972","status":"ready"},{"id":"P296","status":"ready"},{"id":"K747","status":"pending"},{"id":"V834","status":"pending"}]63 tokensCorrect
{"ids":["I385","P296","U972"]}21 tokensCorrect
{"ids":["I385","P296","U972"]}21 tokensinvalid_json_object
["ids":["I385","P296","U972"]]21 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"P717","units":10,"price":3},{"id":"Q248","units":15,"price":8},{"id":"X572","units":4,"price":8},{"id":"K167","units":14,"price":10},{"id":"V976","units":17,"price":19}]Checked answer{"total": 645}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"O305","priority":9},{"id":"F603","priority":8},{"id":"M313","priority":6},{"id":"M834","priority":4},{"id":"F323","priority":1}]Checked answer{"ids": ["O305", "F603"]}wrong_schema_or_value
{"ids": ["O305", "F603", "M313", "M834", "F323"]}35 tokensCorrect
{"ids":["O305","F603"]}16 tokensCorrect
{"ids":["O305","F603"]}16 tokensCorrect
{"ids":["O305","F603"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Q775. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"V790","city":"Perth","units":6},{"id":"K424","city":"Oslo","units":2},{"id":"Q775","city":"Accra","units":13},{"id":"E646","city":"Riga","units":5},{"id":"X165","city":"Kyoto","units":4}]Checked answer{"city": "Accra", "units": 13}Correct
{"city": "Accra", "units": 13}15 tokensCorrect
{"city":"Accra","units":13}12 tokensCorrect
{"city":"Accra","units":13}12 tokensCorrect
{"city":"Accra","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"E306","status":"pending"},{"id":"F436","status":"pending"},{"id":"D623","status":"closed"},{"id":"M405","status":"ready"},{"id":"B240","status":"closed"}]Checked answer{"ids": ["M405"]}wrong_schema_or_value
{"ids": ["D623", "M405", "E306", "F436", "B240"]}35 tokensCorrect
{"ids":["M405"]}11 tokensCorrect
{"ids":["M405"]}11 tokensCorrect
{"ids":["M405"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"P833","units":2,"price":4},{"id":"Q188","units":15,"price":13},{"id":"N282","units":13,"price":14},{"id":"R694","units":19,"price":10},{"id":"Y159","units":16,"price":12}]Checked answer{"total": 767}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"S477","priority":4},{"id":"M281","priority":2},{"id":"Q971","priority":1},{"id":"H289","priority":3},{"id":"P129","priority":7}]Checked answer{"ids": ["P129", "S477"]}invalid_json_object
[{"id":"S477","priority":4},{"id":"M281","priority":2},{"id":"Q971","priority":1},{"id":"H289","priority":3},{"id":"P129","priority":7}]62 tokenswrong_schema_or_value
{"ids":["P129","H289"]}16 tokensCorrect
{"ids":["P129","S477"]}16 tokensCorrect
{"ids":["P129","S477"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"C744","city":"Accra","units":19},{"id":"R772","city":"Oslo","units":7},{"id":"O326","city":"Accra","units":4},{"id":"G325","city":"Perth","units":12},{"id":"H314","city":"Perth","units":19}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{
"city": "Accra",
"units": 19
}19 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Q895","status":"ready"},{"id":"O228","status":"ready"},{"id":"P156","status":"closed"},{"id":"Y388","status":"pending"},{"id":"J777","status":"ready"}]Checked answer{"ids": ["J777", "O228", "Q895"]}wrong_schema_or_value
{"ids": ["O228", "Q895", "P156", "J777", "Y388"]}35 tokensCorrect
{"ids":["J777","O228","Q895"]}21 tokensCorrect
{"ids":["J777","O228","Q895"]}21 tokenswrong_schema_or_value
{"ids":["Q895","J777","O228"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"C594","units":19,"price":8},{"id":"I246","units":8,"price":5},{"id":"M755","units":8,"price":18},{"id":"J590","units":13,"price":12},{"id":"X233","units":18,"price":14}]Checked answer{"total": 744}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":608}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"A738","priority":2},{"id":"N127","priority":7},{"id":"M686","priority":7},{"id":"R850","priority":7},{"id":"J523","priority":5}]Checked answer{"ids": ["M686", "N127"]}wrong_schema_or_value
{"ids": ["A738", "M686", "R850", "J523", "N127"]}35 tokenswrong_schema_or_value
{"ids":["N127","M686"]}16 tokenswrong_schema_or_value
{"ids":["N127","M686"]}16 tokenswrong_schema_or_value
{"ids":["N127","R850"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id U542 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"G433","city":"Lima","units":17},{"id":"F392","city":"Riga","units":16},{"id":"R443","city":"Suva","units":16},{"id":"U542","city":"Accra","units":7},{"id":"G784","city":"Bern","units":1}]Checked answer{"city": "Accra", "units": 7}wrong_schema_or_value
{"city":"Lima","units":17}12 tokensCorrect
{"city":"Accra","units":7}11 tokensCorrect
{"city":"Accra","units":7}11 tokensCorrect
{"city":"Accra","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"R811","status":"pending"},{"id":"F846","status":"closed"},{"id":"O714","status":"ready"},{"id":"U427","status":"pending"},{"id":"F477","status":"ready"}]Checked answer{"ids": ["F477", "O714"]}wrong_schema_or_value
{
"ids": ["O714", "F477", "U427", "R811", "F846"]
}38 tokensCorrect
{"ids":["F477","O714"]}16 tokensCorrect
{"ids":["F477","O714"]}16 tokenswrong_schema_or_value
{"ids":["O714","F477"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"J653","units":9,"price":19},{"id":"U460","units":5,"price":9},{"id":"J542","units":16,"price":13},{"id":"B972","units":11,"price":6},{"id":"Z170","units":14,"price":15}]Checked answer{"total": 700}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":548}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"U375","priority":4},{"id":"J622","priority":5},{"id":"Q401","priority":2},{"id":"Q411","priority":2},{"id":"M382","priority":3}]Checked answer{"ids": ["J622", "U375"]}invalid_json_object
[{"id":"U375","priority":4},{"id":"J622","priority":5},{"id":"Q401","priority":2},{"id":"Q411","priority":2},{"id":"M382","priority":3}]62 tokenswrong_schema_or_value
{"ids":["J622","M382"]}16 tokensCorrect
{"ids":["J622","U375"]}16 tokensCorrect
{"ids":["J622","U375"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to N837? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"R818","city":"Suva","units":6},{"id":"I961","city":"Accra","units":14},{"id":"C129","city":"Kyoto","units":18},{"id":"B620","city":"Suva","units":13},{"id":"N837","city":"Riga","units":14}]Checked answer{"city": "Riga", "units": 14}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Riga","units":14}12 tokenswrong_schema_or_value
{"city":null,"units":14}11 tokensCorrect
{"city":"Riga","units":14}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"Y555","status":"pending"},{"id":"W811","status":"ready"},{"id":"R428","status":"pending"},{"id":"L987","status":"pending"},{"id":"O244","status":"ready"}]Checked answer{"ids": ["O244", "W811"]}invalid_json_object
[{"id":"W811","status":"ready"},{"id":"O244","status":"ready"},{"id":"R428","status":"pending"},{"id":"Y555","status":"pending"},{"id":"L987","status":"pending"}]63 tokensCorrect
{"ids":["O244","W811"]}16 tokensCorrect
{"ids":["O244","W811"]}16 tokensinvalid_json_object
["ids":["O244","W811"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"R235","units":16,"price":13},{"id":"Q763","units":16,"price":12},{"id":"W979","units":16,"price":8},{"id":"Y565","units":0,"price":5},{"id":"Q289","units":19,"price":7}]Checked answer{"total": 661}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"G739","priority":8},{"id":"D843","priority":7},{"id":"E667","priority":3},{"id":"A101","priority":9},{"id":"C990","priority":4}]Checked answer{"ids": ["A101", "G739"]}wrong_schema_or_value
{"ids": ["G739", "D843"]}17 tokensCorrect
{"ids":["A101","G739"]}16 tokensCorrect
{"ids":["A101","G739"]}16 tokensCorrect
{"ids":["A101","G739"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=K283. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"E765","city":"Oslo","units":19},{"id":"K283","city":"Lima","units":7},{"id":"X218","city":"Oslo","units":9},{"id":"A813","city":"Bern","units":15},{"id":"Q172","city":"Lima","units":16}]Checked answer{"city": "Lima", "units": 7}Correct
{"city":"Lima","units":7}11 tokensCorrect
{"city":"Lima","units":7}11 tokensCorrect
{"city":"Lima","units":7}11 tokensCorrect
{"city":"Lima","units":7}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"A421","status":"pending"},{"id":"M800","status":"pending"},{"id":"F539","status":"closed"},{"id":"I553","status":"pending"},{"id":"M174","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["A421", "M800", "F539", "I553", "M174"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"D412","units":2,"price":6},{"id":"Z528","units":2,"price":6},{"id":"I737","units":1,"price":15},{"id":"E676","units":1,"price":19},{"id":"V807","units":15,"price":18}]Checked answer{"total": 328}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"K383","priority":7},{"id":"D523","priority":5},{"id":"X892","priority":9},{"id":"Q543","priority":2},{"id":"X646","priority":6}]Checked answer{"ids": ["X892", "K383"]}invalid_json_object
[{"id":"X892","priority":9},{"id":"Q543","priority":2},{"id":"K383","priority":7},{"id":"D523","priority":5},{"id":"X646","priority":6}]62 tokensCorrect
{"ids":["X892","K383"]}16 tokensCorrect
{"ids":["X892","K383"]}16 tokensCorrect
{"ids":["X892","K383"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record B268. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Q619","city":"Kyoto","units":16},{"id":"W157","city":"Bern","units":16},{"id":"B268","city":"Perth","units":3},{"id":"X102","city":"Kyoto","units":18},{"id":"Q310","city":"Accra","units":13}]Checked answer{"city": "Perth", "units": 3}Correct
{
"city": "Perth",
"units": 3
}18 tokensCorrect
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Perth","units":3}11 tokensCorrect
{"city":"Perth","units":3}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"N717","status":"closed"},{"id":"P960","status":"closed"},{"id":"C910","status":"ready"},{"id":"N175","status":"pending"},{"id":"P802","status":"closed"}]Checked answer{"ids": ["C910"]}wrong_schema_or_value
{"ids": ["C910", "N717", "N175", "P960", "P802"]}35 tokensCorrect
{"ids":["C910"]}11 tokensCorrect
{"ids":["C910"]}11 tokensCorrect
{"ids":["C910"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"O141","units":3,"price":3},{"id":"F415","units":15,"price":13},{"id":"B763","units":11,"price":12},{"id":"O915","units":1,"price":8},{"id":"D288","units":0,"price":2}]Checked answer{"total": 344}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"E146","priority":6},{"id":"S734","priority":2},{"id":"Y953","priority":9},{"id":"Q794","priority":6},{"id":"X418","priority":1}]Checked answer{"ids": ["Y953", "E146"]}wrong_schema_or_value
{"ids": ["S734", "E146", "Y953", "Q794", "X418"]}35 tokenswrong_schema_or_value
{"ids":["Y953","Q794"]}16 tokensCorrect
{"ids":["Y953","E146"]}16 tokensCorrect
{"ids":["Y953","E146"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"P210","city":"Riga","units":7},{"id":"O667","city":"Riga","units":16},{"id":"O453","city":"Bern","units":1},{"id":"U630","city":"Riga","units":9},{"id":"R968","city":"Riga","units":9}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Riga","units":7}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"S967","status":"ready"},{"id":"P427","status":"ready"},{"id":"U535","status":"pending"},{"id":"E837","status":"pending"},{"id":"F612","status":"ready"}]Checked answer{"ids": ["F612", "P427", "S967"]}wrong_schema_or_value
{
"ids": ["S967", "P427", "U535", "E837", "F612"]
}38 tokensCorrect
{"ids":["F612","P427","S967"]}21 tokenswrong_schema_or_value
{"ids":["F612","S967","P427"]}21 tokenswrong_schema_or_value
{"ids":["S967","F612","P427"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"L474","units":19,"price":15},{"id":"S592","units":4,"price":18},{"id":"I504","units":4,"price":10},{"id":"X246","units":2,"price":15},{"id":"A218","units":11,"price":16}]Checked answer{"total": 603}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"O327","priority":8},{"id":"D218","priority":5},{"id":"E992","priority":9},{"id":"I922","priority":5},{"id":"H256","priority":5}]Checked answer{"ids": ["E992", "O327"]}invalid_json_object
[{"id":"O327","priority":8},{"id":"D218","priority":5},{"id":"E992","priority":9},{"id":"I922","priority":5},{"id":"H256","priority":5}]62 tokensCorrect
{"ids":["E992","O327"]}16 tokensCorrect
{"ids":["E992","O327"]}16 tokensCorrect
{"ids":["E992","O327"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to G162? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"G162","city":"Accra","units":13},{"id":"J763","city":"Accra","units":0},{"id":"C656","city":"Lima","units":17},{"id":"K133","city":"Suva","units":19},{"id":"P183","city":"Perth","units":4}]Checked answer{"city": "Accra", "units": 13}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Accra","units":13}12 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Accra","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"H145","status":"ready"},{"id":"M603","status":"pending"},{"id":"D918","status":"closed"},{"id":"M449","status":"pending"},{"id":"B241","status":"ready"}]Checked answer{"ids": ["B241", "H145"]}invalid_json_object
[{"id":"B241","status":"ready"},{"id":"D918","status":"closed"},{"id":"H145","status":"ready"},{"id":"M449","status":"pending"},{"id":"M603","status":"pending"}]63 tokensinvalid_json_object
["B241","H145"]12 tokens
Correct
{"ids":["B241","H145"]}16 tokensinvalid_json_object
["ids":["H145","B241"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"X309","units":12,"price":5},{"id":"O241","units":4,"price":4},{"id":"U603","units":2,"price":15},{"id":"Z571","units":5,"price":8},{"id":"E886","units":19,"price":5}]Checked answer{"total": 241}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"S352","priority":7},{"id":"Q812","priority":5},{"id":"A239","priority":2},{"id":"I112","priority":5},{"id":"L552","priority":9}]Checked answer{"ids": ["L552", "S352"]}wrong_schema_or_value
{"ids": ["S352", "Q812", "A239", "I112", "L552"]}35 tokensCorrect
{"ids":["L552","S352"]}16 tokensCorrect
{"ids":["L552","S352"]}16 tokensCorrect
{"ids":["L552","S352"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=S371. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"S385","city":"Lima","units":19},{"id":"X245","city":"Accra","units":6},{"id":"R329","city":"Lima","units":8},{"id":"H315","city":"Lima","units":17},{"id":"S371","city":"Accra","units":2}]Checked answer{"city": "Accra", "units": 2}wrong_schema_or_value
{"city":"Lima","units":19}12 tokensCorrect
{"city":"Accra","units":2}11 tokensCorrect
{"city":"Accra","units":2}11 tokensCorrect
{"city":"Accra","units":2}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"W317","status":"pending"},{"id":"R962","status":"closed"},{"id":"Y795","status":"pending"},{"id":"B663","status":"ready"},{"id":"J412","status":"pending"}]Checked answer{"ids": ["B663"]}wrong_schema_or_value
{"ids": ["B663", "J412", "Y795", "R962", "W317"]}35 tokensCorrect
{"ids":["B663"]}11 tokensCorrect
{"ids":["B663"]}11 tokensCorrect
{"ids":["B663"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"M228","units":19,"price":13},{"id":"F792","units":2,"price":3},{"id":"X862","units":7,"price":15},{"id":"U360","units":10,"price":1},{"id":"X826","units":14,"price":19}]Checked answer{"total": 634}invalid_json_object
{
"total": 13 + 3 + 15 + 10 + 19 = 52
}30 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"U439","priority":5},{"id":"T130","priority":2},{"id":"V966","priority":2},{"id":"U985","priority":2},{"id":"H364","priority":8}]Checked answer{"ids": ["H364", "U439"]}invalid_json_object
[{"id":"U439"},{"id":"T130"},{"id":"V966"},{"id":"U985"},{"id":"H364"}]43 tokenswrong_schema_or_value
{"ids":["H364","U985"]}16 tokensCorrect
{"ids":["H364","U439"]}16 tokensCorrect
{"ids":["H364","U439"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record J553. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"D198","city":"Accra","units":11},{"id":"E548","city":"Suva","units":4},{"id":"I461","city":"Kyoto","units":6},{"id":"Q220","city":"Bern","units":16},{"id":"J553","city":"Accra","units":19}]Checked answer{"city": "Accra", "units": 19}Correct
{
"city": "Accra",
"units": 19
}19 tokensCorrect
{"city":"Accra","units":19}12 tokensCorrect
{"city":"Accra","units":19}12 tokensCorrect
{"city":"Accra","units":19}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"C443","status":"ready"},{"id":"W496","status":"pending"},{"id":"M654","status":"pending"},{"id":"V954","status":"pending"},{"id":"B528","status":"pending"}]Checked answer{"ids": ["C443"]}wrong_schema_or_value
{"ids": ["C443", "M654", "V954", "B528"]}29 tokensCorrect
{"ids":["C443"]}11 tokensCorrect
{"ids":["C443"]}11 tokensCorrect
{"ids":["C443"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"R715","units":19,"price":16},{"id":"Q615","units":5,"price":6},{"id":"B233","units":2,"price":15},{"id":"I334","units":7,"price":18},{"id":"B343","units":15,"price":5}]Checked answer{"total": 565}invalid_json_object
{"total": 19 * 16 + 5 * 6 + 2 * 15 + 7 * 18 + 15 * 5}39 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"C557","priority":9},{"id":"T297","priority":4},{"id":"M783","priority":6},{"id":"Q592","priority":6},{"id":"B712","priority":1}]Checked answer{"ids": ["C557", "M783"]}wrong_schema_or_value
{"ids": ["M783", "C557", "T297", "Q592", "B712"]}35 tokensCorrect
{"ids":["C557","M783"]}16 tokensCorrect
{"ids":["C557","M783"]}16 tokensCorrect
{"ids":["C557","M783"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id L946 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"L946","city":"Lima","units":18},{"id":"M554","city":"Perth","units":6},{"id":"X689","city":"Oslo","units":8},{"id":"O656","city":"Accra","units":13},{"id":"J998","city":"Bern","units":9}]Checked answer{"city": "Lima", "units": 18}Correct
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensCorrect
{"city":"Lima","units":18}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"W720","status":"closed"},{"id":"Z264","status":"ready"},{"id":"M229","status":"pending"},{"id":"V401","status":"ready"},{"id":"Q358","status":"closed"}]Checked answer{"ids": ["V401", "Z264"]}wrong_schema_or_value
{
"ids": ["M229", "Q358", "Z264", "W720"]
}32 tokensCorrect
{"ids":["V401","Z264"]}16 tokensCorrect
{"ids":["V401","Z264"]}16 tokenswrong_schema_or_value
{"ids":["Z264","V401"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"E856","units":8,"price":14},{"id":"M536","units":9,"price":7},{"id":"W157","units":16,"price":12},{"id":"H684","units":7,"price":18},{"id":"P235","units":9,"price":10}]Checked answer{"total": 583}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"P133","priority":8},{"id":"A885","priority":8},{"id":"D317","priority":5},{"id":"O286","priority":2},{"id":"N503","priority":4}]Checked answer{"ids": ["A885", "P133"]}invalid_json_object
[{"id":"P133","priority":8},{"id":"A885","priority":8},{"id":"D317","priority":5},{"id":"O286","priority":2},{"id":"N503","priority":4}]62 tokensCorrect
{"ids":["A885","P133"]}16 tokensCorrect
{"ids":["A885","P133"]}16 tokensCorrect
{"ids":["A885","P133"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"D842","city":"Oslo","units":2},{"id":"X367","city":"Lima","units":1},{"id":"G279","city":"Kyoto","units":7},{"id":"N525","city":"Accra","units":16},{"id":"E518","city":"Accra","units":13}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"S563","status":"pending"},{"id":"I338","status":"closed"},{"id":"Y281","status":"ready"},{"id":"Y817","status":"closed"},{"id":"A835","status":"ready"}]Checked answer{"ids": ["A835", "Y281"]}invalid_json_object
[{"id":"Y281","status":"ready"},{"id":"Y817","status":"closed"},{"id":"A835","status":"ready"},{"id":"S563","status":"pending"},{"id":"I338","status":"closed"}]63 tokensCorrect
{"ids":["A835","Y281"]}16 tokensCorrect
{"ids":["A835","Y281"]}16 tokensinvalid_json_object
["ids":["A835","Y281"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"I407","units":7,"price":10},{"id":"V671","units":11,"price":8},{"id":"V517","units":3,"price":12},{"id":"A594","units":9,"price":16},{"id":"Z222","units":0,"price":19}]Checked answer{"total": 338}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":378}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"D291","priority":9},{"id":"D545","priority":3},{"id":"G821","priority":4},{"id":"P497","priority":1},{"id":"R215","priority":1}]Checked answer{"ids": ["D291", "G821"]}wrong_schema_or_value
{"ids": ["D291", "P497", "R215", "G821", "D545"]}35 tokensCorrect
{"ids":["D291","G821"]}16 tokensCorrect
{"ids":["D291","G821"]}16 tokensCorrect
{"ids":["D291","G821"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=V577. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"V577","city":"Lima","units":1},{"id":"Z815","city":"Lima","units":5},{"id":"K604","city":"Perth","units":9},{"id":"Q167","city":"Suva","units":17},{"id":"Q235","city":"Kyoto","units":10}]Checked answer{"city": "Lima", "units": 1}Correct
{"city":"Lima","units":1}11 tokensCorrect
{"city":"Lima","units":1}11 tokensCorrect
{"city":"Lima","units":1}11 tokensCorrect
{"city":"Lima","units":1}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"K104","status":"pending"},{"id":"K358","status":"closed"},{"id":"X870","status":"ready"},{"id":"C739","status":"ready"},{"id":"Z757","status":"closed"}]Checked answer{"ids": ["C739", "X870"]}wrong_schema_or_value
{"ids": ["X870", "C739", "K104", "K358", "Z757"]}35 tokensCorrect
{"ids":["C739","X870"]}16 tokensCorrect
{"ids":["C739","X870"]}16 tokensCorrect
{"ids":["C739","X870"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"F126","units":0,"price":10},{"id":"N185","units":7,"price":15},{"id":"T928","units":15,"price":15},{"id":"G181","units":0,"price":3},{"id":"E920","units":3,"price":5}]Checked answer{"total": 345}invalid_json_object
{
"total": 10 + 105 + 135 + 45 + 15 = 285
}34 tokenswrong_schema_or_value
{"total":485}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"V706","priority":3},{"id":"T411","priority":4},{"id":"A134","priority":2},{"id":"W666","priority":2},{"id":"I888","priority":7}]Checked answer{"ids": ["I888", "T411"]}invalid_json_object
[{"id":"V706","priority":3},{"id":"T411","priority":4},{"id":"A134","priority":2},{"id":"W666","priority":2},{"id":"I888","priority":7}]62 tokensCorrect
{"ids":["I888","T411"]}16 tokensCorrect
{"ids":["I888","T411"]}16 tokensCorrect
{"ids":["I888","T411"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record I637. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"L649","city":"Lima","units":19},{"id":"I637","city":"Oslo","units":15},{"id":"G681","city":"Lima","units":0},{"id":"V332","city":"Riga","units":0},{"id":"N615","city":"Suva","units":16}]Checked answer{"city": "Oslo", "units": 15}wrong_schema_or_value
{
"city": "Lima",
"units": 19
}19 tokensCorrect
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Oslo","units":15}12 tokensCorrect
{"city":"Oslo","units":15}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"B506","status":"ready"},{"id":"W393","status":"closed"},{"id":"E134","status":"ready"},{"id":"Y510","status":"pending"},{"id":"W792","status":"pending"}]Checked answer{"ids": ["B506", "E134"]}wrong_schema_or_value
{"ids": ["B506", "E134", "Y510"]}23 tokensCorrect
{"ids":["B506","E134"]}16 tokensCorrect
{"ids":["B506","E134"]}16 tokenswrong_schema_or_value
{"ids":["E134","B506"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"K196","units":8,"price":9},{"id":"T230","units":9,"price":5},{"id":"R203","units":4,"price":3},{"id":"X849","units":9,"price":11},{"id":"J181","units":7,"price":14}]Checked answer{"total": 326}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":348}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Z426","priority":4},{"id":"A635","priority":8},{"id":"J294","priority":2},{"id":"J570","priority":5},{"id":"I713","priority":7}]Checked answer{"ids": ["A635", "I713"]}wrong_schema_or_value
{"ids": ["J294", "J570", "J713", "A635", "Z426"]}35 tokensCorrect
{"ids":["A635","I713"]}16 tokensCorrect
{"ids":["A635","I713"]}16 tokensCorrect
{"ids":["A635","I713"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id O692 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"W875","city":"Oslo","units":4},{"id":"W216","city":"Perth","units":16},{"id":"R273","city":"Perth","units":19},{"id":"O692","city":"Suva","units":4},{"id":"W256","city":"Accra","units":2}]Checked answer{"city": "Suva", "units": 4}Correct
{"city":"Suva","units":4}11 tokensCorrect
{"city":"Suva","units":4}11 tokensCorrect
{"city":"Suva","units":4}11 tokensCorrect
{"city":"Suva","units":4}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"D857","status":"ready"},{"id":"B927","status":"ready"},{"id":"V887","status":"closed"},{"id":"O187","status":"pending"},{"id":"M736","status":"ready"}]Checked answer{"ids": ["B927", "D857", "M736"]}wrong_schema_or_value
{
"ids": ["B927", "D857", "M736", "O187", "V887"]
}38 tokensCorrect
{"ids":["B927","D857","M736"]}21 tokensCorrect
{"ids":["B927","D857","M736"]}21 tokenswrong_schema_or_value
{"ids":["D857","M736","B927"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"C120","units":3,"price":16},{"id":"O439","units":4,"price":4},{"id":"E722","units":18,"price":16},{"id":"A730","units":9,"price":17},{"id":"W971","units":6,"price":17}]Checked answer{"total": 607}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"V892","priority":3},{"id":"K120","priority":5},{"id":"V715","priority":1},{"id":"S136","priority":4},{"id":"U344","priority":2}]Checked answer{"ids": ["K120", "S136"]}invalid_json_object
[{"id":"V892","priority":3},{"id":"U344","priority":2}]26 tokensCorrect
{"ids":["K120","S136"]}16 tokensCorrect
{"ids":["K120","S136"]}16 tokensCorrect
{"ids":["K120","S136"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to R989? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"T611","city":"Oslo","units":18},{"id":"G725","city":"Bern","units":14},{"id":"M189","city":"Perth","units":1},{"id":"R989","city":"Oslo","units":9},{"id":"R953","city":"Kyoto","units":11}]Checked answer{"city": "Oslo", "units": 9}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Oslo","units":9}11 tokensCorrect
{"city":"Oslo","units":9}11 tokensCorrect
{"city":"Oslo","units":9}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"V816","status":"ready"},{"id":"U470","status":"pending"},{"id":"R607","status":"pending"},{"id":"V786","status":"pending"},{"id":"T902","status":"closed"}]Checked answer{"ids": ["V816"]}invalid_json_object
[{"id":"V816"},{"id":"U470"},{"id":"R607"},{"id":"V786"},{"id":"T902"}]43 tokensCorrect
{"ids":["V816"]}11 tokensCorrect
{"ids":["V816"]}11 tokensinvalid_json_object
["ids":["V816"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"I877","units":5,"price":8},{"id":"Z199","units":11,"price":13},{"id":"I378","units":10,"price":4},{"id":"W653","units":17,"price":18},{"id":"C148","units":9,"price":13}]Checked answer{"total": 646}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"W416","priority":4},{"id":"I540","priority":9},{"id":"N297","priority":8},{"id":"U218","priority":2},{"id":"S426","priority":3}]Checked answer{"ids": ["I540", "N297"]}wrong_schema_or_value
{"ids": ["U218", "S426", "N297", "I540", "W416", "N297"]}41 tokensCorrect
{"ids":["I540","N297"]}16 tokensCorrect
{"ids":["I540","N297"]}16 tokensCorrect
{"ids":["I540","N297"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"E840","city":"Perth","units":7},{"id":"R632","city":"Accra","units":13},{"id":"R741","city":"Kyoto","units":13},{"id":"U348","city":"Oslo","units":5},{"id":"F417","city":"Bern","units":3}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Perth","units":7}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"D426","status":"pending"},{"id":"N584","status":"closed"},{"id":"S293","status":"ready"},{"id":"L913","status":"ready"},{"id":"I101","status":"pending"}]Checked answer{"ids": ["L913", "S293"]}wrong_schema_or_value
{"ids": ["D426", "S293", "L913", "N584", "I101"]}35 tokensCorrect
{"ids":["L913","S293"]}16 tokenswrong_schema_or_value
{"ids":["D426","L913","S293"]}21 tokensCorrect
{"ids":["L913","S293"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"E792","units":16,"price":19},{"id":"B750","units":2,"price":19},{"id":"G363","units":10,"price":6},{"id":"U591","units":9,"price":5},{"id":"G877","units":5,"price":15}]Checked answer{"total": 522}invalid_json_object
{
"total": 19 + 38 + 60 + 55 + 75 = 199
}32 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":478}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"D374","priority":6},{"id":"Y562","priority":6},{"id":"S361","priority":7},{"id":"O752","priority":6},{"id":"H834","priority":6}]Checked answer{"ids": ["S361", "D374"]}invalid_json_object
[{"id":"D374"},{"id":"Y562"},{"id":"S361"},{"id":"O752"},{"id":"H834"}]43 tokensCorrect
{"ids":["S361","D374"]}16 tokensCorrect
{"ids":["S361","D374"]}16 tokensCorrect
{"ids":["S361","D374"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record C786. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"H730","city":"Accra","units":0},{"id":"C786","city":"Kyoto","units":15},{"id":"D939","city":"Bern","units":1},{"id":"E270","city":"Accra","units":19},{"id":"S268","city":"Suva","units":2}]Checked answer{"city": "Kyoto", "units": 15}Correct
{
"city": "Kyoto",
"units": 15
}20 tokensCorrect
{"city":"Kyoto","units":15}13 tokensCorrect
{"city":"Kyoto","units":15}13 tokensCorrect
{"city":"Kyoto","units":15}13 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"P267","status":"closed"},{"id":"V776","status":"ready"},{"id":"P674","status":"ready"},{"id":"F840","status":"closed"},{"id":"J102","status":"pending"}]Checked answer{"ids": ["P674", "V776"]}wrong_schema_or_value
{"ids": ["P267", "V776", "P674", "F840", "J102"]}35 tokenswrong_schema_or_value
{"ids":["V776","P674"]}16 tokenswrong_schema_or_value
{"ids":["V776","P674"]}16 tokenswrong_schema_or_value
{"ids":["J102","V776"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"P771","units":18,"price":8},{"id":"A307","units":8,"price":5},{"id":"X422","units":14,"price":13},{"id":"R224","units":15,"price":9},{"id":"G857","units":0,"price":7}]Checked answer{"total": 501}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"U743","priority":5},{"id":"Y246","priority":9},{"id":"E938","priority":9},{"id":"D631","priority":5},{"id":"F651","priority":5}]Checked answer{"ids": ["E938", "Y246"]}wrong_schema_or_value
{"ids": ["U743", "F651", "D631", "E938", "Y246"]}35 tokensCorrect
{"ids":["E938","Y246"]}16 tokensCorrect
{"ids":["E938","Y246"]}16 tokensCorrect
{"ids":["E938","Y246"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id M536 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"M613","city":"Lima","units":19},{"id":"T817","city":"Bern","units":13},{"id":"C764","city":"Lima","units":15},{"id":"C348","city":"Perth","units":2},{"id":"M536","city":"Riga","units":13}]Checked answer{"city": "Riga", "units": 13}wrong_schema_or_value
{"city":"Lima","units":19}12 tokensCorrect
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensCorrect
{"city":"Riga","units":13}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"M950","status":"pending"},{"id":"T356","status":"ready"},{"id":"C339","status":"closed"},{"id":"R667","status":"closed"},{"id":"T947","status":"pending"}]Checked answer{"ids": ["T356"]}wrong_schema_or_value
{
"ids": ["M950", "T356", "C339", "R667", "T947"]
}38 tokensCorrect
{"ids":["T356"]}11 tokensCorrect
{"ids":["T356"]}11 tokensCorrect
{"ids":["T356"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"V268","units":16,"price":11},{"id":"E874","units":19,"price":5},{"id":"P738","units":12,"price":14},{"id":"M789","units":17,"price":19},{"id":"O576","units":7,"price":10}]Checked answer{"total": 832}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"S111","priority":4},{"id":"O835","priority":4},{"id":"X555","priority":6},{"id":"T865","priority":9},{"id":"M904","priority":8}]Checked answer{"ids": ["T865", "M904"]}invalid_json_object
[{"id":"S111","priority":4},{"id":"O835","priority":4},{"id":"X555","priority":6},{"id":"T865","priority":9},{"id":"M904","priority":8}]62 tokensCorrect
{"ids":["T865","M904"]}16 tokensCorrect
{"ids":["T865","M904"]}16 tokensCorrect
{"ids":["T865","M904"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to J852? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"J852","city":"Perth","units":17},{"id":"O510","city":"Kyoto","units":7},{"id":"X679","city":"Kyoto","units":13},{"id":"E393","city":"Oslo","units":5},{"id":"H334","city":"Riga","units":10}]Checked answer{"city": "Perth", "units": 17}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensCorrect
{"city":"Perth","units":17}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"N496","status":"pending"},{"id":"Y338","status":"pending"},{"id":"U801","status":"closed"},{"id":"Q771","status":"closed"},{"id":"Z348","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"N496"},{"id":"Y338"},{"id":"U801"},{"id":"Q771"},{"id":"Z348"}]43 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensinvalid_json_object
["ids":[]}6 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"F185","units":17,"price":9},{"id":"V223","units":19,"price":3},{"id":"B676","units":12,"price":12},{"id":"U769","units":5,"price":2},{"id":"N246","units":18,"price":5}]Checked answer{"total": 454}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"J922","priority":8},{"id":"G422","priority":8},{"id":"N196","priority":7},{"id":"E556","priority":7},{"id":"V110","priority":6}]Checked answer{"ids": ["G422", "J922"]}wrong_schema_or_value
{"ids": ["J922", "G422"]}17 tokenswrong_schema_or_value
{"ids":["J922","G422"]}16 tokenswrong_schema_or_value
{"ids":["J922","G422"]}16 tokenswrong_schema_or_value
{"ids":["J922","G422"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=H733. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"H733","city":"Kyoto","units":2},{"id":"D472","city":"Riga","units":9},{"id":"N488","city":"Riga","units":8},{"id":"W396","city":"Bern","units":15},{"id":"P986","city":"Riga","units":19}]Checked answer{"city": "Kyoto", "units": 2}Correct
{"city":"Kyoto","units":2}12 tokensCorrect
{"city":"Kyoto","units":2}12 tokensCorrect
{"city":"Kyoto","units":2}12 tokensCorrect
{"city":"Kyoto","units":2}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"V133","status":"closed"},{"id":"H804","status":"ready"},{"id":"V339","status":"ready"},{"id":"C770","status":"pending"},{"id":"Z295","status":"pending"}]Checked answer{"ids": ["H804", "V339"]}wrong_schema_or_value
{"ids": ["H804", "V133", "V339", "C770", "Z295"]}35 tokensCorrect
{"ids":["H804","V339"]}16 tokensCorrect
{"ids":["H804","V339"]}16 tokensCorrect
{"ids":["H804","V339"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"B805","units":5,"price":3},{"id":"A743","units":1,"price":14},{"id":"W818","units":14,"price":4},{"id":"A753","units":17,"price":15},{"id":"C192","units":17,"price":1}]Checked answer{"total": 357}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"O136","priority":4},{"id":"P737","priority":9},{"id":"W444","priority":4},{"id":"G316","priority":7},{"id":"W160","priority":9}]Checked answer{"ids": ["P737", "W160"]}invalid_json_object
[{"id":"O136"},{"id":"P737"},{"id":"W444"},{"id":"G316"},{"id":"W160"}]43 tokenswrong_schema_or_value
{"ids":["W160","P737"]}16 tokenswrong_schema_or_value
{"ids":["W160","P737"]}16 tokenswrong_schema_or_value
{"ids":["O136","W160"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"P188","city":"Oslo","units":14},{"id":"P912","city":"Oslo","units":12},{"id":"U762","city":"Riga","units":17},{"id":"L175","city":"Oslo","units":11},{"id":"C219","city":"Perth","units":17}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{
"city": "Oslo",
"units": 14
}19 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Q926","status":"pending"},{"id":"M753","status":"closed"},{"id":"Y159","status":"ready"},{"id":"Z261","status":"ready"},{"id":"P958","status":"closed"}]Checked answer{"ids": ["Y159", "Z261"]}wrong_schema_or_value
{"ids": ["Y159", "Z261", "P958", "M753", "Q926"]}35 tokensCorrect
{"ids":["Y159","Z261"]}16 tokensCorrect
{"ids":["Y159","Z261"]}16 tokensCorrect
{"ids":["Y159","Z261"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Y581","units":15,"price":3},{"id":"N285","units":16,"price":2},{"id":"T470","units":11,"price":14},{"id":"L658","units":2,"price":10},{"id":"X656","units":14,"price":5}]Checked answer{"total": 321}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"G497","priority":4},{"id":"G798","priority":2},{"id":"P806","priority":8},{"id":"M790","priority":7},{"id":"U340","priority":9}]Checked answer{"ids": ["U340", "P806"]}wrong_schema_or_value
{"ids": ["G798", "G497", "P806", "M790", "U340"]}35 tokenswrong_schema_or_value
{"ids":["M790","U340"]}16 tokenswrong_schema_or_value
{"ids":["M790","U340"]}16 tokenswrong_schema_or_value
{"ids":["U340","M790"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id N415 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"N415","city":"Lima","units":11},{"id":"F817","city":"Bern","units":17},{"id":"E380","city":"Lima","units":16},{"id":"X385","city":"Kyoto","units":2},{"id":"U750","city":"Oslo","units":3}]Checked answer{"city": "Lima", "units": 11}Correct
{"city":"Lima","units":11}12 tokensCorrect
{"city":"Lima","units":11}12 tokensCorrect
{"city":"Lima","units":11}12 tokensCorrect
{"city":"Lima","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"N368","status":"pending"},{"id":"O183","status":"closed"},{"id":"Z485","status":"closed"},{"id":"P866","status":"pending"},{"id":"A666","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{
"ids": ["A666", "P866", "N368", "O183", "Z485"]
}38 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"F813","units":19,"price":19},{"id":"X231","units":16,"price":2},{"id":"B574","units":15,"price":6},{"id":"J301","units":11,"price":13},{"id":"H837","units":6,"price":13}]Checked answer{"total": 704}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"V510","priority":5},{"id":"O323","priority":9},{"id":"M236","priority":9},{"id":"A466","priority":9},{"id":"M714","priority":5}]Checked answer{"ids": ["A466", "M236"]}invalid_json_object
[{"id":"V510","priority":5},{"id":"O323","priority":9},{"id":"M236","priority":9},{"id":"A466","priority":9},{"id":"M714","priority":5}]62 tokensCorrect
{"ids":["A466","M236"]}16 tokensCorrect
{"ids":["A466","M236"]}16 tokenswrong_schema_or_value
{"ids":["M236","O323"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to G758? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"W865","city":"Suva","units":10},{"id":"G758","city":"Oslo","units":0},{"id":"M632","city":"Lima","units":13},{"id":"B419","city":"Riga","units":9},{"id":"P937","city":"Bern","units":17}]Checked answer{"city": "Oslo", "units": 0}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokenswrong_schema_or_value
{"city":null,"units":0}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Oslo","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"J743","status":"closed"},{"id":"N716","status":"pending"},{"id":"P354","status":"closed"},{"id":"Z155","status":"closed"},{"id":"S550","status":"pending"}]Checked answer{"ids": []}invalid_json_object
[{"id":"J743"},{"id":"N716"},{"id":"P354"},{"id":"Z155"},{"id":"S550"}]43 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"A597","units":1,"price":15},{"id":"S642","units":6,"price":15},{"id":"V595","units":14,"price":6},{"id":"O277","units":16,"price":6},{"id":"Y442","units":6,"price":7}]Checked answer{"total": 327}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"J658","priority":3},{"id":"R143","priority":9},{"id":"A339","priority":7},{"id":"U144","priority":3},{"id":"Q649","priority":6}]Checked answer{"ids": ["R143", "A339"]}wrong_schema_or_value
{"ids": ["U144", "A339", "Q649", "J658", "R143", "J658"]}41 tokensCorrect
{"ids":["R143","A339"]}16 tokensCorrect
{"ids":["R143","A339"]}16 tokensCorrect
{"ids":["R143","A339"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z665. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"S823","city":"Oslo","units":10},{"id":"U249","city":"Kyoto","units":14},{"id":"W663","city":"Accra","units":2},{"id":"Z665","city":"Perth","units":0},{"id":"E928","city":"Suva","units":13}]Checked answer{"city": "Perth", "units": 0}Correct
{"city":"Perth","units":0}11 tokenswrong_schema_or_value
{"city":null,"units":0}10 tokenswrong_schema_or_value
{"city":null,"units":0}10 tokensCorrect
{"city":"Perth","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"S976","status":"pending"},{"id":"F493","status":"pending"},{"id":"A449","status":"pending"},{"id":"P274","status":"pending"},{"id":"F339","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["A449", "F339", "F493", "S976", "P274"]}35 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensCorrect
{"ids":[]}6 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"C287","units":7,"price":19},{"id":"C227","units":11,"price":19},{"id":"D965","units":9,"price":16},{"id":"H810","units":18,"price":2},{"id":"L646","units":0,"price":3}]Checked answer{"total": 522}invalid_json_object
{
"total": 19 * 7 + 19 * 11 + 16 * 9 + 2 * 18 + 3 * 0
}42 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":468}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"T895","priority":1},{"id":"O259","priority":9},{"id":"A773","priority":7},{"id":"N250","priority":8},{"id":"Q138","priority":3}]Checked answer{"ids": ["O259", "N250"]}invalid_json_object
[{"id":"T895","priority":1},{"id":"O259","priority":9},{"id":"A773","priority":7},{"id":"N250","priority":8},{"id":"Q138","priority":3}]62 tokensCorrect
{"ids":["O259","N250"]}16 tokenswrong_schema_or_value
{"ids":["N250","O259"]}16 tokenswrong_schema_or_value
{"ids":["O259","A773"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record U259. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"M874","city":"Lima","units":0},{"id":"U259","city":"Suva","units":11},{"id":"G515","city":"Bern","units":2},{"id":"D976","city":"Perth","units":14},{"id":"G914","city":"Suva","units":0}]Checked answer{"city": "Suva", "units": 11}Correct
{
"city": "Suva",
"units": 11
}19 tokensCorrect
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Suva","units":11}12 tokensCorrect
{"city":"Suva","units":11}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"D444","status":"pending"},{"id":"P835","status":"closed"},{"id":"H800","status":"ready"},{"id":"B797","status":"ready"},{"id":"W114","status":"closed"}]Checked answer{"ids": ["B797", "H800"]}wrong_schema_or_value
{"ids": ["H800", "B797", "D444", "P835", "W114"]}35 tokensCorrect
{"ids":["B797","H800"]}16 tokensCorrect
{"ids":["B797","H800"]}16 tokenswrong_schema_or_value
{"ids":["H800","B797"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"U116","units":2,"price":2},{"id":"D177","units":0,"price":5},{"id":"D789","units":5,"price":6},{"id":"H212","units":8,"price":6},{"id":"R561","units":11,"price":5}]Checked answer{"total": 137}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":28}7 tokenswrong_schema_or_value
{"total":278}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Y839","priority":2},{"id":"R227","priority":5},{"id":"V691","priority":6},{"id":"D446","priority":8},{"id":"D343","priority":6}]Checked answer{"ids": ["D446", "D343"]}wrong_schema_or_value
{"ids": ["R227", "Y839", "V691", "D446", "D343"]}35 tokenswrong_schema_or_value
{"ids":["D446","R227"]}16 tokenswrong_schema_or_value
{"ids":["D446","R227"]}16 tokenswrong_schema_or_value
{"ids":["D446","V691"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"F635","city":"Riga","units":6},{"id":"O736","city":"Accra","units":11},{"id":"X224","city":"Suva","units":13},{"id":"I416","city":"Kyoto","units":1},{"id":"O333","city":"Perth","units":0}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Riga","units":6}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"O825","status":"ready"},{"id":"S429","status":"closed"},{"id":"Q645","status":"pending"},{"id":"G454","status":"closed"},{"id":"M807","status":"ready"}]Checked answer{"ids": ["M807", "O825"]}wrong_schema_or_value
{
"ids": ["O825", "S429", "G454", "M807"]
}32 tokensCorrect
{"ids":["M807","O825"]}16 tokensCorrect
{"ids":["M807","O825"]}16 tokenswrong_schema_or_value
{"ids":["O825","M807"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"D842","units":0,"price":12},{"id":"B162","units":17,"price":3},{"id":"U292","units":19,"price":14},{"id":"F764","units":8,"price":17},{"id":"L374","units":1,"price":19}]Checked answer{"total": 472}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"U453","priority":8},{"id":"B987","priority":3},{"id":"I461","priority":1},{"id":"W565","priority":7},{"id":"M744","priority":8}]Checked answer{"ids": ["M744", "U453"]}invalid_json_object
[{"id":"U453","priority":8},{"id":"B987","priority":3},{"id":"I461","priority":1},{"id":"W565","priority":7},{"id":"M744","priority":8}]62 tokensCorrect
{"ids":["M744","U453"]}16 tokensCorrect
{"ids":["M744","U453"]}16 tokensCorrect
{"ids":["M744","U453"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to C365? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"C365","city":"Kyoto","units":5},{"id":"N506","city":"Riga","units":14},{"id":"P944","city":"Accra","units":0},{"id":"A133","city":"Oslo","units":16},{"id":"Z597","city":"Riga","units":0}]Checked answer{"city": "Kyoto", "units": 5}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokenswrong_schema_or_value
{"city":null,"units":null}10 tokensCorrect
{"city":"Kyoto","units":5}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"X747","status":"pending"},{"id":"C940","status":"ready"},{"id":"S955","status":"pending"},{"id":"Y430","status":"ready"},{"id":"F898","status":"closed"}]Checked answer{"ids": ["C940", "Y430"]}invalid_json_object
[{"id":"C940","status":"ready"},{"id":"Y430","status":"ready"},{"id":"F898","status":"closed"}]39 tokensCorrect
{"ids":["C940","Y430"]}16 tokensCorrect
{"ids":["C940","Y430"]}16 tokensinvalid_json_object
["ids":["C940","Y430"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"A451","units":12,"price":9},{"id":"V112","units":9,"price":5},{"id":"T586","units":12,"price":11},{"id":"Z486","units":12,"price":4},{"id":"X336","units":11,"price":8}]Checked answer{"total": 421}wrong_schema_or_value
{"total": 120}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"N190","priority":1},{"id":"W374","priority":4},{"id":"D678","priority":8},{"id":"W362","priority":5},{"id":"R311","priority":8}]Checked answer{"ids": ["D678", "R311"]}wrong_schema_or_value
{"ids": ["N190", "W374", "D678", "R311", "W362"]}35 tokensCorrect
{"ids":["D678","R311"]}16 tokensCorrect
{"ids":["D678","R311"]}16 tokensCorrect
{"ids":["D678","R311"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=G560. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"P844","city":"Oslo","units":7},{"id":"G560","city":"Perth","units":19},{"id":"O371","city":"Suva","units":18},{"id":"H971","city":"Bern","units":17},{"id":"T852","city":"Perth","units":3}]Checked answer{"city": "Perth", "units": 19}wrong_schema_or_value
{"city":"Oslo","units":7}11 tokensCorrect
{"city":"Perth","units":19}12 tokensCorrect
{"city":"Perth","units":19}12 tokensCorrect
{"city":"Perth","units":19}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"X678","status":"pending"},{"id":"O813","status":"ready"},{"id":"G474","status":"pending"},{"id":"Y674","status":"pending"},{"id":"F153","status":"ready"}]Checked answer{"ids": ["F153", "O813"]}wrong_schema_or_value
{"ids": ["O813", "G474", "F153", "X678", "Y674"]}35 tokensCorrect
{"ids":["F153","O813"]}16 tokensCorrect
{"ids":["F153","O813"]}16 tokensCorrect
{"ids":["F153","O813"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"K292","units":9,"price":18},{"id":"B109","units":11,"price":13},{"id":"S749","units":6,"price":18},{"id":"U331","units":13,"price":5},{"id":"K125","units":6,"price":17}]Checked answer{"total": 580}wrong_schema_or_value
{
"total": 122
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"O119","priority":6},{"id":"U812","priority":5},{"id":"G876","priority":2},{"id":"Z420","priority":9},{"id":"U283","priority":3}]Checked answer{"ids": ["Z420", "O119"]}invalid_json_object
[{"id":"U283","priority":3},{"id":"U812","priority":5},{"id":"G876","priority":2},{"id":"O119","priority":6},{"id":"Z420","priority":9}]62 tokenswrong_schema_or_value
{"ids":["O119","Z420"]}16 tokenswrong_schema_or_value
{"ids":["O119","Z420"]}16 tokensCorrect
{"ids":["Z420","O119"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record J940. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"J940","city":"Oslo","units":8},{"id":"L478","city":"Bern","units":8},{"id":"M455","city":"Riga","units":1},{"id":"A711","city":"Lima","units":2},{"id":"V731","city":"Suva","units":18}]Checked answer{"city": "Oslo", "units": 8}Correct
{
"city": "Oslo",
"units": 8
}18 tokensCorrect
{"city":"Oslo","units":8}11 tokensCorrect
{"city":"Oslo","units":8}11 tokensCorrect
{"city":"Oslo","units":8}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"K775","status":"closed"},{"id":"W397","status":"pending"},{"id":"Y665","status":"ready"},{"id":"G129","status":"closed"},{"id":"M295","status":"closed"}]Checked answer{"ids": ["Y665"]}wrong_schema_or_value
{"ids": ["Y665", "G129", "M295", "K775", "W397"]}35 tokensCorrect
{"ids":["Y665"]}11 tokensCorrect
{"ids":["Y665"]}11 tokensCorrect
{"ids":["Y665"]}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"H449","units":8,"price":16},{"id":"S516","units":15,"price":15},{"id":"L684","units":4,"price":3},{"id":"I455","units":5,"price":7},{"id":"I495","units":6,"price":12}]Checked answer{"total": 472}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"J300","priority":1},{"id":"N564","priority":3},{"id":"I823","priority":1},{"id":"W323","priority":2},{"id":"F616","priority":8}]Checked answer{"ids": ["F616", "N564"]}wrong_schema_or_value
{"ids": ["J300", "I823", "W323", "F616"]}29 tokensCorrect
{"ids":["F616","N564"]}16 tokensCorrect
{"ids":["F616","N564"]}16 tokensCorrect
{"ids":["F616","N564"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id S579 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"D662","city":"Accra","units":2},{"id":"S579","city":"Perth","units":5},{"id":"D107","city":"Accra","units":16},{"id":"A529","city":"Suva","units":12},{"id":"E524","city":"Accra","units":12}]Checked answer{"city": "Perth", "units": 5}wrong_schema_or_value
{"city":"Accra","units":16}12 tokensCorrect
{"city":"Perth","units":5}11 tokensCorrect
{"city":"Perth","units":5}11 tokensCorrect
{"city":"Perth","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"M662","status":"closed"},{"id":"N643","status":"ready"},{"id":"P162","status":"closed"},{"id":"B803","status":"ready"},{"id":"C823","status":"ready"}]Checked answer{"ids": ["B803", "C823", "N643"]}wrong_schema_or_value
{
"ids": ["B803", "C823", "M662", "N643", "P162"]
}38 tokensCorrect
{"ids":["B803","C823","N643"]}21 tokensCorrect
{"ids":["B803","C823","N643"]}21 tokenswrong_schema_or_value
{"ids":["C823","N643","B803"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"B893","units":16,"price":19},{"id":"A270","units":6,"price":9},{"id":"K871","units":7,"price":5},{"id":"R813","units":7,"price":14},{"id":"L765","units":13,"price":10}]Checked answer{"total": 621}wrong_schema_or_value
{"total": 103}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"A706","priority":4},{"id":"M487","priority":7},{"id":"D221","priority":5},{"id":"T626","priority":6},{"id":"K405","priority":9}]Checked answer{"ids": ["K405", "M487"]}invalid_json_object
[{"id":"A706","priority":4},{"id":"M487","priority":7},{"id":"D221","priority":5},{"id":"T626","priority":6},{"id":"K405","priority":9}]62 tokensCorrect
{"ids":["K405","M487"]}16 tokensCorrect
{"ids":["K405","M487"]}16 tokensCorrect
{"ids":["K405","M487"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"K962","city":"Kyoto","units":4},{"id":"B968","city":"Riga","units":14},{"id":"H984","city":"Lima","units":7},{"id":"P348","city":"Suva","units":2},{"id":"E503","city":"Kyoto","units":13}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"U839","status":"closed"},{"id":"N287","status":"pending"},{"id":"P935","status":"pending"},{"id":"I270","status":"ready"},{"id":"X881","status":"closed"}]Checked answer{"ids": ["I270"]}invalid_json_object
[{"id":"I270","status":"ready"}]15 tokensCorrect
{"ids":["I270"]}11 tokensCorrect
{"ids":["I270"]}11 tokensinvalid_json_object
["ids":["I270"]]11 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"R854","units":19,"price":13},{"id":"V165","units":13,"price":5},{"id":"T877","units":17,"price":17},{"id":"Y186","units":10,"price":13},{"id":"Z115","units":0,"price":15}]Checked answer{"total": 731}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":648}8 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"L127","priority":5},{"id":"E102","priority":5},{"id":"D910","priority":5},{"id":"K526","priority":7},{"id":"C537","priority":2}]Checked answer{"ids": ["K526", "D910"]}wrong_schema_or_value
{"ids": ["L127", "E102", "D910", "K526", "C537"]}35 tokenswrong_schema_or_value
{"ids":["K526","L127"]}16 tokenswrong_schema_or_value
{"ids":["K526","L127"]}16 tokenswrong_schema_or_value
{"ids":["K526","L127"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=N920. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"F242","city":"Bern","units":4},{"id":"S919","city":"Suva","units":0},{"id":"Q505","city":"Bern","units":0},{"id":"Y161","city":"Oslo","units":5},{"id":"N920","city":"Kyoto","units":2}]Checked answer{"city": "Kyoto", "units": 2}wrong_schema_or_value
{"city":"Bern","units":4}10 tokensCorrect
{"city":"Kyoto","units":2}12 tokensCorrect
{"city":"Kyoto","units":2}12 tokensCorrect
{"city":"Kyoto","units":2}12 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"V433","status":"ready"},{"id":"E925","status":"pending"},{"id":"F834","status":"ready"},{"id":"P223","status":"closed"},{"id":"H791","status":"pending"}]Checked answer{"ids": ["F834", "V433"]}wrong_schema_or_value
{"ids": ["F834", "V433", "E925", "H791", "P223"]}35 tokensCorrect
{"ids":["F834","V433"]}16 tokensCorrect
{"ids":["F834","V433"]}16 tokensCorrect
{"ids":["F834","V433"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"V886","units":4,"price":5},{"id":"T148","units":15,"price":10},{"id":"M872","units":9,"price":18},{"id":"U593","units":6,"price":1},{"id":"N842","units":12,"price":11}]Checked answer{"total": 470}wrong_schema_or_value
{
"total": 100
}12 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":448}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"K142","priority":5},{"id":"O510","priority":3},{"id":"C633","priority":3},{"id":"B251","priority":9},{"id":"Y747","priority":4}]Checked answer{"ids": ["B251", "K142"]}invalid_json_object
[{"id":"K142","priority":5},{"id":"O510","priority":3},{"id":"C633","priority":3},{"id":"B251","priority":9},{"id":"Y747","priority":4}]62 tokenswrong_schema_or_value
{"ids":["B251","C633"]}16 tokensCorrect
{"ids":["B251","K142"]}16 tokensCorrect
{"ids":["B251","K142"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record G233. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"V284","city":"Bern","units":17},{"id":"G233","city":"Riga","units":5},{"id":"E102","city":"Perth","units":16},{"id":"I924","city":"Kyoto","units":0},{"id":"M745","city":"Kyoto","units":13}]Checked answer{"city": "Riga", "units": 5}Correct
{
"city": "Riga",
"units": 5
}18 tokensCorrect
{"city":"Riga","units":5}11 tokensCorrect
{"city":"Riga","units":5}11 tokensCorrect
{"city":"Riga","units":5}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"G265","status":"ready"},{"id":"N938","status":"closed"},{"id":"R493","status":"closed"},{"id":"J797","status":"ready"},{"id":"A376","status":"ready"}]Checked answer{"ids": ["A376", "G265", "J797"]}wrong_schema_or_value
{"ids": ["A376", "G265", "J797", "N938", "R493"]}35 tokensCorrect
{"ids":["A376","G265","J797"]}21 tokensCorrect
{"ids":["A376","G265","J797"]}21 tokenswrong_schema_or_value
{"ids":["G265","J797","A376"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"B455","units":7,"price":7},{"id":"G923","units":1,"price":12},{"id":"P300","units":6,"price":18},{"id":"F954","units":0,"price":2},{"id":"N647","units":17,"price":17}]Checked answer{"total": 458}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"S637","priority":1},{"id":"R478","priority":7},{"id":"O956","priority":9},{"id":"X113","priority":6},{"id":"V313","priority":1}]Checked answer{"ids": ["O956", "R478"]}wrong_schema_or_value
{"ids": ["S637", "R478", "X113", "V313", "O956"]}35 tokensCorrect
{"ids":["O956","R478"]}16 tokensCorrect
{"ids":["O956","R478"]}16 tokenswrong_schema_or_value
{"ids":["O956","S637"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id M248 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"M248","city":"Suva","units":0},{"id":"B525","city":"Perth","units":17},{"id":"B383","city":"Perth","units":15},{"id":"H801","city":"Perth","units":7},{"id":"I941","city":"Kyoto","units":8}]Checked answer{"city": "Suva", "units": 0}wrong_schema_or_value
{"city":"Suva","units":null}11 tokensCorrect
{"city":"Suva","units":0}11 tokensCorrect
{"city":"Suva","units":0}11 tokensCorrect
{"city":"Suva","units":0}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"E579","status":"ready"},{"id":"Z379","status":"pending"},{"id":"Q268","status":"pending"},{"id":"C958","status":"ready"},{"id":"O903","status":"ready"}]Checked answer{"ids": ["C958", "E579", "O903"]}wrong_schema_or_value
{
"ids": ["C958", "E579", "O903", "Q268"]
}32 tokensCorrect
{"ids":["C958","E579","O903"]}21 tokensCorrect
{"ids":["C958","E579","O903"]}21 tokenswrong_schema_or_value
{"ids":["E579","C958","O903"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"Q159","units":1,"price":13},{"id":"V941","units":0,"price":19},{"id":"Z446","units":2,"price":15},{"id":"B481","units":15,"price":19},{"id":"V899","units":14,"price":10}]Checked answer{"total": 468}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":477}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"M974","priority":5},{"id":"P256","priority":3},{"id":"O821","priority":1},{"id":"A825","priority":8},{"id":"L906","priority":1}]Checked answer{"ids": ["A825", "M974"]}invalid_json_object
[{"id":"M974","priority":5},{"id":"P256","priority":3},{"id":"O821","priority":1},{"id":"A825","priority":8},{"id":"L906","priority":1}]62 tokensCorrect
{"ids":["A825","M974"]}16 tokensCorrect
{"ids":["A825","M974"]}16 tokensCorrect
{"ids":["A825","M974"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to F403? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"E292","city":"Bern","units":7},{"id":"F403","city":"Bern","units":10},{"id":"S327","city":"Lima","units":13},{"id":"O231","city":"Suva","units":18},{"id":"X567","city":"Accra","units":0}]Checked answer{"city": "Bern", "units": 10}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokensCorrect
{"city":"Bern","units":10}11 tokensCorrect
{"city":"Bern","units":10}11 tokensCorrect
{"city":"Bern","units":10}11 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"T119","status":"pending"},{"id":"J148","status":"pending"},{"id":"P274","status":"closed"},{"id":"R661","status":"ready"},{"id":"Y922","status":"ready"}]Checked answer{"ids": ["R661", "Y922"]}invalid_json_object
[{"id":"R661","status":"ready"},{"id":"Y922","status":"ready"}]27 tokensinvalid_json_object
["R661","Y922"]12 tokens
Correct
{"ids":["R661","Y922"]}16 tokensinvalid_json_object
["ids":["R661","Y922"]]16 tokens
Use only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"B346","units":9,"price":15},{"id":"B624","units":16,"price":4},{"id":"C190","units":0,"price":10},{"id":"B207","units":16,"price":8},{"id":"R700","units":5,"price":10}]Checked answer{"total": 377}wrong_schema_or_value
{"total": 100}9 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":478}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Y631","priority":6},{"id":"J690","priority":8},{"id":"R477","priority":4},{"id":"C949","priority":5},{"id":"F205","priority":6}]Checked answer{"ids": ["J690", "F205"]}wrong_schema_or_value
{"ids": ["J690", "Y631", "R477", "C949", "F205"]}35 tokensCorrect
{"ids":["J690","F205"]}16 tokenswrong_schema_or_value
{"ids":["J690","Y631"]}16 tokenswrong_schema_or_value
{"ids":["J690","C949"]}16 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"P767","city":"Suva","units":9},{"id":"N880","city":"Lima","units":11},{"id":"M637","city":"Perth","units":11},{"id":"O572","city":"Kyoto","units":5},{"id":"J452","city":"Bern","units":8}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Suva","units":9}11 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensCorrect
{"city":null,"units":null}10 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"H179","status":"closed"},{"id":"K608","status":"ready"},{"id":"P657","status":"closed"},{"id":"I233","status":"ready"},{"id":"L359","status":"ready"}]Checked answer{"ids": ["I233", "K608", "L359"]}wrong_schema_or_value
{"ids": ["H179", "K608", "P657", "I233", "L359"]}35 tokenswrong_schema_or_value
{"ids":["I233","L359","K608"]}21 tokenswrong_schema_or_value
{"ids":["I233","L359","K608"]}21 tokensCorrect
{"ids":["I233","K608","L359"]}21 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"L303","units":18,"price":19},{"id":"L953","units":12,"price":9},{"id":"W563","units":14,"price":14},{"id":"L729","units":7,"price":2},{"id":"B116","units":18,"price":8}]Checked answer{"total": 804}invalid_json_object
{
"total": 19 + 156 + 78 + 114 + 174 = 406
}35 tokenswrong_schema_or_value
{"total":578}8 tokenswrong_schema_or_value
{"total":488}8 tokenswrong_schema_or_value
{"total":407}8 tokensUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"X340","priority":1},{"id":"W160","priority":1},{"id":"T700","priority":7},{"id":"C575","priority":3},{"id":"B815","priority":3}]Checked answer{"ids": ["T700", "B815"]}invalid_json_object
[{"id":"X340"},{"id":"W160"},{"id":"T700"},{"id":"C575"},{"id":"B815"}]43 tokenswrong_schema_or_value
{"ids":["T700","C575"]}16 tokenswrong_schema_or_value
{"ids":["T700","C575"]}16 tokenswrong_schema_or_value
{"ids":["T700","C575"]}16 tokensLZ · Apache-2.0 · All final candidates and evaluations retained; no native promotion.