NeuroShard / Research evidence / 13 September 2026
Three fixed training arms start from the same 1.7B seed. Checked task targets, a deliberately damaged-target control, and a shared two-GPU run separate target quality from hardware cooperation. These are operated experiments; the public native serving model and ledger are separate.
| Model | Exact answers | Valid JSON objects | Mean retention change |
|---|---|---|---|
| Seed | 23 / 256 (8.98%) | 198 | — |
| Clean · 1 GPU | 169 / 256 (66.02%) | 256 | -0.002195 |
| Damaged · 1 GPU | 152 / 256 (59.38%) | 256 | -0.003081 |
| Clean · 2 GPUs | 168 / 256 (65.62%) | 256 | -0.002346 |
Methods and interpretation · Recomputed metrics · Frozen inputs · Candidate commitments · Checksums · Clean single weights · Shared training weights
Every answer is shown. The two unseen prompt variants are marked 2 and 3. “Correct” requires both the requested JSON schema and exact values; valid JSON alone earns no credit.
Use only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"H635","city":"Oslo","units":2},{"id":"F873","city":"Bern","units":0},{"id":"E150","city":"Suva","units":1},{"id":"T980","city":"Suva","units":2},{"id":"Y205","city":"Kyoto","units":11}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Oslo","units":2}]12 tokens · 2.211sCorrect
{"city":null,"units":null}10 tokens · 1.340sCorrect
{"city":null,"units":null}10 tokens · 1.211sCorrect
{"city":null,"units":null}10 tokens · 0.880sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"U984","status":"ready"},{"id":"R199","status":"pending"},{"id":"J556","status":"pending"},{"id":"X404","status":"closed"},{"id":"A572","status":"pending"}]Checked answer{"ids": ["U984"]}wrong_schema_or_value
{"ids": ["A572", "J556", "U984", "R199", "X404"]}35 tokens · 0.972sCorrect
{"ids":["U984"]}11 tokens · 0.316sCorrect
{"ids":["U984"]}11 tokens · 0.441sCorrect
{"ids":["U984"]}11 tokens · 0.323sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Q857","units":5,"price":7},{"id":"S261","units":6,"price":4},{"id":"F625","units":4,"price":16},{"id":"F534","units":18,"price":9},{"id":"F501","units":10,"price":8}]Checked answer{"total": 365}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":431}8 tokens · 0.226swrong_schema_or_value
{"total":342}8 tokens · 0.330swrong_schema_or_value
{"total":431}8 tokens · 0.225sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"U687","priority":2},{"id":"X278","priority":6},{"id":"T638","priority":8},{"id":"Q352","priority":6},{"id":"E277","priority":2}]Checked answer{"ids": ["T638", "Q352"]}wrong_schema_or_value
{"ids": ["U687", "X278", "T638", "Q352", "E277"]}35 tokens · 0.959sCorrect
{"ids":["T638","Q352"]}16 tokens · 0.441sCorrect
{"ids":["T638","Q352"]}16 tokens · 0.676sCorrect
{"ids":["T638","Q352"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id C398 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"C398","city":"Oslo","units":0},{"id":"Z327","city":"Perth","units":15},{"id":"V662","city":"Perth","units":19},{"id":"Z297","city":"Suva","units":12},{"id":"O756","city":"Suva","units":4}]Checked answer{"city": "Oslo", "units": 0}Correct
{"city":"Oslo","units":0}11 tokens · 0.307sCorrect
{"city":"Oslo","units":0}11 tokens · 0.309sCorrect
{"city":"Oslo","units":0}11 tokens · 0.480sCorrect
{"city":"Oslo","units":0}11 tokens · 0.310sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"W186","status":"pending"},{"id":"X341","status":"closed"},{"id":"J338","status":"pending"},{"id":"T557","status":"pending"},{"id":"D680","status":"ready"}]Checked answer{"ids": ["D680"]}wrong_schema_or_value
{
"ids": ["D680", "J338", "T557", "W186", "X341"]
}38 tokens · 1.038sCorrect
{"ids":["D680"]}11 tokens · 0.310sCorrect
{"ids":["D680"]}11 tokens · 0.436sCorrect
{"ids":["D680"]}11 tokens · 0.310sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"G781","units":14,"price":2},{"id":"Z845","units":13,"price":17},{"id":"Q815","units":9,"price":13},{"id":"O480","units":8,"price":5},{"id":"Y725","units":10,"price":14}]Checked answer{"total": 546}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":341}8 tokens · 0.225swrong_schema_or_value
{"total":342}8 tokens · 0.341swrong_schema_or_value
{"total":341}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"W217","priority":2},{"id":"T168","priority":1},{"id":"W989","priority":9},{"id":"S415","priority":6},{"id":"F529","priority":7}]Checked answer{"ids": ["W989", "F529"]}invalid_json_object
[{"id":"W217","priority":2},{"id":"T168","priority":1},{"id":"W989","priority":9},{"id":"S415","priority":6},{"id":"F529","priority":7}]62 tokens · 1.692sCorrect
{"ids":["W989","F529"]}16 tokens · 0.440sCorrect
{"ids":["W989","F529"]}16 tokens · 0.633sCorrect
{"ids":["W989","F529"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z498? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"R504","city":"Oslo","units":10},{"id":"E636","city":"Riga","units":13},{"id":"Z873","city":"Lima","units":13},{"id":"Z498","city":"Kyoto","units":8},{"id":"N279","city":"Bern","units":17}]Checked answer{"city": "Kyoto", "units": 8}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.417sCorrect
{"city":"Kyoto","units":8}12 tokens · 0.331sCorrect
{"city":"Kyoto","units":8}12 tokens · 0.480sCorrect
{"city":"Kyoto","units":8}12 tokens · 0.337sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"W906","status":"ready"},{"id":"A719","status":"ready"},{"id":"I561","status":"ready"},{"id":"B204","status":"closed"},{"id":"V610","status":"closed"}]Checked answer{"ids": ["A719", "I561", "W906"]}invalid_json_object
[{"id":"A719","status":"ready"},{"id":"I561","status":"ready"},{"id":"W906","status":"ready"},{"id":"V610","status":"closed"}]51 tokens · 1.396sCorrect
{"ids":["A719","I561","W906"]}21 tokens · 0.578sCorrect
{"ids":["A719","I561","W906"]}21 tokens · 0.830sCorrect
{"ids":["A719","I561","W906"]}21 tokens · 0.585sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"W347","units":19,"price":6},{"id":"Z828","units":3,"price":10},{"id":"I926","units":2,"price":16},{"id":"C603","units":3,"price":12},{"id":"I552","units":2,"price":18}]Checked answer{"total": 248}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":331}8 tokens · 0.223swrong_schema_or_value
{"total":332}8 tokens · 0.347swrong_schema_or_value
{"total":333}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"U540","priority":3},{"id":"I773","priority":6},{"id":"O299","priority":8},{"id":"H103","priority":7},{"id":"V453","priority":1}]Checked answer{"ids": ["O299", "H103"]}wrong_schema_or_value
{"ids": ["O299", "U540"]}17 tokens · 0.469swrong_schema_or_value
{"ids":["O299","I773"]}16 tokens · 0.443sCorrect
{"ids":["O299","H103"]}16 tokens · 0.471swrong_schema_or_value
{"ids":["O299","I773"]}16 tokens · 0.445sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=H177. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"M381","city":"Accra","units":4},{"id":"H177","city":"Accra","units":18},{"id":"U691","city":"Riga","units":18},{"id":"R308","city":"Perth","units":1},{"id":"E977","city":"Riga","units":7}]Checked answer{"city": "Accra", "units": 18}wrong_schema_or_value
{"city":"Accra","units":4}11 tokens · 0.306sCorrect
{"city":"Accra","units":18}12 tokens · 0.338sCorrect
{"city":"Accra","units":18}12 tokens · 0.338sCorrect
{"city":"Accra","units":18}12 tokens · 0.336sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"K597","status":"closed"},{"id":"J765","status":"ready"},{"id":"N160","status":"closed"},{"id":"E299","status":"ready"},{"id":"E512","status":"closed"}]Checked answer{"ids": ["E299", "J765"]}wrong_schema_or_value
{"ids": ["J765", "E299", "E512", "K597", "N160"]}35 tokens · 0.958sCorrect
{"ids":["E299","J765"]}16 tokens · 0.440sCorrect
{"ids":["E299","J765"]}16 tokens · 0.453sCorrect
{"ids":["E299","J765"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"A619","units":4,"price":8},{"id":"B858","units":6,"price":18},{"id":"A408","units":19,"price":13},{"id":"U852","units":13,"price":16},{"id":"M840","units":7,"price":15}]Checked answer{"total": 700}wrong_schema_or_value
{
"total": 40
}11 tokens · 0.307swrong_schema_or_value
{"total":341}8 tokens · 0.224swrong_schema_or_value
{"total":332}8 tokens · 0.230swrong_schema_or_value
{"total":431}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"S442","priority":3},{"id":"P982","priority":6},{"id":"R780","priority":1},{"id":"E296","priority":3},{"id":"R286","priority":4}]Checked answer{"ids": ["P982", "R286"]}invalid_json_object
[{"id":"S442","priority":3},{"id":"P982","priority":6},{"id":"R780","priority":1},{"id":"E296","priority":3},{"id":"R286","priority":4}]62 tokens · 1.697sCorrect
{"ids":["P982","R286"]}16 tokens · 0.448swrong_schema_or_value
{"ids":["P982","S442"]}16 tokens · 0.451sCorrect
{"ids":["P982","R286"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z213. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"W903","city":"Bern","units":12},{"id":"D753","city":"Lima","units":11},{"id":"F927","city":"Bern","units":8},{"id":"F655","city":"Perth","units":5},{"id":"Z213","city":"Suva","units":2}]Checked answer{"city": "Suva", "units": 2}Correct
{
"city": "Suva",
"units": 2
}18 tokens · 0.498sCorrect
{"city":"Suva","units":2}11 tokens · 0.306sCorrect
{"city":"Suva","units":2}11 tokens · 0.308sCorrect
{"city":"Suva","units":2}11 tokens · 0.309sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"Y872","status":"closed"},{"id":"H853","status":"closed"},{"id":"D836","status":"pending"},{"id":"T843","status":"ready"},{"id":"Z138","status":"closed"}]Checked answer{"ids": ["T843"]}wrong_schema_or_value
{"ids": ["D836", "T843", "Z138"]}23 tokens · 0.636sCorrect
{"ids":["T843"]}11 tokens · 0.305sCorrect
{"ids":["T843"]}11 tokens · 0.314sCorrect
{"ids":["T843"]}11 tokens · 0.308sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"J191","units":15,"price":12},{"id":"D848","units":5,"price":2},{"id":"O230","units":6,"price":5},{"id":"D354","units":16,"price":19},{"id":"Q709","units":11,"price":14}]Checked answer{"total": 678}wrong_schema_or_value
{"total": 120}9 tokens · 0.253swrong_schema_or_value
{"total":371}8 tokens · 0.225swrong_schema_or_value
{"total":342}8 tokens · 0.232swrong_schema_or_value
{"total":431}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"M612","priority":8},{"id":"I511","priority":2},{"id":"W399","priority":1},{"id":"S895","priority":5},{"id":"S742","priority":1}]Checked answer{"ids": ["M612", "S895"]}wrong_schema_or_value
{"ids": ["I511", "W399", "S895", "S742", "M612"]}35 tokens · 0.963sCorrect
{"ids":["M612","S895"]}16 tokens · 0.446sCorrect
{"ids":["M612","S895"]}16 tokens · 0.460sCorrect
{"ids":["M612","S895"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"N970","city":"Accra","units":13},{"id":"J610","city":"Kyoto","units":9},{"id":"E224","city":"Kyoto","units":4},{"id":"A249","city":"Accra","units":14},{"id":"K547","city":"Lima","units":9}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Accra","units":13}12 tokens · 0.333sCorrect
{"city":null,"units":null}10 tokens · 0.281sCorrect
{"city":null,"units":null}10 tokens · 0.288sCorrect
{"city":null,"units":null}10 tokens · 0.287sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"K668","status":"closed"},{"id":"G229","status":"ready"},{"id":"W427","status":"pending"},{"id":"N127","status":"ready"},{"id":"E656","status":"ready"}]Checked answer{"ids": ["E656", "G229", "N127"]}wrong_schema_or_value
{
"ids": ["G229", "N127", "E656", "K668"]
}32 tokens · 0.876sCorrect
{"ids":["E656","G229","N127"]}21 tokens · 0.585sCorrect
{"ids":["E656","G229","N127"]}21 tokens · 0.591sCorrect
{"ids":["E656","G229","N127"]}21 tokens · 0.590sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"O226","units":13,"price":18},{"id":"E697","units":6,"price":6},{"id":"B163","units":13,"price":16},{"id":"Y513","units":5,"price":6},{"id":"U101","units":14,"price":1}]Checked answer{"total": 522}wrong_schema_or_value
{
"total": 100
}12 tokens · 0.335swrong_schema_or_value
{"total":341}8 tokens · 0.225swrong_schema_or_value
{"total":342}8 tokens · 0.225swrong_schema_or_value
{"total":341}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"C894","priority":7},{"id":"R639","priority":4},{"id":"L693","priority":9},{"id":"X945","priority":2},{"id":"S346","priority":5}]Checked answer{"ids": ["L693", "C894"]}invalid_json_object
[{"id":"C894","priority":7},{"id":"R639","priority":4},{"id":"L693","priority":9},{"id":"X945","priority":2},{"id":"S346","priority":5}]62 tokens · 1.697sCorrect
{"ids":["L693","C894"]}16 tokens · 0.448sCorrect
{"ids":["L693","C894"]}16 tokens · 0.451sCorrect
{"ids":["L693","C894"]}16 tokens · 0.451sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to W391? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"T892","city":"Oslo","units":11},{"id":"T894","city":"Accra","units":0},{"id":"J355","city":"Suva","units":8},{"id":"O991","city":"Riga","units":8},{"id":"W391","city":"Kyoto","units":9}]Checked answer{"city": "Kyoto", "units": 9}wrong_schema_or_value
{
"city": null,
"units": 9
}16 tokens · 0.446sCorrect
{"city":"Kyoto","units":9}12 tokens · 0.333sCorrect
{"city":"Kyoto","units":9}12 tokens · 0.337sCorrect
{"city":"Kyoto","units":9}12 tokens · 0.340sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"X740","status":"ready"},{"id":"Z600","status":"ready"},{"id":"I226","status":"ready"},{"id":"G927","status":"pending"},{"id":"M607","status":"ready"}]Checked answer{"ids": ["I226", "M607", "X740", "Z600"]}invalid_json_object
[{"id":"X740","status":"ready"},{"id":"Z600","status":"ready"},{"id":"I226","status":"ready"},{"id":"G927","status":"pending"},{"id":"M607","status":"ready"}]63 tokens · 1.731sCorrect
{"ids":["I226","M607","X740","Z600"]}26 tokens · 0.723swrong_schema_or_value
{"ids":["I226","M607","X740"]}21 tokens · 0.587sCorrect
{"ids":["I226","M607","X740","Z600"]}26 tokens · 0.729sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"V860","units":7,"price":6},{"id":"O233","units":9,"price":16},{"id":"Q267","units":8,"price":19},{"id":"B717","units":17,"price":13},{"id":"B371","units":4,"price":2}]Checked answer{"total": 567}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":371}8 tokens · 0.228swrong_schema_or_value
{"total":432}8 tokens · 0.225swrong_schema_or_value
{"total":410}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"L672","priority":4},{"id":"I528","priority":1},{"id":"J434","priority":3},{"id":"T270","priority":2},{"id":"U382","priority":2}]Checked answer{"ids": ["L672", "J434"]}wrong_schema_or_value
{"ids": ["L672", "U382", "T270", "I528", "J434", "J434"]}41 tokens · 1.120sCorrect
{"ids":["L672","J434"]}16 tokens · 0.451sCorrect
{"ids":["L672","J434"]}16 tokens · 0.445sCorrect
{"ids":["L672","J434"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=G260. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"T996","city":"Lima","units":8},{"id":"J873","city":"Riga","units":16},{"id":"G260","city":"Perth","units":3},{"id":"G466","city":"Perth","units":7},{"id":"F183","city":"Oslo","units":17}]Checked answer{"city": "Perth", "units": 3}wrong_schema_or_value
{"city":"Lima","units":8}11 tokens · 0.303sCorrect
{"city":"Perth","units":3}11 tokens · 0.310sCorrect
{"city":"Perth","units":3}11 tokens · 0.306sCorrect
{"city":"Perth","units":3}11 tokens · 0.312sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"C623","status":"ready"},{"id":"K561","status":"closed"},{"id":"G720","status":"closed"},{"id":"F411","status":"closed"},{"id":"G530","status":"closed"}]Checked answer{"ids": ["C623"]}wrong_schema_or_value
{"ids": ["C623", "K561", "G720", "F411", "G530"]}35 tokens · 0.954sCorrect
{"ids":["C623"]}11 tokens · 0.309sCorrect
{"ids":["C623"]}11 tokens · 0.307sCorrect
{"ids":["C623"]}11 tokens · 0.312sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"F554","units":4,"price":13},{"id":"A549","units":10,"price":2},{"id":"Y424","units":18,"price":13},{"id":"U347","units":5,"price":17},{"id":"X634","units":10,"price":7}]Checked answer{"total": 461}wrong_schema_or_value
{
"total": 100
}12 tokens · 0.332swrong_schema_or_value
{"total":341}8 tokens · 0.228swrong_schema_or_value
{"total":342}8 tokens · 0.224swrong_schema_or_value
{"total":341}8 tokens · 0.230sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"C315","priority":3},{"id":"S748","priority":3},{"id":"R133","priority":4},{"id":"D365","priority":8},{"id":"P746","priority":4}]Checked answer{"ids": ["D365", "P746"]}invalid_json_object
[{"id":"C315"},{"id":"S748"},{"id":"R133"},{"id":"D365"},{"id":"P746"}]43 tokens · 1.174swrong_schema_or_value
{"ids":["D365","R133"]}16 tokens · 0.449swrong_schema_or_value
{"ids":["D365","R133"]}16 tokens · 0.443swrong_schema_or_value
{"ids":["D365","R133"]}16 tokens · 0.452sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record E991. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"B667","city":"Accra","units":16},{"id":"E991","city":"Bern","units":17},{"id":"G987","city":"Riga","units":8},{"id":"V244","city":"Lima","units":19},{"id":"E341","city":"Accra","units":0}]Checked answer{"city": "Bern", "units": 17}invalid_json_object
[{"city":"Accra","units":16}]13 tokens · 0.361sCorrect
{"city":"Bern","units":17}11 tokens · 0.308sCorrect
{"city":"Bern","units":17}11 tokens · 0.307sCorrect
{"city":"Bern","units":17}11 tokens · 0.313sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"D183","status":"pending"},{"id":"S596","status":"pending"},{"id":"K601","status":"pending"},{"id":"Q734","status":"ready"},{"id":"K843","status":"ready"}]Checked answer{"ids": ["K843", "Q734"]}wrong_schema_or_value
{"ids": ["D183", "K601", "K843", "Q734"]}29 tokens · 0.796sCorrect
{"ids":["K843","Q734"]}16 tokens · 0.446sCorrect
{"ids":["K843","Q734"]}16 tokens · 0.447swrong_schema_or_value
{"ids":["K601","K843","Q734"]}21 tokens · 0.591sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"X514","units":3,"price":9},{"id":"V435","units":14,"price":19},{"id":"G770","units":16,"price":13},{"id":"M687","units":8,"price":18},{"id":"M471","units":8,"price":9}]Checked answer{"total": 717}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":431}8 tokens · 0.225swrong_schema_or_value
{"total":332}8 tokens · 0.225swrong_schema_or_value
{"total":441}8 tokens · 0.230sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"S198","priority":5},{"id":"C993","priority":3},{"id":"E344","priority":4},{"id":"F735","priority":7},{"id":"V862","priority":4}]Checked answer{"ids": ["F735", "S198"]}wrong_schema_or_value
{"ids": ["C993", "E344", "S198", "V862"]}29 tokens · 0.798sCorrect
{"ids":["F735","S198"]}16 tokens · 0.449sCorrect
{"ids":["F735","S198"]}16 tokens · 0.444swrong_schema_or_value
{"ids":["F735","E344"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id H531 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"F841","city":"Lima","units":0},{"id":"O373","city":"Oslo","units":12},{"id":"D951","city":"Suva","units":8},{"id":"H531","city":"Kyoto","units":19},{"id":"O798","city":"Oslo","units":19}]Checked answer{"city": "Kyoto", "units": 19}wrong_schema_or_value
{"city":"Lima","units":0}11 tokens · 0.307sCorrect
{"city":"Kyoto","units":19}13 tokens · 0.362sCorrect
{"city":"Kyoto","units":19}13 tokens · 0.368sCorrect
{"city":"Kyoto","units":19}13 tokens · 0.366sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"U110","status":"closed"},{"id":"J154","status":"closed"},{"id":"G875","status":"ready"},{"id":"H389","status":"closed"},{"id":"A251","status":"closed"}]Checked answer{"ids": ["G875"]}wrong_schema_or_value
{
"ids": ["A251", "G875", "H389", "J154", "U110"]
}38 tokens · 1.050sCorrect
{"ids":["G875"]}11 tokens · 0.306sCorrect
{"ids":["G875"]}11 tokens · 0.313sCorrect
{"ids":["G875"]}11 tokens · 0.311sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"F650","units":5,"price":1},{"id":"A114","units":4,"price":9},{"id":"S490","units":13,"price":11},{"id":"O982","units":15,"price":12},{"id":"P978","units":6,"price":14}]Checked answer{"total": 448}wrong_schema_or_value
{"total": 100}9 tokens · 0.261swrong_schema_or_value
{"total":341}8 tokens · 0.227swrong_schema_or_value
{"total":342}8 tokens · 0.228swrong_schema_or_value
{"total":341}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"F329","priority":8},{"id":"C681","priority":3},{"id":"B293","priority":9},{"id":"C478","priority":5},{"id":"P172","priority":6}]Checked answer{"ids": ["B293", "F329"]}invalid_json_object
[{"id":"C478","priority":5},{"id":"P172","priority":6}]26 tokens · 0.723sCorrect
{"ids":["B293","F329"]}16 tokens · 0.447sCorrect
{"ids":["B293","F329"]}16 tokens · 0.453sCorrect
{"ids":["B293","F329"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"Y759","city":"Oslo","units":6},{"id":"X316","city":"Perth","units":14},{"id":"P655","city":"Suva","units":6},{"id":"K585","city":"Kyoto","units":19},{"id":"Q123","city":"Accra","units":11}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokens · 0.416sCorrect
{"city":null,"units":null}10 tokens · 0.284sCorrect
{"city":null,"units":null}10 tokens · 0.286sCorrect
{"city":null,"units":null}10 tokens · 0.284sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"V697","status":"pending"},{"id":"R249","status":"ready"},{"id":"S407","status":"closed"},{"id":"B343","status":"ready"},{"id":"U466","status":"closed"}]Checked answer{"ids": ["B343", "R249"]}invalid_json_object
[{"id":"B343","status":"ready"},{"id":"U466","status":"closed"},{"id":"R249","status":"ready"},{"id":"V697","status":"pending"},{"id":"S407","status":"closed"}]63 tokens · 1.715sCorrect
{"ids":["B343","R249"]}16 tokens · 0.443sCorrect
{"ids":["B343","R249"]}16 tokens · 0.447sCorrect
{"ids":["B343","R249"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"G876","units":10,"price":8},{"id":"G796","units":18,"price":8},{"id":"D935","units":3,"price":14},{"id":"Y826","units":16,"price":16},{"id":"D798","units":14,"price":1}]Checked answer{"total": 536}wrong_schema_or_value
{"total": 100}9 tokens · 0.250swrong_schema_or_value
{"total":431}8 tokens · 0.226swrong_schema_or_value
{"total":342}8 tokens · 0.225swrong_schema_or_value
{"total":431}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"X620","priority":2},{"id":"M609","priority":2},{"id":"D420","priority":9},{"id":"P508","priority":7},{"id":"H223","priority":9}]Checked answer{"ids": ["D420", "H223"]}wrong_schema_or_value
{"ids": ["X620", "M609", "D420", "P508", "H223"]}35 tokens · 0.957swrong_schema_or_value
{"ids":["H223","D420"]}16 tokens · 0.448swrong_schema_or_value
{"ids":["H223","D420"]}16 tokens · 0.449swrong_schema_or_value
{"ids":["H223","D420"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=J682. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"O768","city":"Kyoto","units":14},{"id":"Q485","city":"Riga","units":18},{"id":"J682","city":"Kyoto","units":10},{"id":"J920","city":"Kyoto","units":11},{"id":"C288","city":"Perth","units":19}]Checked answer{"city": "Kyoto", "units": 10}Correct
{"city":"Kyoto","units":10}13 tokens · 0.359sCorrect
{"city":"Kyoto","units":10}13 tokens · 0.364sCorrect
{"city":"Kyoto","units":10}13 tokens · 0.366sCorrect
{"city":"Kyoto","units":10}13 tokens · 0.365sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"V190","status":"pending"},{"id":"R298","status":"pending"},{"id":"Z107","status":"ready"},{"id":"S426","status":"pending"},{"id":"R496","status":"closed"}]Checked answer{"ids": ["Z107"]}wrong_schema_or_value
{"ids": ["Z107", "S426", "V190", "R298", "R496"]}35 tokens · 0.957sCorrect
{"ids":["Z107"]}11 tokens · 0.309sCorrect
{"ids":["Z107"]}11 tokens · 0.310sCorrect
{"ids":["Z107"]}11 tokens · 0.308sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"Q749","units":1,"price":3},{"id":"V106","units":14,"price":14},{"id":"G505","units":9,"price":11},{"id":"A278","units":3,"price":15},{"id":"P540","units":18,"price":7}]Checked answer{"total": 469}wrong_schema_or_value
{
"total": 56
}11 tokens · 0.305swrong_schema_or_value
{"total":331}8 tokens · 0.226swrong_schema_or_value
{"total":332}8 tokens · 0.225swrong_schema_or_value
{"total":331}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"M177","priority":7},{"id":"N720","priority":9},{"id":"M432","priority":7},{"id":"H859","priority":4},{"id":"T422","priority":1}]Checked answer{"ids": ["N720", "M177"]}invalid_json_object
[{"id":"M177","priority":7},{"id":"N720","priority":9},{"id":"M432","priority":7},{"id":"H859","priority":4},{"id":"T422","priority":1}]62 tokens · 1.685sCorrect
{"ids":["N720","M177"]}16 tokens · 0.446swrong_schema_or_value
{"ids":["N720","M432"]}16 tokens · 0.449sCorrect
{"ids":["N720","M177"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record N578. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"N578","city":"Riga","units":18},{"id":"B358","city":"Oslo","units":2},{"id":"N486","city":"Lima","units":9},{"id":"E602","city":"Bern","units":16},{"id":"F353","city":"Riga","units":0}]Checked answer{"city": "Riga", "units": 18}Correct
{
"city": "Riga",
"units": 18
}19 tokens · 0.519sCorrect
{"city":"Riga","units":18}12 tokens · 0.338sCorrect
{"city":"Riga","units":18}12 tokens · 0.337sCorrect
{"city":"Riga","units":18}12 tokens · 0.337sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"V819","status":"ready"},{"id":"Y982","status":"ready"},{"id":"N711","status":"pending"},{"id":"A888","status":"ready"},{"id":"Q232","status":"closed"}]Checked answer{"ids": ["A888", "V819", "Y982"]}wrong_schema_or_value
{"ids": ["A888", "V819", "Y982", "N711", "Q232"]}35 tokens · 0.952sCorrect
{"ids":["A888","V819","Y982"]}21 tokens · 0.585sCorrect
{"ids":["A888","V819","Y982"]}21 tokens · 0.591sCorrect
{"ids":["A888","V819","Y982"]}21 tokens · 0.585sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"R402","units":7,"price":1},{"id":"R270","units":7,"price":12},{"id":"R203","units":18,"price":15},{"id":"Y381","units":10,"price":16},{"id":"O813","units":16,"price":8}]Checked answer{"total": 649}wrong_schema_or_value
{"total": 100}9 tokens · 0.250swrong_schema_or_value
{"total":431}8 tokens · 0.225swrong_schema_or_value
{"total":342}8 tokens · 0.228swrong_schema_or_value
{"total":471}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"R942","priority":7},{"id":"R322","priority":9},{"id":"I351","priority":9},{"id":"E801","priority":4},{"id":"N417","priority":8}]Checked answer{"ids": ["I351", "R322"]}wrong_schema_or_value
{"ids": ["R322", "R942", "I351", "E801", "N417"]}35 tokens · 0.960sCorrect
{"ids":["I351","R322"]}16 tokens · 0.446swrong_schema_or_value
{"ids":["R322","N417"]}16 tokens · 0.453sCorrect
{"ids":["I351","R322"]}16 tokens · 0.444sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Y337 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"Y337","city":"Perth","units":8},{"id":"Q613","city":"Accra","units":1},{"id":"K475","city":"Perth","units":6},{"id":"I837","city":"Riga","units":4},{"id":"S810","city":"Accra","units":1}]Checked answer{"city": "Perth", "units": 8}Correct
{"city":"Perth","units":8}11 tokens · 0.312sCorrect
{"city":"Perth","units":8}11 tokens · 0.311sCorrect
{"city":"Perth","units":8}11 tokens · 0.313sCorrect
{"city":"Perth","units":8}11 tokens · 0.311sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"M929","status":"pending"},{"id":"G773","status":"closed"},{"id":"Q818","status":"pending"},{"id":"V120","status":"pending"},{"id":"L778","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{
"ids": ["M929", "G773", "Q818", "V120", "L778"]
}38 tokens · 1.043sCorrect
{"ids":[]}6 tokens · 0.172swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.315sCorrect
{"ids":[]}6 tokens · 0.173sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"B989","units":5,"price":12},{"id":"D774","units":0,"price":8},{"id":"Q197","units":12,"price":15},{"id":"W721","units":14,"price":19},{"id":"G535","units":17,"price":17}]Checked answer{"total": 795}wrong_schema_or_value
{"total": 120}9 tokens · 0.253swrong_schema_or_value
{"total":441}8 tokens · 0.227swrong_schema_or_value
{"total":442}8 tokens · 0.228swrong_schema_or_value
{"total":711}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"Y194","priority":1},{"id":"R553","priority":1},{"id":"H693","priority":8},{"id":"J171","priority":7},{"id":"M311","priority":8}]Checked answer{"ids": ["H693", "M311"]}invalid_json_object
[{"id":"Y194","priority":1},{"id":"R553","priority":1},{"id":"H693","priority":8},{"id":"J171","priority":7},{"id":"M311","priority":8}]62 tokens · 1.694swrong_schema_or_value
{"ids":["M311","H693"]}16 tokens · 0.449sCorrect
{"ids":["H693","M311"]}16 tokens · 0.453swrong_schema_or_value
{"ids":["M311","H693"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to G838? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"F960","city":"Riga","units":9},{"id":"D144","city":"Kyoto","units":0},{"id":"Q181","city":"Perth","units":0},{"id":"X937","city":"Riga","units":18},{"id":"G838","city":"Accra","units":4}]Checked answer{"city": "Accra", "units": 4}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.416sCorrect
{"city":"Accra","units":4}11 tokens · 0.312sCorrect
{"city":"Accra","units":4}11 tokens · 0.312sCorrect
{"city":"Accra","units":4}11 tokens · 0.311sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"Z252","status":"pending"},{"id":"N503","status":"ready"},{"id":"P890","status":"ready"},{"id":"U179","status":"ready"},{"id":"U218","status":"closed"}]Checked answer{"ids": ["N503", "P890", "U179"]}invalid_json_object
[{"id":"N503","status":"ready"},{"id":"P890","status":"ready"},{"id":"U179","status":"ready"},{"id":"U218","status":"closed"}]51 tokens · 1.400swrong_schema_or_value
{"ids":["P890","N503","U179"]}21 tokens · 0.588sCorrect
{"ids":["N503","P890","U179"]}21 tokens · 0.592swrong_schema_or_value
{"ids":["P890","N503","U179"]}21 tokens · 0.589sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"I517","units":3,"price":13},{"id":"I200","units":12,"price":11},{"id":"B294","units":6,"price":14},{"id":"O308","units":14,"price":3},{"id":"R482","units":19,"price":16}]Checked answer{"total": 601}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":431}8 tokens · 0.229swrong_schema_or_value
{"total":432}8 tokens · 0.228swrong_schema_or_value
{"total":441}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"A991","priority":6},{"id":"Q537","priority":9},{"id":"I377","priority":6},{"id":"J979","priority":1},{"id":"X738","priority":1}]Checked answer{"ids": ["Q537", "A991"]}wrong_schema_or_value
{"ids": ["A991", "Q537", "I377", "J979", "X738"]}35 tokens · 0.960swrong_schema_or_value
{"ids":["Q537","I377"]}16 tokens · 0.450swrong_schema_or_value
{"ids":["Q537","I377"]}16 tokens · 0.447swrong_schema_or_value
{"ids":["Q537","I377"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"P550","city":"Bern","units":10},{"id":"K284","city":"Kyoto","units":16},{"id":"L945","city":"Lima","units":12},{"id":"P678","city":"Bern","units":15},{"id":"L144","city":"Riga","units":17}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Bern","units":10}11 tokens · 0.307sCorrect
{"city":null,"units":null}10 tokens · 0.284sCorrect
{"city":null,"units":null}10 tokens · 0.282sCorrect
{"city":null,"units":null}10 tokens · 0.285sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Q221","status":"pending"},{"id":"F645","status":"ready"},{"id":"E463","status":"ready"},{"id":"P383","status":"pending"},{"id":"W967","status":"closed"}]Checked answer{"ids": ["E463", "F645"]}wrong_schema_or_value
{"ids": ["F645", "E463", "P383", "W967", "Q221"]}35 tokens · 0.956sCorrect
{"ids":["E463","F645"]}16 tokens · 0.454swrong_schema_or_value
{"ids":["F645","E463"]}16 tokens · 0.451sCorrect
{"ids":["E463","F645"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"N337","units":11,"price":2},{"id":"Q640","units":16,"price":10},{"id":"M913","units":4,"price":3},{"id":"P231","units":9,"price":3},{"id":"E428","units":2,"price":14}]Checked answer{"total": 249}wrong_schema_or_value
{
"total": 10
}11 tokens · 0.305swrong_schema_or_value
{"total":311}8 tokens · 0.229swrong_schema_or_value
{"total":332}8 tokens · 0.227swrong_schema_or_value
{"total":231}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"R147","priority":3},{"id":"Z835","priority":8},{"id":"D312","priority":6},{"id":"C400","priority":8},{"id":"E680","priority":8}]Checked answer{"ids": ["C400", "E680"]}invalid_json_object
[{"id":"R147"},{"id":"Z835"},{"id":"D312"},{"id":"C400"},{"id":"E680"}]43 tokens · 1.177swrong_schema_or_value
{"ids":["E680","C400"]}16 tokens · 0.451swrong_schema_or_value
{"ids":["E680","C400"]}16 tokens · 0.449swrong_schema_or_value
{"ids":["E680","C400"]}16 tokens · 0.451sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record H498. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"X347","city":"Lima","units":13},{"id":"H498","city":"Kyoto","units":10},{"id":"B681","city":"Oslo","units":13},{"id":"U595","city":"Suva","units":10},{"id":"U188","city":"Lima","units":2}]Checked answer{"city": "Kyoto", "units": 10}wrong_schema_or_value
{
"city": "Lima",
"units": 13
}19 tokens · 0.523sCorrect
{"city":"Kyoto","units":10}13 tokens · 0.368sCorrect
{"city":"Kyoto","units":10}13 tokens · 0.367sCorrect
{"city":"Kyoto","units":10}13 tokens · 0.368sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"L476","status":"closed"},{"id":"S638","status":"closed"},{"id":"E125","status":"ready"},{"id":"H131","status":"ready"},{"id":"G659","status":"ready"}]Checked answer{"ids": ["E125", "G659", "H131"]}wrong_schema_or_value
{"ids": ["E125", "G659", "H131", "L476", "S638"]}35 tokens · 0.959swrong_schema_or_value
{"ids":["G659","H131","E125"]}21 tokens · 0.590swrong_schema_or_value
{"ids":["E125","H131","G659"]}21 tokens · 0.590swrong_schema_or_value
{"ids":["G659","E125","H131"]}21 tokens · 0.590sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"D785","units":17,"price":9},{"id":"K902","units":4,"price":14},{"id":"Y706","units":17,"price":5},{"id":"F297","units":5,"price":5},{"id":"J197","units":2,"price":7}]Checked answer{"total": 333}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":371}8 tokens · 0.229swrong_schema_or_value
{"total":332}8 tokens · 0.225swrong_schema_or_value
{"total":371}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"K401","priority":7},{"id":"I934","priority":1},{"id":"H149","priority":4},{"id":"V754","priority":4},{"id":"W832","priority":5}]Checked answer{"ids": ["K401", "W832"]}wrong_schema_or_value
{"ids": ["I934", "H149", "V754", "W832", "K401"]}35 tokens · 0.962sCorrect
{"ids":["K401","W832"]}16 tokens · 0.449sCorrect
{"ids":["K401","W832"]}16 tokens · 0.447sCorrect
{"ids":["K401","W832"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id O816 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"O816","city":"Suva","units":8},{"id":"A580","city":"Riga","units":10},{"id":"Y937","city":"Accra","units":18},{"id":"D668","city":"Oslo","units":17},{"id":"O706","city":"Perth","units":8}]Checked answer{"city": "Suva", "units": 8}Correct
{"city":"Suva","units":8}11 tokens · 0.306sCorrect
{"city":"Suva","units":8}11 tokens · 0.310sCorrect
{"city":"Suva","units":8}11 tokens · 0.310sCorrect
{"city":"Suva","units":8}11 tokens · 0.312sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"F769","status":"pending"},{"id":"B648","status":"closed"},{"id":"A871","status":"ready"},{"id":"W845","status":"ready"},{"id":"G188","status":"pending"}]Checked answer{"ids": ["A871", "W845"]}wrong_schema_or_value
{
"ids": ["A871", "B648", "G188", "F769", "W845"]
}38 tokens · 1.039sCorrect
{"ids":["A871","W845"]}16 tokens · 0.448sCorrect
{"ids":["A871","W845"]}16 tokens · 0.447sCorrect
{"ids":["A871","W845"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"S550","units":7,"price":19},{"id":"Z102","units":1,"price":6},{"id":"B913","units":4,"price":11},{"id":"A445","units":0,"price":7},{"id":"P695","units":14,"price":17}]Checked answer{"total": 421}wrong_schema_or_value
{"total": 100}9 tokens · 0.250swrong_schema_or_value
{"total":341}8 tokens · 0.228swrong_schema_or_value
{"total":342}8 tokens · 0.226swrong_schema_or_value
{"total":341}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"F658","priority":2},{"id":"V837","priority":7},{"id":"H912","priority":4},{"id":"O716","priority":3},{"id":"K763","priority":7}]Checked answer{"ids": ["K763", "V837"]}invalid_json_object
[{"id":"F658","priority":2},{"id":"V837","priority":7},{"id":"H912","priority":4},{"id":"O716","priority":3},{"id":"K763","priority":7}]62 tokens · 1.689sCorrect
{"ids":["K763","V837"]}16 tokens · 0.449sCorrect
{"ids":["K763","V837"]}16 tokens · 0.451sCorrect
{"ids":["K763","V837"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z273? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"D371","city":"Riga","units":5},{"id":"U563","city":"Oslo","units":9},{"id":"U198","city":"Riga","units":11},{"id":"N654","city":"Accra","units":16},{"id":"Z273","city":"Kyoto","units":17}]Checked answer{"city": "Kyoto", "units": 17}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.415sCorrect
{"city":"Kyoto","units":17}13 tokens · 0.365sCorrect
{"city":"Kyoto","units":17}13 tokens · 0.366sCorrect
{"city":"Kyoto","units":17}13 tokens · 0.363sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"H813","status":"closed"},{"id":"M685","status":"closed"},{"id":"G343","status":"pending"},{"id":"U264","status":"pending"},{"id":"Z979","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"H813"},{"id":"M685"},{"id":"G343"},{"id":"U264"},{"id":"Z979"}]43 tokens · 1.173sCorrect
{"ids":[]}6 tokens · 0.170swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.309sCorrect
{"ids":[]}6 tokens · 0.170sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"C832","units":3,"price":4},{"id":"Z446","units":17,"price":19},{"id":"I625","units":13,"price":8},{"id":"U310","units":3,"price":18},{"id":"D267","units":8,"price":2}]Checked answer{"total": 509}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":341}8 tokens · 0.228swrong_schema_or_value
{"total":342}8 tokens · 0.230swrong_schema_or_value
{"total":333}8 tokens · 0.223sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"C895","priority":1},{"id":"X136","priority":8},{"id":"U502","priority":8},{"id":"C788","priority":2},{"id":"M546","priority":1}]Checked answer{"ids": ["U502", "X136"]}wrong_schema_or_value
{"ids": ["C895", "X136", "U502", "C788", "M546"]}35 tokens · 0.960sCorrect
{"ids":["U502","X136"]}16 tokens · 0.451swrong_schema_or_value
{"ids":["U502","C895"]}16 tokens · 0.450sCorrect
{"ids":["U502","X136"]}16 tokens · 0.456sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=V469. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"H602","city":"Oslo","units":9},{"id":"N412","city":"Perth","units":4},{"id":"V469","city":"Oslo","units":11},{"id":"X898","city":"Accra","units":12},{"id":"E510","city":"Accra","units":3}]Checked answer{"city": "Oslo", "units": 11}Correct
{"city":"Oslo","units":11}12 tokens · 0.335sCorrect
{"city":"Oslo","units":11}12 tokens · 0.342sCorrect
{"city":"Oslo","units":11}12 tokens · 0.341sCorrect
{"city":"Oslo","units":11}12 tokens · 0.347sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Z718","status":"ready"},{"id":"W426","status":"closed"},{"id":"F515","status":"closed"},{"id":"Q836","status":"ready"},{"id":"W840","status":"closed"}]Checked answer{"ids": ["Q836", "Z718"]}wrong_schema_or_value
{"ids": ["Z718", "W426", "F515", "Q836", "W840"]}35 tokens · 0.962sCorrect
{"ids":["Q836","Z718"]}16 tokens · 0.450sCorrect
{"ids":["Q836","Z718"]}16 tokens · 0.449sCorrect
{"ids":["Q836","Z718"]}16 tokens · 0.443sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"C209","units":5,"price":4},{"id":"O687","units":10,"price":2},{"id":"V388","units":7,"price":13},{"id":"K923","units":14,"price":13},{"id":"V603","units":12,"price":11}]Checked answer{"total": 445}wrong_schema_or_value
{
"total": 100
}12 tokens · 0.333swrong_schema_or_value
{"total":311}8 tokens · 0.229swrong_schema_or_value
{"total":342}8 tokens · 0.228swrong_schema_or_value
{"total":341}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"B221","priority":7},{"id":"X581","priority":4},{"id":"P914","priority":2},{"id":"J474","priority":7},{"id":"H316","priority":4}]Checked answer{"ids": ["B221", "J474"]}invalid_json_object
[{"id":"B221","priority":7},{"id":"P914","priority":2}]26 tokens · 0.712sCorrect
{"ids":["B221","J474"]}16 tokens · 0.450sCorrect
{"ids":["B221","J474"]}16 tokens · 0.445sCorrect
{"ids":["B221","J474"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"G461","city":"Bern","units":5},{"id":"E606","city":"Riga","units":8},{"id":"M733","city":"Oslo","units":18},{"id":"N216","city":"Bern","units":12},{"id":"N326","city":"Suva","units":12}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Bern","units":5},{"city":"Riga","units":8},{"city":"Oslo","units":18},{"city":"Bern","units":12},{"city":"Suva","units":12}]53 tokens · 1.455sCorrect
{"city":null,"units":null}10 tokens · 0.284sCorrect
{"city":null,"units":null}10 tokens · 0.282sCorrect
{"city":null,"units":null}10 tokens · 0.281sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"H832","status":"closed"},{"id":"N179","status":"pending"},{"id":"O981","status":"ready"},{"id":"O537","status":"ready"},{"id":"P783","status":"pending"}]Checked answer{"ids": ["O537", "O981"]}wrong_schema_or_value
{"ids": ["O537", "O981", "P783"]}23 tokens · 0.632sCorrect
{"ids":["O537","O981"]}16 tokens · 0.448sCorrect
{"ids":["O537","O981"]}16 tokens · 0.448sCorrect
{"ids":["O537","O981"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"U790","units":2,"price":17},{"id":"A196","units":19,"price":5},{"id":"Y162","units":13,"price":2},{"id":"T483","units":14,"price":17},{"id":"B708","units":4,"price":12}]Checked answer{"total": 441}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":371}8 tokens · 0.227swrong_schema_or_value
{"total":342}8 tokens · 0.226swrong_schema_or_value
{"total":371}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"R716","priority":7},{"id":"L769","priority":7},{"id":"J227","priority":9},{"id":"Q873","priority":1},{"id":"A348","priority":8}]Checked answer{"ids": ["J227", "A348"]}wrong_schema_or_value
{"ids": ["R716", "L769", "J227", "Q873", "A348"]}35 tokens · 0.960swrong_schema_or_value
{"ids":["J227","R716"]}16 tokens · 0.444sCorrect
{"ids":["J227","A348"]}16 tokens · 0.452swrong_schema_or_value
{"ids":["J227","R716"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id A813 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"A813","city":"Lima","units":8},{"id":"T753","city":"Perth","units":15},{"id":"K809","city":"Bern","units":19},{"id":"P758","city":"Riga","units":9},{"id":"N678","city":"Oslo","units":2}]Checked answer{"city": "Lima", "units": 8}Correct
{"city":"Lima","units":8}11 tokens · 0.306sCorrect
{"city":"Lima","units":8}11 tokens · 0.309sCorrect
{"city":"Lima","units":8}11 tokens · 0.311sCorrect
{"city":"Lima","units":8}11 tokens · 0.308sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"A141","status":"closed"},{"id":"B778","status":"closed"},{"id":"B122","status":"ready"},{"id":"Q592","status":"ready"},{"id":"G612","status":"pending"}]Checked answer{"ids": ["B122", "Q592"]}wrong_schema_or_value
{
"ids": ["A141", "B122", "B778", "G612"]
}32 tokens · 0.876sCorrect
{"ids":["B122","Q592"]}16 tokens · 0.445sCorrect
{"ids":["B122","Q592"]}16 tokens · 0.450sCorrect
{"ids":["B122","Q592"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"V582","units":19,"price":11},{"id":"K460","units":19,"price":9},{"id":"J105","units":17,"price":6},{"id":"M704","units":4,"price":4},{"id":"S865","units":13,"price":9}]Checked answer{"total": 615}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":441}8 tokens · 0.226swrong_schema_or_value
{"total":432}8 tokens · 0.229swrong_schema_or_value
{"total":731}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"M543","priority":7},{"id":"X773","priority":2},{"id":"L424","priority":7},{"id":"P121","priority":7},{"id":"Q414","priority":7}]Checked answer{"ids": ["L424", "M543"]}invalid_json_object
[{"id":"M543","priority":7},{"id":"L424","priority":7},{"id":"P121","priority":7},{"id":"Q414","priority":7}]50 tokens · 1.367swrong_schema_or_value
{"ids":["P121","M543","Q414"]}21 tokens · 0.585swrong_schema_or_value
{"ids":["M543","Q414"]}16 tokens · 0.444swrong_schema_or_value
{"ids":["P121","M543","Q414"]}21 tokens · 0.581sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to M793? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"A622","city":"Bern","units":2},{"id":"M793","city":"Kyoto","units":8},{"id":"K226","city":"Bern","units":13},{"id":"J736","city":"Oslo","units":8},{"id":"V652","city":"Accra","units":15}]Checked answer{"city": "Kyoto", "units": 8}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.415sCorrect
{"city":"Kyoto","units":8}12 tokens · 0.339sCorrect
{"city":"Kyoto","units":8}12 tokens · 0.337sCorrect
{"city":"Kyoto","units":8}12 tokens · 0.336sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"H523","status":"ready"},{"id":"A488","status":"ready"},{"id":"I235","status":"ready"},{"id":"L312","status":"ready"},{"id":"X625","status":"closed"}]Checked answer{"ids": ["A488", "H523", "I235", "L312"]}invalid_json_object
[{"id":"A488","status":"ready"},{"id":"H523","status":"ready"},{"id":"I235","status":"ready"},{"id":"L312","status":"ready"},{"id":"X625","status":"closed"}]63 tokens · 1.717swrong_schema_or_value
{"ids":["A488","I235","H523","L312"]}26 tokens · 0.725sCorrect
{"ids":["A488","H523","I235","L312"]}26 tokens · 0.730swrong_schema_or_value
{"ids":["A488","I235","H523","L312"]}26 tokens · 0.724sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Y640","units":6,"price":5},{"id":"K843","units":3,"price":14},{"id":"L845","units":13,"price":14},{"id":"Z980","units":7,"price":14},{"id":"M431","units":3,"price":9}]Checked answer{"total": 379}wrong_schema_or_value
{"total": 100}9 tokens · 0.253swrong_schema_or_value
{"total":311}8 tokens · 0.226swrong_schema_or_value
{"total":232}8 tokens · 0.228swrong_schema_or_value
{"total":333}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"L615","priority":6},{"id":"X685","priority":7},{"id":"F142","priority":7},{"id":"I751","priority":1},{"id":"R564","priority":4}]Checked answer{"ids": ["F142", "X685"]}wrong_schema_or_value
{"ids": ["L615", "X685", "F142", "I751", "R564"]}35 tokens · 0.960sCorrect
{"ids":["F142","X685"]}16 tokens · 0.447sCorrect
{"ids":["F142","X685"]}16 tokens · 0.450sCorrect
{"ids":["F142","X685"]}16 tokens · 0.439sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=P806. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"E114","city":"Perth","units":13},{"id":"P806","city":"Lima","units":14},{"id":"S991","city":"Kyoto","units":5},{"id":"I430","city":"Riga","units":11},{"id":"Y836","city":"Suva","units":0}]Checked answer{"city": "Lima", "units": 14}Correct
{"city":"Lima","units":14}12 tokens · 0.332sCorrect
{"city":"Lima","units":14}12 tokens · 0.339sCorrect
{"city":"Lima","units":14}12 tokens · 0.339sCorrect
{"city":"Lima","units":14}12 tokens · 0.329sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"I796","status":"ready"},{"id":"H706","status":"pending"},{"id":"X747","status":"pending"},{"id":"I420","status":"closed"},{"id":"H694","status":"closed"}]Checked answer{"ids": ["I796"]}wrong_schema_or_value
{"ids": ["I796", "H706", "X747", "I420", "H694"]}35 tokens · 0.961sCorrect
{"ids":["I796"]}11 tokens · 0.310sCorrect
{"ids":["I796"]}11 tokens · 0.312sCorrect
{"ids":["I796"]}11 tokens · 0.303sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"N884","units":9,"price":12},{"id":"P317","units":15,"price":6},{"id":"A505","units":7,"price":3},{"id":"F465","units":10,"price":7},{"id":"H976","units":12,"price":12}]Checked answer{"total": 433}invalid_json_object
{
"total": 12 + 6 + 3 + 7 + 12 = 40
}28 tokens · 0.773swrong_schema_or_value
{"total":311}8 tokens · 0.227swrong_schema_or_value
{"total":342}8 tokens · 0.227swrong_schema_or_value
{"total":311}8 tokens · 0.223sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"T141","priority":9},{"id":"N308","priority":5},{"id":"Z282","priority":2},{"id":"Q573","priority":7},{"id":"W649","priority":9}]Checked answer{"ids": ["T141", "W649"]}invalid_json_object
[{"id":"T141","priority":9},{"id":"W649","priority":9}]26 tokens · 0.716sCorrect
{"ids":["T141","W649"]}16 tokens · 0.449sCorrect
{"ids":["T141","W649"]}16 tokens · 0.444sCorrect
{"ids":["T141","W649"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Y925. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"W710","city":"Bern","units":17},{"id":"V111","city":"Kyoto","units":7},{"id":"Z162","city":"Kyoto","units":9},{"id":"Y925","city":"Kyoto","units":15},{"id":"K799","city":"Bern","units":19}]Checked answer{"city": "Kyoto", "units": 15}Correct
{
"city": "Kyoto",
"units": 15
}20 tokens · 0.553sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.368sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.363sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.362sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"G267","status":"closed"},{"id":"M889","status":"closed"},{"id":"P892","status":"ready"},{"id":"T277","status":"closed"},{"id":"D467","status":"closed"}]Checked answer{"ids": ["P892"]}wrong_schema_or_value
{"ids": ["P892", "T277", "D467", "G267", "M889"]}35 tokens · 0.962sCorrect
{"ids":["P892"]}11 tokens · 0.312sCorrect
{"ids":["P892"]}11 tokens · 0.309sCorrect
{"ids":["P892"]}11 tokens · 0.308sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"C159","units":7,"price":18},{"id":"C601","units":16,"price":13},{"id":"W752","units":9,"price":18},{"id":"I959","units":4,"price":4},{"id":"G723","units":2,"price":19}]Checked answer{"total": 550}wrong_schema_or_value
{"total": 120}9 tokens · 0.251swrong_schema_or_value
{"total":341}8 tokens · 0.228swrong_schema_or_value
{"total":342}8 tokens · 0.224swrong_schema_or_value
{"total":341}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"E626","priority":5},{"id":"B601","priority":8},{"id":"E494","priority":9},{"id":"F296","priority":2},{"id":"T368","priority":8}]Checked answer{"ids": ["E494", "B601"]}wrong_schema_or_value
{"ids": ["E626", "B601", "E494", "F296", "T368"]}35 tokens · 0.958sCorrect
{"ids":["E494","B601"]}16 tokens · 0.447sCorrect
{"ids":["E494","B601"]}16 tokens · 0.444sCorrect
{"ids":["E494","B601"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"D482","city":"Accra","units":6},{"id":"W234","city":"Lima","units":18},{"id":"Y911","city":"Riga","units":10},{"id":"Z586","city":"Kyoto","units":17},{"id":"W375","city":"Lima","units":14}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Accra","units":6}11 tokens · 0.306swrong_schema_or_value
{"city":"Kyoto","units":17}13 tokens · 0.364swrong_schema_or_value
{"city":"Kyoto","units":17}13 tokens · 0.365sCorrect
{"city":null,"units":null}10 tokens · 0.282sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"R757","status":"pending"},{"id":"S792","status":"pending"},{"id":"A260","status":"closed"},{"id":"F451","status":"pending"},{"id":"I366","status":"ready"}]Checked answer{"ids": ["I366"]}wrong_schema_or_value
{
"ids": ["A260", "F451", "I366"]
}26 tokens · 0.716sCorrect
{"ids":["I366"]}11 tokens · 0.310sCorrect
{"ids":["I366"]}11 tokens · 0.307sCorrect
{"ids":["I366"]}11 tokens · 0.307sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"Z626","units":12,"price":6},{"id":"C153","units":10,"price":5},{"id":"L912","units":5,"price":10},{"id":"G980","units":3,"price":12},{"id":"L268","units":2,"price":17}]Checked answer{"total": 242}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":341}8 tokens · 0.227swrong_schema_or_value
{"total":342}8 tokens · 0.225swrong_schema_or_value
{"total":311}8 tokens · 0.225sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"V414","priority":6},{"id":"E294","priority":6},{"id":"N965","priority":3},{"id":"Q893","priority":2},{"id":"V330","priority":1}]Checked answer{"ids": ["E294", "V414"]}invalid_json_object
[{"id":"V414","priority":6},{"id":"E294","priority":6},{"id":"N965","priority":3},{"id":"Q893","priority":2},{"id":"V330","priority":1}]62 tokens · 1.698sCorrect
{"ids":["E294","V414"]}16 tokens · 0.448sCorrect
{"ids":["E294","V414"]}16 tokens · 0.445sCorrect
{"ids":["E294","V414"]}16 tokens · 0.444sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to F933? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"L870","city":"Perth","units":5},{"id":"T792","city":"Suva","units":3},{"id":"F933","city":"Riga","units":9},{"id":"D151","city":"Riga","units":10},{"id":"I197","city":"Oslo","units":11}]Checked answer{"city": "Riga", "units": 9}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.415sCorrect
{"city":"Riga","units":9}11 tokens · 0.310sCorrect
{"city":"Riga","units":9}11 tokens · 0.310sCorrect
{"city":"Riga","units":9}11 tokens · 0.309sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"A825","status":"pending"},{"id":"H420","status":"closed"},{"id":"T814","status":"pending"},{"id":"X170","status":"closed"},{"id":"S462","status":"closed"}]Checked answer{"ids": []}invalid_json_object
[{"id":"A825"},{"id":"H420"},{"id":"T814"},{"id":"X170"},{"id":"S462"}]43 tokens · 1.173sCorrect
{"ids":[]}6 tokens · 0.172swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.307sCorrect
{"ids":[]}6 tokens · 0.179sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"I817","units":19,"price":2},{"id":"Y912","units":2,"price":15},{"id":"O886","units":9,"price":16},{"id":"Q380","units":18,"price":12},{"id":"R125","units":5,"price":13}]Checked answer{"total": 493}wrong_schema_or_value
{"total": 100}9 tokens · 0.250swrong_schema_or_value
{"total":431}8 tokens · 0.228swrong_schema_or_value
{"total":432}8 tokens · 0.225swrong_schema_or_value
{"total":471}8 tokens · 0.252sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"S176","priority":9},{"id":"Y906","priority":6},{"id":"I397","priority":1},{"id":"Y583","priority":7},{"id":"U219","priority":8}]Checked answer{"ids": ["S176", "U219"]}wrong_schema_or_value
{"ids": ["Y583", "Y906", "I397", "S176", "U219"]}35 tokens · 0.956sCorrect
{"ids":["S176","U219"]}16 tokens · 0.448sCorrect
{"ids":["S176","U219"]}16 tokens · 0.446sCorrect
{"ids":["S176","U219"]}16 tokens · 0.464sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=C828. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"T209","city":"Suva","units":15},{"id":"C828","city":"Riga","units":0},{"id":"I187","city":"Suva","units":10},{"id":"A627","city":"Lima","units":2},{"id":"U267","city":"Suva","units":4}]Checked answer{"city": "Riga", "units": 0}wrong_schema_or_value
{"city": "Suva", "units": 15}15 tokens · 0.413sCorrect
{"city":"Riga","units":0}11 tokens · 0.311swrong_schema_or_value
{"city":null,"units":null}10 tokens · 0.279sCorrect
{"city":"Riga","units":0}11 tokens · 0.311sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Z212","status":"pending"},{"id":"P469","status":"pending"},{"id":"E459","status":"pending"},{"id":"D991","status":"closed"},{"id":"B265","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["B265", "D991", "E459", "P469", "Z212"]}35 tokens · 0.953sCorrect
{"ids":[]}6 tokens · 0.173swrong_schema_or_value
{"ids":["D991"]}11 tokens · 0.310sCorrect
{"ids":[]}6 tokens · 0.173sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"L534","units":3,"price":14},{"id":"K768","units":1,"price":14},{"id":"T471","units":14,"price":6},{"id":"M129","units":10,"price":7},{"id":"D197","units":6,"price":9}]Checked answer{"total": 264}wrong_schema_or_value
{
"total": 100
}12 tokens · 0.332swrong_schema_or_value
{"total":311}8 tokens · 0.228swrong_schema_or_value
{"total":232}8 tokens · 0.225swrong_schema_or_value
{"total":241}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"D847","priority":7},{"id":"Y836","priority":4},{"id":"L699","priority":8},{"id":"P652","priority":8},{"id":"K182","priority":7}]Checked answer{"ids": ["L699", "P652"]}invalid_json_object
[{"id":"D847","priority":7},{"id":"Y836","priority":4},{"id":"L699","priority":8},{"id":"P652","priority":8},{"id":"K182","priority":7}]62 tokens · 1.689sCorrect
{"ids":["L699","P652"]}16 tokens · 0.448swrong_schema_or_value
{"ids":["D847","L699"]}16 tokens · 0.443swrong_schema_or_value
{"ids":["P652","L699"]}16 tokens · 0.445sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record V488. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Z796","city":"Riga","units":6},{"id":"X289","city":"Kyoto","units":2},{"id":"T687","city":"Riga","units":2},{"id":"B320","city":"Suva","units":16},{"id":"V488","city":"Kyoto","units":15}]Checked answer{"city": "Kyoto", "units": 15}wrong_schema_or_value
{
"city": "Riga",
"units": 6
}18 tokens · 0.498sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.366sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.365sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.362sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"X422","status":"closed"},{"id":"W182","status":"pending"},{"id":"J757","status":"closed"},{"id":"I108","status":"closed"},{"id":"G265","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["I108", "G265", "J757", "W182", "X422"]}35 tokens · 0.962sCorrect
{"ids":[]}6 tokens · 0.172swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.306sCorrect
{"ids":[]}6 tokens · 0.172sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"G994","units":13,"price":14},{"id":"X676","units":17,"price":19},{"id":"S331","units":9,"price":3},{"id":"O614","units":15,"price":4},{"id":"G228","units":12,"price":9}]Checked answer{"total": 700}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":431}8 tokens · 0.229swrong_schema_or_value
{"total":332}8 tokens · 0.224swrong_schema_or_value
{"total":471}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Z831","priority":9},{"id":"X484","priority":4},{"id":"T414","priority":7},{"id":"P115","priority":9},{"id":"Z183","priority":9}]Checked answer{"ids": ["P115", "Z183"]}wrong_schema_or_value
{"ids": ["X484", "Z831", "T414", "P115", "Z183"]}35 tokens · 0.957swrong_schema_or_value
{"ids":["Z183","P115"]}16 tokens · 0.450swrong_schema_or_value
{"ids":["Z831","Z183"]}16 tokens · 0.443sCorrect
{"ids":["P115","Z183"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id E898 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"M638","city":"Oslo","units":1},{"id":"E742","city":"Suva","units":6},{"id":"M800","city":"Riga","units":11},{"id":"R542","city":"Suva","units":18},{"id":"E898","city":"Kyoto","units":15}]Checked answer{"city": "Kyoto", "units": 15}wrong_schema_or_value
{"city":"Oslo","units":1}11 tokens · 0.305sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.361sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.362sCorrect
{"city":"Kyoto","units":15}13 tokens · 0.366sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"T474","status":"closed"},{"id":"U534","status":"closed"},{"id":"I284","status":"ready"},{"id":"H883","status":"ready"},{"id":"D153","status":"closed"}]Checked answer{"ids": ["H883", "I284"]}wrong_schema_or_value
{
"ids": ["I284", "H883", "U534", "T474", "D153"]
}38 tokens · 1.039sCorrect
{"ids":["H883","I284"]}16 tokens · 0.447sCorrect
{"ids":["H883","I284"]}16 tokens · 0.442sCorrect
{"ids":["H883","I284"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"P425","units":9,"price":19},{"id":"B231","units":14,"price":6},{"id":"X156","units":15,"price":3},{"id":"P257","units":4,"price":17},{"id":"Y418","units":15,"price":13}]Checked answer{"total": 563}wrong_schema_or_value
{"total": 103}9 tokens · 0.252swrong_schema_or_value
{"total":341}8 tokens · 0.224swrong_schema_or_value
{"total":332}8 tokens · 0.225swrong_schema_or_value
{"total":431}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"O691","priority":6},{"id":"S185","priority":1},{"id":"W678","priority":5},{"id":"T329","priority":2},{"id":"M123","priority":3}]Checked answer{"ids": ["O691", "W678"]}invalid_json_object
[{"id":"O691","priority":6},{"id":"S185","priority":1},{"id":"W678","priority":5},{"id":"T329","priority":2},{"id":"M123","priority":3}]62 tokens · 1.696sCorrect
{"ids":["O691","W678"]}16 tokens · 0.447sCorrect
{"ids":["O691","W678"]}16 tokens · 0.450sCorrect
{"ids":["O691","W678"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"G237","city":"Perth","units":13},{"id":"H795","city":"Riga","units":5},{"id":"N566","city":"Lima","units":13},{"id":"E450","city":"Bern","units":5},{"id":"C889","city":"Accra","units":5}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokens · 0.415sCorrect
{"city":null,"units":null}10 tokens · 0.283sCorrect
{"city":null,"units":null}10 tokens · 0.280sCorrect
{"city":null,"units":null}10 tokens · 0.283sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"V930","status":"ready"},{"id":"V671","status":"ready"},{"id":"K382","status":"closed"},{"id":"M290","status":"ready"},{"id":"F693","status":"pending"}]Checked answer{"ids": ["M290", "V671", "V930"]}invalid_json_object
[{"id":"V930","status":"ready"},{"id":"V671","status":"ready"},{"id":"K382","status":"closed"},{"id":"M290","status":"ready"},{"id":"F693","status":"pending"}]63 tokens · 1.726sCorrect
{"ids":["M290","V671","V930"]}21 tokens · 0.586sCorrect
{"ids":["M290","V671","V930"]}21 tokens · 0.588swrong_schema_or_value
{"ids":["M290","V930","V671"]}21 tokens · 0.587sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"O513","units":15,"price":4},{"id":"U258","units":9,"price":7},{"id":"W746","units":19,"price":16},{"id":"E323","units":17,"price":19},{"id":"M209","units":11,"price":3}]Checked answer{"total": 783}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":471}8 tokens · 0.223swrong_schema_or_value
{"total":432}8 tokens · 0.225swrong_schema_or_value
{"total":531}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"I608","priority":3},{"id":"D657","priority":7},{"id":"V322","priority":7},{"id":"W169","priority":7},{"id":"I211","priority":2}]Checked answer{"ids": ["D657", "V322"]}wrong_schema_or_value
{"ids": ["I608", "V322", "I211", "D657", "W169", "I608"]}41 tokens · 1.119sCorrect
{"ids":["D657","V322"]}16 tokens · 0.446swrong_schema_or_value
{"ids":["D657","W169"]}16 tokens · 0.443sCorrect
{"ids":["D657","V322"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=V991. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"A564","city":"Kyoto","units":5},{"id":"U112","city":"Riga","units":13},{"id":"D436","city":"Bern","units":19},{"id":"V991","city":"Lima","units":2},{"id":"I635","city":"Kyoto","units":3}]Checked answer{"city": "Lima", "units": 2}wrong_schema_or_value
{"city":"Kyoto","units":5}12 tokens · 0.335sCorrect
{"city":"Lima","units":2}11 tokens · 0.310sCorrect
{"city":"Lima","units":2}11 tokens · 0.306sCorrect
{"city":"Lima","units":2}11 tokens · 0.310sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"Y680","status":"closed"},{"id":"N522","status":"pending"},{"id":"M628","status":"closed"},{"id":"Y130","status":"pending"},{"id":"I473","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["M628", "Y680", "Y130", "N522", "I473"]}35 tokens · 0.961sCorrect
{"ids":[]}6 tokens · 0.170swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.307sCorrect
{"ids":[]}6 tokens · 0.172sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"D498","units":19,"price":7},{"id":"O779","units":11,"price":17},{"id":"W260","units":18,"price":4},{"id":"O901","units":17,"price":2},{"id":"W744","units":9,"price":1}]Checked answer{"total": 435}wrong_schema_or_value
{
"total": 100
}12 tokens · 0.333swrong_schema_or_value
{"total":371}8 tokens · 0.223swrong_schema_or_value
{"total":342}8 tokens · 0.226swrong_schema_or_value
{"total":471}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"C907","priority":8},{"id":"C254","priority":3},{"id":"I259","priority":2},{"id":"Z806","priority":2},{"id":"G483","priority":4}]Checked answer{"ids": ["C907", "G483"]}invalid_json_object
[{"id":"C907","priority":8},{"id":"C254","priority":3},{"id":"I259","priority":2},{"id":"Z806","priority":2},{"id":"G483","priority":4}]62 tokens · 1.696swrong_schema_or_value
{"ids":["C907","C254"]}16 tokens · 0.445swrong_schema_or_value
{"ids":["C907","C254"]}16 tokens · 0.446swrong_schema_or_value
{"ids":["C907","C254"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record E679. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"P182","city":"Bern","units":16},{"id":"G476","city":"Riga","units":1},{"id":"O407","city":"Bern","units":2},{"id":"E679","city":"Bern","units":14},{"id":"I647","city":"Kyoto","units":8}]Checked answer{"city": "Bern", "units": 14}wrong_schema_or_value
{
"city": "Bern",
"units": 16
}18 tokens · 0.497sCorrect
{"city":"Bern","units":14}11 tokens · 0.310sCorrect
{"city":"Bern","units":14}11 tokens · 0.313sCorrect
{"city":"Bern","units":14}11 tokens · 0.307sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"L848","status":"ready"},{"id":"L472","status":"closed"},{"id":"P127","status":"pending"},{"id":"Y674","status":"pending"},{"id":"T191","status":"ready"}]Checked answer{"ids": ["L848", "T191"]}wrong_schema_or_value
{"ids": ["L472", "L848", "P127", "T191", "Y674"]}35 tokens · 0.956sCorrect
{"ids":["L848","T191"]}16 tokens · 0.448sCorrect
{"ids":["L848","T191"]}16 tokens · 0.446sCorrect
{"ids":["L848","T191"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"B702","units":11,"price":14},{"id":"C617","units":13,"price":4},{"id":"W964","units":18,"price":5},{"id":"Y164","units":19,"price":11},{"id":"K530","units":8,"price":5}]Checked answer{"total": 545}invalid_json_object
{"total": 11 * 14 + 13 * 4 + 18 * 5 + 19 * 11 + 8 * 5}40 tokens · 1.094swrong_schema_or_value
{"total":431}8 tokens · 0.227swrong_schema_or_value
{"total":432}8 tokens · 0.225swrong_schema_or_value
{"total":441}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"C328","priority":1},{"id":"P886","priority":6},{"id":"Y951","priority":4},{"id":"H191","priority":8},{"id":"T605","priority":1}]Checked answer{"ids": ["H191", "P886"]}wrong_schema_or_value
{"ids": ["C328", "P886", "Y951", "H191", "T605"]}35 tokens · 0.960sCorrect
{"ids":["H191","P886"]}16 tokens · 0.447sCorrect
{"ids":["H191","P886"]}16 tokens · 0.446sCorrect
{"ids":["H191","P886"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id J510 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"J510","city":"Oslo","units":6},{"id":"V992","city":"Accra","units":7},{"id":"L267","city":"Kyoto","units":10},{"id":"K208","city":"Riga","units":6},{"id":"F652","city":"Suva","units":14}]Checked answer{"city": "Oslo", "units": 6}Correct
{"city":"Oslo","units":6}11 tokens · 0.306sCorrect
{"city":"Oslo","units":6}11 tokens · 0.309sCorrect
{"city":"Oslo","units":6}11 tokens · 0.307sCorrect
{"city":"Oslo","units":6}11 tokens · 0.308sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"H596","status":"closed"},{"id":"G222","status":"pending"},{"id":"Z134","status":"closed"},{"id":"X850","status":"ready"},{"id":"D388","status":"ready"}]Checked answer{"ids": ["D388", "X850"]}wrong_schema_or_value
{
"ids": ["X850", "D388", "H596", "G222", "Z134"]
}38 tokens · 1.037sCorrect
{"ids":["D388","X850"]}16 tokens · 0.441sCorrect
{"ids":["D388","X850"]}16 tokens · 0.445sCorrect
{"ids":["D388","X850"]}16 tokens · 0.445sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"N577","units":11,"price":6},{"id":"L703","units":15,"price":5},{"id":"X678","units":11,"price":6},{"id":"U939","units":0,"price":2},{"id":"R349","units":3,"price":5}]Checked answer{"total": 222}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":341}8 tokens · 0.225swrong_schema_or_value
{"total":232}8 tokens · 0.225swrong_schema_or_value
{"total":331}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"B225","priority":9},{"id":"L710","priority":4},{"id":"Q308","priority":8},{"id":"Z890","priority":4},{"id":"K818","priority":7}]Checked answer{"ids": ["B225", "Q308"]}invalid_json_object
[{"id":"B225","priority":9},{"id":"L710","priority":4},{"id":"Q308","priority":8},{"id":"Z890","priority":4},{"id":"K818","priority":7}]62 tokens · 1.699sCorrect
{"ids":["B225","Q308"]}16 tokens · 0.444swrong_schema_or_value
{"ids":["B225","K818"]}16 tokens · 0.448sCorrect
{"ids":["B225","Q308"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to F599? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"Q466","city":"Oslo","units":4},{"id":"Z307","city":"Accra","units":6},{"id":"V864","city":"Accra","units":9},{"id":"C323","city":"Perth","units":17},{"id":"F599","city":"Bern","units":10}]Checked answer{"city": "Bern", "units": 10}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.415sCorrect
{"city":"Bern","units":10}11 tokens · 0.307sCorrect
{"city":"Bern","units":10}11 tokens · 0.309sCorrect
{"city":"Bern","units":10}11 tokens · 0.324sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"J354","status":"ready"},{"id":"K529","status":"pending"},{"id":"P265","status":"closed"},{"id":"L396","status":"closed"},{"id":"U461","status":"pending"}]Checked answer{"ids": ["J354"]}invalid_json_object
[{"id":"J354"},{"id":"K529"},{"id":"P265"},{"id":"L396"},{"id":"U461"}]43 tokens · 1.176sCorrect
{"ids":["J354"]}11 tokens · 0.308sCorrect
{"ids":["J354"]}11 tokens · 0.308sCorrect
{"ids":["J354"]}11 tokens · 0.313sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"V607","units":0,"price":6},{"id":"W219","units":11,"price":5},{"id":"J430","units":18,"price":9},{"id":"Z693","units":7,"price":18},{"id":"B728","units":6,"price":2}]Checked answer{"total": 355}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":331}8 tokens · 0.225swrong_schema_or_value
{"total":232}8 tokens · 0.225swrong_schema_or_value
{"total":341}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"N748","priority":2},{"id":"I457","priority":7},{"id":"O652","priority":3},{"id":"M306","priority":8},{"id":"M164","priority":1}]Checked answer{"ids": ["M306", "I457"]}wrong_schema_or_value
{"ids": ["M306", "M164", "N748", "O652", "I457", "M306"]}41 tokens · 1.118sCorrect
{"ids":["M306","I457"]}16 tokens · 0.446sCorrect
{"ids":["M306","I457"]}16 tokens · 0.441sCorrect
{"ids":["M306","I457"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"B704","city":"Riga","units":0},{"id":"Z171","city":"Kyoto","units":19},{"id":"G606","city":"Lima","units":9},{"id":"H711","city":"Suva","units":16},{"id":"U828","city":"Bern","units":19}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Riga","units":0}11 tokens · 0.305sCorrect
{"city":null,"units":null}10 tokens · 0.280swrong_schema_or_value
{"city":"Kyoto","units":19}13 tokens · 0.364sCorrect
{"city":null,"units":null}10 tokens · 0.283sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"M517","status":"pending"},{"id":"W602","status":"pending"},{"id":"T805","status":"ready"},{"id":"M250","status":"ready"},{"id":"P530","status":"closed"}]Checked answer{"ids": ["M250", "T805"]}wrong_schema_or_value
{"ids": ["M250", "M517", "P530", "T805", "W602"]}35 tokens · 0.956sCorrect
{"ids":["M250","T805"]}16 tokens · 0.448swrong_schema_or_value
{"ids":["T805","M250"]}16 tokens · 0.449sCorrect
{"ids":["M250","T805"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"G193","units":14,"price":5},{"id":"E989","units":6,"price":9},{"id":"I519","units":19,"price":2},{"id":"A501","units":12,"price":18},{"id":"T116","units":4,"price":11}]Checked answer{"total": 422}wrong_schema_or_value
{
"total": 100
}12 tokens · 0.331swrong_schema_or_value
{"total":331}8 tokens · 0.226swrong_schema_or_value
{"total":332}8 tokens · 0.228swrong_schema_or_value
{"total":341}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"I784","priority":7},{"id":"T782","priority":1},{"id":"O151","priority":9},{"id":"H847","priority":5},{"id":"A254","priority":2}]Checked answer{"ids": ["O151", "I784"]}invalid_json_object
[{"id":"I784","priority":7},{"id":"T782","priority":1},{"id":"O151","priority":9},{"id":"H847","priority":5},{"id":"A254","priority":2}]62 tokens · 1.689sCorrect
{"ids":["O151","I784"]}16 tokens · 0.447sCorrect
{"ids":["O151","I784"]}16 tokens · 0.446sCorrect
{"ids":["O151","I784"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record H467. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"H467","city":"Riga","units":15},{"id":"I606","city":"Riga","units":5},{"id":"X767","city":"Suva","units":15},{"id":"B662","city":"Perth","units":13},{"id":"Y375","city":"Bern","units":8}]Checked answer{"city": "Riga", "units": 15}Correct
{
"city": "Riga",
"units": 15
}19 tokens · 0.522sCorrect
{"city":"Riga","units":15}12 tokens · 0.338sCorrect
{"city":"Riga","units":15}12 tokens · 0.337sCorrect
{"city":"Riga","units":15}12 tokens · 0.341sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"J817","status":"pending"},{"id":"N779","status":"pending"},{"id":"R100","status":"closed"},{"id":"M770","status":"closed"},{"id":"M201","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["M201", "M770", "N779", "R100", "J817"]}35 tokens · 0.952sCorrect
{"ids":[]}6 tokens · 0.171swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.308sCorrect
{"ids":[]}6 tokens · 0.173sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"J897","units":18,"price":18},{"id":"Y772","units":15,"price":2},{"id":"D578","units":14,"price":4},{"id":"R955","units":0,"price":11},{"id":"R757","units":18,"price":9}]Checked answer{"total": 572}invalid_json_object
{"total": 18 * 18 + 15 * 2 + 14 * 4 + 0 * 11 + 18 * 9}40 tokens · 1.093swrong_schema_or_value
{"total":431}8 tokens · 0.225swrong_schema_or_value
{"total":432}8 tokens · 0.227swrong_schema_or_value
{"total":433}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Q225","priority":9},{"id":"K964","priority":7},{"id":"Q943","priority":7},{"id":"U541","priority":5},{"id":"H578","priority":7}]Checked answer{"ids": ["Q225", "H578"]}wrong_schema_or_value
{"ids": ["Q225", "K964", "Q943", "U541", "H578"]}35 tokens · 0.957swrong_schema_or_value
{"ids":["Q225","Q943"]}16 tokens · 0.446sCorrect
{"ids":["Q225","H578"]}16 tokens · 0.448sCorrect
{"ids":["Q225","H578"]}16 tokens · 0.451sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id P288 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"L158","city":"Bern","units":5},{"id":"D571","city":"Perth","units":14},{"id":"G804","city":"Lima","units":7},{"id":"P288","city":"Perth","units":16},{"id":"J726","city":"Accra","units":17}]Checked answer{"city": "Perth", "units": 16}Correct
{"city":"Perth","units":16}12 tokens · 0.333sCorrect
{"city":"Perth","units":16}12 tokens · 0.336sCorrect
{"city":"Perth","units":16}12 tokens · 0.337sCorrect
{"city":"Perth","units":16}12 tokens · 0.339sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"D881","status":"closed"},{"id":"O535","status":"ready"},{"id":"V311","status":"ready"},{"id":"F692","status":"pending"},{"id":"K588","status":"pending"}]Checked answer{"ids": ["O535", "V311"]}wrong_schema_or_value
{
"ids": ["O535", "V311", "F692", "D881"]
}32 tokens · 0.874sCorrect
{"ids":["O535","V311"]}16 tokens · 0.461sCorrect
{"ids":["O535","V311"]}16 tokens · 0.449sCorrect
{"ids":["O535","V311"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"X729","units":15,"price":8},{"id":"R813","units":0,"price":17},{"id":"L250","units":15,"price":3},{"id":"B953","units":11,"price":13},{"id":"B141","units":6,"price":3}]Checked answer{"total": 326}wrong_schema_or_value
{"total": 103}9 tokens · 0.250swrong_schema_or_value
{"total":331}8 tokens · 0.233swrong_schema_or_value
{"total":332}8 tokens · 0.225swrong_schema_or_value
{"total":341}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"W838","priority":9},{"id":"V296","priority":2},{"id":"L682","priority":8},{"id":"Q158","priority":2},{"id":"Z359","priority":4}]Checked answer{"ids": ["W838", "L682"]}invalid_json_object
[{"id":"W838","priority":9},{"id":"V296","priority":2},{"id":"L682","priority":8},{"id":"Q158","priority":2},{"id":"Z359","priority":4}]62 tokens · 1.697sCorrect
{"ids":["W838","L682"]}16 tokens · 0.461sCorrect
{"ids":["W838","L682"]}16 tokens · 0.445sCorrect
{"ids":["W838","L682"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z323? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"G549","city":"Kyoto","units":16},{"id":"F235","city":"Perth","units":15},{"id":"Q300","city":"Accra","units":3},{"id":"V687","city":"Suva","units":5},{"id":"Z323","city":"Perth","units":11}]Checked answer{"city": "Perth", "units": 11}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.416sCorrect
{"city":"Perth","units":11}12 tokens · 0.335sCorrect
{"city":"Perth","units":11}12 tokens · 0.339sCorrect
{"city":"Perth","units":11}12 tokens · 0.337sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"S396","status":"pending"},{"id":"H236","status":"ready"},{"id":"Q470","status":"closed"},{"id":"V334","status":"ready"},{"id":"X795","status":"pending"}]Checked answer{"ids": ["H236", "V334"]}invalid_json_object
[{"id":"H236","status":"ready"},{"id":"V334","status":"ready"},{"id":"X795","status":"pending"},{"id":"S396","status":"pending"},{"id":"Q470","status":"closed"}]63 tokens · 1.726sCorrect
{"ids":["H236","V334"]}16 tokens · 0.445sCorrect
{"ids":["H236","V334"]}16 tokens · 0.449sCorrect
{"ids":["H236","V334"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Z104","units":13,"price":18},{"id":"S798","units":3,"price":9},{"id":"Z678","units":6,"price":1},{"id":"D289","units":18,"price":13},{"id":"Q653","units":13,"price":1}]Checked answer{"total": 514}wrong_schema_or_value
{"total": 120}9 tokens · 0.252swrong_schema_or_value
{"total":341}8 tokens · 0.226swrong_schema_or_value
{"total":432}8 tokens · 0.225swrong_schema_or_value
{"total":371}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"J613","priority":8},{"id":"L946","priority":9},{"id":"F341","priority":2},{"id":"Z394","priority":5},{"id":"D522","priority":7}]Checked answer{"ids": ["L946", "J613"]}wrong_schema_or_value
{"ids": ["J613", "L946", "F341", "Z394", "D522"]}35 tokens · 0.963sCorrect
{"ids":["L946","J613"]}16 tokens · 0.448swrong_schema_or_value
{"ids":["L946","D522"]}16 tokens · 0.448sCorrect
{"ids":["L946","J613"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=G389. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"W911","city":"Riga","units":4},{"id":"L644","city":"Riga","units":13},{"id":"X330","city":"Kyoto","units":11},{"id":"S845","city":"Lima","units":13},{"id":"G389","city":"Riga","units":3}]Checked answer{"city": "Riga", "units": 3}wrong_schema_or_value
{"city":"Riga","units":4}11 tokens · 0.308sCorrect
{"city":"Riga","units":3}11 tokens · 0.310sCorrect
{"city":"Riga","units":3}11 tokens · 0.308sCorrect
{"city":"Riga","units":3}11 tokens · 0.311sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"T144","status":"closed"},{"id":"F224","status":"closed"},{"id":"Q143","status":"closed"},{"id":"Y754","status":"closed"},{"id":"X259","status":"ready"}]Checked answer{"ids": ["X259"]}wrong_schema_or_value
{"ids": ["X259", "F224", "Y754", "Q143", "T144"]}35 tokens · 0.963sCorrect
{"ids":["X259"]}11 tokens · 0.308sCorrect
{"ids":["X259"]}11 tokens · 0.308sCorrect
{"ids":["X259"]}11 tokens · 0.307sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"L302","units":7,"price":4},{"id":"R585","units":19,"price":17},{"id":"P644","units":14,"price":9},{"id":"H764","units":15,"price":11},{"id":"B472","units":2,"price":8}]Checked answer{"total": 658}wrong_schema_or_value
{
"total": 100
}12 tokens · 0.336swrong_schema_or_value
{"total":431}8 tokens · 0.229swrong_schema_or_value
{"total":342}8 tokens · 0.225swrong_schema_or_value
{"total":441}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"E556","priority":2},{"id":"Z217","priority":4},{"id":"L487","priority":6},{"id":"O954","priority":7},{"id":"V273","priority":3}]Checked answer{"ids": ["O954", "L487"]}invalid_json_object
[{"id":"O954","priority":7},{"id":"V273","priority":3},{"id":"E556","priority":2},{"id":"Z217","priority":4},{"id":"L487","priority":6}]62 tokens · 1.697sCorrect
{"ids":["O954","L487"]}16 tokens · 0.449sCorrect
{"ids":["O954","L487"]}16 tokens · 0.448sCorrect
{"ids":["O954","L487"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"R340","city":"Lima","units":6},{"id":"X222","city":"Kyoto","units":13},{"id":"T494","city":"Perth","units":9},{"id":"P347","city":"Bern","units":2},{"id":"S277","city":"Lima","units":4}]Checked answer{"city": null, "units": null}invalid_json_object
[{"city":"Lima","units":6},{"city":"Kyoto","units":13},{"city":"Perth","units":9},{"city":"Bern","units":2},{"city":"Lima","units":4}]53 tokens · 1.453sCorrect
{"city":null,"units":null}10 tokens · 0.281sCorrect
{"city":null,"units":null}10 tokens · 0.285sCorrect
{"city":null,"units":null}10 tokens · 0.283sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"N577","status":"ready"},{"id":"C453","status":"pending"},{"id":"K576","status":"closed"},{"id":"V390","status":"ready"},{"id":"L496","status":"pending"}]Checked answer{"ids": ["N577", "V390"]}wrong_schema_or_value
{"ids": ["C453", "K576", "L496", "N577", "V390"]}35 tokens · 0.960sCorrect
{"ids":["N577","V390"]}16 tokens · 0.446sCorrect
{"ids":["N577","V390"]}16 tokens · 0.443sCorrect
{"ids":["N577","V390"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"R943","units":16,"price":13},{"id":"F718","units":12,"price":11},{"id":"Q357","units":15,"price":8},{"id":"U549","units":12,"price":9},{"id":"P180","units":12,"price":9}]Checked answer{"total": 676}wrong_schema_or_value
{"total": 130}9 tokens · 0.252swrong_schema_or_value
{"total":471}8 tokens · 0.226swrong_schema_or_value
{"total":342}8 tokens · 0.225swrong_schema_or_value
{"total":441}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"M883","priority":7},{"id":"C961","priority":6},{"id":"Y898","priority":9},{"id":"T528","priority":9},{"id":"N948","priority":9}]Checked answer{"ids": ["N948", "T528"]}wrong_schema_or_value
{"ids": ["M883", "Y898", "T528", "N948", "C961", "P900"]}41 tokens · 1.124sCorrect
{"ids":["N948","T528"]}16 tokens · 0.446sCorrect
{"ids":["N948","T528"]}16 tokens · 0.447sCorrect
{"ids":["N948","T528"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id P787 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"F762","city":"Lima","units":12},{"id":"L621","city":"Kyoto","units":3},{"id":"P787","city":"Kyoto","units":18},{"id":"Z955","city":"Oslo","units":6},{"id":"T121","city":"Bern","units":10}]Checked answer{"city": "Kyoto", "units": 18}wrong_schema_or_value
{"city":"Lima","units":12}12 tokens · 0.333sCorrect
{"city":"Kyoto","units":18}13 tokens · 0.367sCorrect
{"city":"Kyoto","units":18}13 tokens · 0.364sCorrect
{"city":"Kyoto","units":18}13 tokens · 0.369sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"E926","status":"closed"},{"id":"T526","status":"closed"},{"id":"T856","status":"ready"},{"id":"U675","status":"pending"},{"id":"A415","status":"ready"}]Checked answer{"ids": ["A415", "T856"]}wrong_schema_or_value
{
"ids": ["A415", "E926", "T856", "T526"]
}32 tokens · 0.873sCorrect
{"ids":["A415","T856"]}16 tokens · 0.447sCorrect
{"ids":["A415","T856"]}16 tokens · 0.445sCorrect
{"ids":["A415","T856"]}16 tokens · 0.456sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"D336","units":5,"price":1},{"id":"K510","units":11,"price":11},{"id":"L663","units":19,"price":5},{"id":"A715","units":11,"price":5},{"id":"F438","units":11,"price":17}]Checked answer{"total": 463}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":341}8 tokens · 0.223swrong_schema_or_value
{"total":342}8 tokens · 0.226swrong_schema_or_value
{"total":441}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"L492","priority":1},{"id":"B322","priority":2},{"id":"O528","priority":3},{"id":"Z222","priority":4},{"id":"S494","priority":7}]Checked answer{"ids": ["S494", "Z222"]}invalid_json_object
[{"id":"L492","priority":1},{"id":"B322","priority":2},{"id":"O528","priority":3},{"id":"Z222","priority":4},{"id":"S494","priority":7}]62 tokens · 1.693sCorrect
{"ids":["S494","Z222"]}16 tokens · 0.442sCorrect
{"ids":["S494","Z222"]}16 tokens · 0.447sCorrect
{"ids":["S494","Z222"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to K226? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"U175","city":"Lima","units":12},{"id":"K226","city":"Accra","units":15},{"id":"V879","city":"Riga","units":5},{"id":"K396","city":"Bern","units":5},{"id":"H410","city":"Perth","units":15}]Checked answer{"city": "Accra", "units": 15}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.416sCorrect
{"city":"Accra","units":15}12 tokens · 0.338sCorrect
{"city":"Accra","units":15}12 tokens · 0.337sCorrect
{"city":"Accra","units":15}12 tokens · 0.341sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"S529","status":"ready"},{"id":"E209","status":"pending"},{"id":"Z535","status":"ready"},{"id":"F265","status":"ready"},{"id":"A593","status":"ready"}]Checked answer{"ids": ["A593", "F265", "S529", "Z535"]}invalid_json_object
[{"id":"A593","status":"ready"},{"id":"F265","status":"ready"},{"id":"E209","status":"pending"},{"id":"S529","status":"ready"},{"id":"Z535","status":"ready"}]63 tokens · 1.716sCorrect
{"ids":["A593","F265","S529","Z535"]}26 tokens · 0.723sCorrect
{"ids":["A593","F265","S529","Z535"]}26 tokens · 0.725sCorrect
{"ids":["A593","F265","S529","Z535"]}26 tokens · 0.745sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"D976","units":12,"price":12},{"id":"K382","units":5,"price":16},{"id":"C933","units":13,"price":1},{"id":"N247","units":4,"price":5},{"id":"T482","units":9,"price":4}]Checked answer{"total": 293}wrong_schema_or_value
{"total": 100}9 tokens · 0.250swrong_schema_or_value
{"total":311}8 tokens · 0.228swrong_schema_or_value
{"total":342}8 tokens · 0.224swrong_schema_or_value
{"total":311}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"Z804","priority":8},{"id":"Z554","priority":4},{"id":"M943","priority":3},{"id":"I527","priority":1},{"id":"O770","priority":8}]Checked answer{"ids": ["O770", "Z804"]}wrong_schema_or_value
{"ids": ["M943", "O770"]}17 tokens · 0.467sCorrect
{"ids":["O770","Z804"]}16 tokens · 0.449sCorrect
{"ids":["O770","Z804"]}16 tokens · 0.443sCorrect
{"ids":["O770","Z804"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z746. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"P953","city":"Kyoto","units":5},{"id":"U301","city":"Kyoto","units":3},{"id":"C823","city":"Oslo","units":4},{"id":"C565","city":"Riga","units":18},{"id":"Z746","city":"Lima","units":13}]Checked answer{"city": "Lima", "units": 13}wrong_schema_or_value
{"city":"Kyoto","units":5}12 tokens · 0.331sCorrect
{"city":"Lima","units":13}12 tokens · 0.339sCorrect
{"city":"Lima","units":13}12 tokens · 0.337sCorrect
{"city":"Lima","units":13}12 tokens · 0.338sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"F889","status":"pending"},{"id":"Z138","status":"ready"},{"id":"R442","status":"pending"},{"id":"X883","status":"ready"},{"id":"J765","status":"ready"}]Checked answer{"ids": ["J765", "X883", "Z138"]}wrong_schema_or_value
{"ids": ["F889", "Z138", "R442", "X883", "J765"]}35 tokens · 0.953sCorrect
{"ids":["J765","X883","Z138"]}21 tokens · 0.580sCorrect
{"ids":["J765","X883","Z138"]}21 tokens · 0.589sCorrect
{"ids":["J765","X883","Z138"]}21 tokens · 0.585sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"Q474","units":16,"price":16},{"id":"B467","units":13,"price":9},{"id":"S974","units":19,"price":15},{"id":"L605","units":4,"price":13},{"id":"M503","units":3,"price":12}]Checked answer{"total": 746}wrong_schema_or_value
{
"total": 438
}12 tokens · 0.331swrong_schema_or_value
{"total":431}8 tokens · 0.227swrong_schema_or_value
{"total":432}8 tokens · 0.227swrong_schema_or_value
{"total":473}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"F479","priority":5},{"id":"U459","priority":7},{"id":"G610","priority":7},{"id":"D455","priority":4},{"id":"D981","priority":6}]Checked answer{"ids": ["G610", "U459"]}invalid_json_object
[{"id":"U459","priority":7},{"id":"D455","priority":4},{"id":"D981","priority":6},{"id":"F479","priority":5},{"id":"G610","priority":7}]62 tokens · 1.682sCorrect
{"ids":["G610","U459"]}16 tokens · 0.445sCorrect
{"ids":["G610","U459"]}16 tokens · 0.447sCorrect
{"ids":["G610","U459"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Q273. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Y160","city":"Suva","units":10},{"id":"Q273","city":"Bern","units":18},{"id":"O778","city":"Perth","units":1},{"id":"T379","city":"Bern","units":19},{"id":"W999","city":"Perth","units":15}]Checked answer{"city": "Bern", "units": 18}wrong_schema_or_value
{
"city": "Suva",
"units": 10
}19 tokens · 0.520sCorrect
{"city":"Bern","units":18}11 tokens · 0.308sCorrect
{"city":"Bern","units":18}11 tokens · 0.310sCorrect
{"city":"Bern","units":18}11 tokens · 0.306sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"L188","status":"closed"},{"id":"Z871","status":"pending"},{"id":"Q833","status":"ready"},{"id":"F816","status":"pending"},{"id":"B924","status":"pending"}]Checked answer{"ids": ["Q833"]}wrong_schema_or_value
{"ids": ["B924", "F816", "L188", "Q833", "Z871"]}35 tokens · 0.952sCorrect
{"ids":["Q833"]}11 tokens · 0.307sCorrect
{"ids":["Q833"]}11 tokens · 0.311sCorrect
{"ids":["Q833"]}11 tokens · 0.304sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"G679","units":4,"price":19},{"id":"S918","units":3,"price":3},{"id":"E569","units":9,"price":4},{"id":"F598","units":0,"price":4},{"id":"G282","units":3,"price":19}]Checked answer{"total": 178}wrong_schema_or_value
{"total": 100}9 tokens · 0.250swrong_schema_or_value
{"total":223}8 tokens · 0.223swrong_schema_or_value
{"total":422}8 tokens · 0.228swrong_schema_or_value
{"total":222}8 tokens · 0.223sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Z715","priority":2},{"id":"Y705","priority":6},{"id":"N328","priority":7},{"id":"Z335","priority":7},{"id":"N566","priority":6}]Checked answer{"ids": ["N328", "Z335"]}wrong_schema_or_value
{"ids": ["N328", "N566", "Z715", "Z335", "Y705"]}35 tokens · 0.959swrong_schema_or_value
{"ids":["N328","N566"]}16 tokens · 0.454sCorrect
{"ids":["N328","Z335"]}16 tokens · 0.449swrong_schema_or_value
{"ids":["N328","N566"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Z000 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"J491","city":"Suva","units":0},{"id":"E639","city":"Suva","units":6},{"id":"S649","city":"Riga","units":16},{"id":"D922","city":"Bern","units":5},{"id":"Y937","city":"Oslo","units":18}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Suva","units":0}11 tokens · 0.305sCorrect
{"city":null,"units":null}10 tokens · 0.283sCorrect
{"city":null,"units":null}10 tokens · 0.281sCorrect
{"city":null,"units":null}10 tokens · 0.280sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"K774","status":"pending"},{"id":"R869","status":"ready"},{"id":"Z235","status":"closed"},{"id":"Z232","status":"pending"},{"id":"R769","status":"pending"}]Checked answer{"ids": ["R869"]}wrong_schema_or_value
{
"ids": ["R769", "R869", "Z232", "Z235"]
}32 tokens · 0.874sCorrect
{"ids":["R869"]}11 tokens · 0.307sCorrect
{"ids":["R869"]}11 tokens · 0.309sCorrect
{"ids":["R869"]}11 tokens · 0.310sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"X194","units":19,"price":8},{"id":"E167","units":15,"price":2},{"id":"H200","units":14,"price":3},{"id":"M957","units":14,"price":14},{"id":"V311","units":17,"price":19}]Checked answer{"total": 743}wrong_schema_or_value
{"total": 103}9 tokens · 0.251swrong_schema_or_value
{"total":711}8 tokens · 0.229swrong_schema_or_value
{"total":432}8 tokens · 0.224swrong_schema_or_value
{"total":731}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"E273","priority":3},{"id":"J604","priority":8},{"id":"S773","priority":9},{"id":"H860","priority":7},{"id":"B343","priority":6}]Checked answer{"ids": ["S773", "J604"]}invalid_json_object
[{"id":"E273","priority":3},{"id":"J604","priority":8},{"id":"S773","priority":9},{"id":"H860","priority":7},{"id":"B343","priority":6}]62 tokens · 1.689sCorrect
{"ids":["S773","J604"]}16 tokens · 0.450sCorrect
{"ids":["S773","J604"]}16 tokens · 0.443sCorrect
{"ids":["S773","J604"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to X963? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"E407","city":"Perth","units":12},{"id":"N326","city":"Perth","units":17},{"id":"X963","city":"Accra","units":12},{"id":"O547","city":"Kyoto","units":16},{"id":"W405","city":"Oslo","units":7}]Checked answer{"city": "Accra", "units": 12}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.415sCorrect
{"city":"Accra","units":12}12 tokens · 0.339sCorrect
{"city":"Accra","units":12}12 tokens · 0.333sCorrect
{"city":"Accra","units":12}12 tokens · 0.339sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"B568","status":"closed"},{"id":"S199","status":"pending"},{"id":"N852","status":"closed"},{"id":"D377","status":"pending"},{"id":"Z917","status":"ready"}]Checked answer{"ids": ["Z917"]}invalid_json_object
[{"id":"B568"},{"id":"D377"},{"id":"Z917"}]27 tokens · 0.741sCorrect
{"ids":["Z917"]}11 tokens · 0.310sCorrect
{"ids":["Z917"]}11 tokens · 0.302sCorrect
{"ids":["Z917"]}11 tokens · 0.311sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"U468","units":2,"price":11},{"id":"X819","units":8,"price":5},{"id":"M327","units":9,"price":12},{"id":"Q719","units":4,"price":15},{"id":"P362","units":13,"price":6}]Checked answer{"total": 308}wrong_schema_or_value
{"total": 110}9 tokens · 0.251swrong_schema_or_value
{"total":310}8 tokens · 0.227swrong_schema_or_value
{"total":332}8 tokens · 0.225swrong_schema_or_value
{"total":310}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"S118","priority":2},{"id":"S398","priority":2},{"id":"U697","priority":7},{"id":"E461","priority":8},{"id":"T600","priority":1}]Checked answer{"ids": ["E461", "U697"]}wrong_schema_or_value
{"ids": ["S118", "S398", "U697", "E461", "T600"]}35 tokens · 0.971sCorrect
{"ids":["E461","U697"]}16 tokens · 0.447sCorrect
{"ids":["E461","U697"]}16 tokens · 0.444sCorrect
{"ids":["E461","U697"]}16 tokens · 0.449sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=G660. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"J406","city":"Perth","units":13},{"id":"Y276","city":"Accra","units":19},{"id":"E194","city":"Perth","units":13},{"id":"G660","city":"Suva","units":14},{"id":"D532","city":"Suva","units":18}]Checked answer{"city": "Suva", "units": 14}Correct
{"city":"Suva","units":14}12 tokens · 0.345sCorrect
{"city":"Suva","units":14}12 tokens · 0.337sCorrect
{"city":"Suva","units":14}12 tokens · 0.334sCorrect
{"city":"Suva","units":14}12 tokens · 0.339sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"W565","status":"closed"},{"id":"M982","status":"closed"},{"id":"W874","status":"pending"},{"id":"E498","status":"closed"},{"id":"M213","status":"closed"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["M213", "E498", "M982", "W565", "W874", "W874"]}41 tokens · 1.126sCorrect
{"ids":[]}6 tokens · 0.172swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.304sCorrect
{"ids":[]}6 tokens · 0.173sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"T646","units":14,"price":14},{"id":"T944","units":16,"price":18},{"id":"P543","units":12,"price":13},{"id":"H735","units":18,"price":12},{"id":"N824","units":4,"price":6}]Checked answer{"total": 880}wrong_schema_or_value
{
"total": 43
}11 tokens · 0.307swrong_schema_or_value
{"total":431}8 tokens · 0.228swrong_schema_or_value
{"total":432}8 tokens · 0.223swrong_schema_or_value
{"total":471}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"D659","priority":8},{"id":"U848","priority":5},{"id":"G539","priority":4},{"id":"K553","priority":6},{"id":"V724","priority":1}]Checked answer{"ids": ["D659", "K553"]}invalid_json_object
[{"id":"U848","priority":5},{"id":"G539","priority":4},{"id":"V724","priority":1},{"id":"D659","priority":8},{"id":"K553","priority":6}]62 tokens · 1.685sCorrect
{"ids":["D659","K553"]}16 tokens · 0.443sCorrect
{"ids":["D659","K553"]}16 tokens · 0.443sCorrect
{"ids":["D659","K553"]}16 tokens · 0.448sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record U671. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"K828","city":"Suva","units":1},{"id":"K789","city":"Accra","units":8},{"id":"I930","city":"Kyoto","units":8},{"id":"U671","city":"Kyoto","units":5},{"id":"V740","city":"Riga","units":18}]Checked answer{"city": "Kyoto", "units": 5}Correct
{
"city": "Kyoto",
"units": 5
}19 tokens · 0.523sCorrect
{"city":"Kyoto","units":5}12 tokens · 0.337sCorrect
{"city":"Kyoto","units":5}12 tokens · 0.332sCorrect
{"city":"Kyoto","units":5}12 tokens · 0.337sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"E676","status":"pending"},{"id":"P851","status":"closed"},{"id":"V690","status":"pending"},{"id":"C339","status":"closed"},{"id":"B503","status":"ready"}]Checked answer{"ids": ["B503"]}wrong_schema_or_value
{"ids": ["B503", "C339", "E676", "P851", "V690"]}35 tokens · 0.956sCorrect
{"ids":["B503"]}11 tokens · 0.305sCorrect
{"ids":["B503"]}11 tokens · 0.305sCorrect
{"ids":["B503"]}11 tokens · 0.311sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"V274","units":14,"price":11},{"id":"E908","units":1,"price":12},{"id":"Z906","units":10,"price":3},{"id":"Y264","units":15,"price":16},{"id":"P994","units":5,"price":7}]Checked answer{"total": 471}invalid_json_object
{"total": 11 + 12 + 3 + 16 + 7 = 47}26 tokens · 0.716swrong_schema_or_value
{"total":341}8 tokens · 0.224swrong_schema_or_value
{"total":342}8 tokens · 0.224swrong_schema_or_value
{"total":331}8 tokens · 0.228sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"T873","priority":2},{"id":"K859","priority":4},{"id":"E504","priority":9},{"id":"G751","priority":8},{"id":"A923","priority":7}]Checked answer{"ids": ["E504", "G751"]}wrong_schema_or_value
{"ids": ["K859", "E504", "G751", "A923", "T873"]}35 tokens · 0.960sCorrect
{"ids":["E504","G751"]}16 tokens · 0.440sCorrect
{"ids":["E504","G751"]}16 tokens · 0.443sCorrect
{"ids":["E504","G751"]}16 tokens · 0.442sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id X286 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"V946","city":"Kyoto","units":9},{"id":"L220","city":"Suva","units":4},{"id":"Q258","city":"Riga","units":14},{"id":"W955","city":"Bern","units":11},{"id":"X286","city":"Kyoto","units":13}]Checked answer{"city": "Kyoto", "units": 13}Correct
{"city":"Kyoto","units":13}13 tokens · 0.361sCorrect
{"city":"Kyoto","units":13}13 tokens · 0.357sCorrect
{"city":"Kyoto","units":13}13 tokens · 0.374sCorrect
{"city":"Kyoto","units":13}13 tokens · 0.366sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"H965","status":"closed"},{"id":"S431","status":"pending"},{"id":"N312","status":"closed"},{"id":"Q506","status":"ready"},{"id":"A210","status":"pending"}]Checked answer{"ids": ["Q506"]}wrong_schema_or_value
{
"ids": ["A210", "H965", "N312", "Q506", "S431"]
}38 tokens · 1.041sCorrect
{"ids":["Q506"]}11 tokens · 0.305sCorrect
{"ids":["Q506"]}11 tokens · 0.315sCorrect
{"ids":["Q506"]}11 tokens · 0.310sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"W808","units":5,"price":1},{"id":"Q528","units":7,"price":3},{"id":"U704","units":0,"price":12},{"id":"V292","units":8,"price":11},{"id":"N719","units":11,"price":11}]Checked answer{"total": 235}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":311}8 tokens · 0.227swrong_schema_or_value
{"total":342}8 tokens · 0.236swrong_schema_or_value
{"total":241}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"L244","priority":4},{"id":"S210","priority":6},{"id":"H721","priority":2},{"id":"E968","priority":4},{"id":"Z960","priority":5}]Checked answer{"ids": ["S210", "Z960"]}invalid_json_object
[{"id":"L244","priority":4},{"id":"S210","priority":6},{"id":"H721","priority":2},{"id":"E968","priority":4},{"id":"Z960","priority":5}]62 tokens · 1.697sCorrect
{"ids":["S210","Z960"]}16 tokens · 0.450sCorrect
{"ids":["S210","Z960"]}16 tokens · 0.449sCorrect
{"ids":["S210","Z960"]}16 tokens · 0.454sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to Z000? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"T839","city":"Suva","units":7},{"id":"Z919","city":"Accra","units":5},{"id":"Z176","city":"Perth","units":17},{"id":"Q236","city":"Perth","units":6},{"id":"M604","city":"Riga","units":19}]Checked answer{"city": null, "units": null}Correct
{
"city": null,
"units": null
}15 tokens · 0.420sCorrect
{"city":null,"units":null}10 tokens · 0.282sCorrect
{"city":null,"units":null}10 tokens · 0.281sCorrect
{"city":null,"units":null}10 tokens · 0.283sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"A727","status":"ready"},{"id":"J276","status":"ready"},{"id":"W160","status":"closed"},{"id":"G430","status":"closed"},{"id":"X981","status":"ready"}]Checked answer{"ids": ["A727", "J276", "X981"]}invalid_json_object
[{"id":"A727","status":"ready"},{"id":"J276","status":"ready"},{"id":"W160","status":"closed"},{"id":"G430","status":"closed"},{"id":"X981","status":"ready"}]63 tokens · 1.719sCorrect
{"ids":["A727","J276","X981"]}21 tokens · 0.585sCorrect
{"ids":["A727","J276","X981"]}21 tokens · 0.585sCorrect
{"ids":["A727","J276","X981"]}21 tokens · 0.586sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"T858","units":0,"price":10},{"id":"C525","units":1,"price":5},{"id":"A400","units":17,"price":10},{"id":"G425","units":11,"price":19},{"id":"E667","units":17,"price":2}]Checked answer{"total": 418}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":371}8 tokens · 0.224swrong_schema_or_value
{"total":432}8 tokens · 0.225swrong_schema_or_value
{"total":431}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"M498","priority":1},{"id":"F692","priority":9},{"id":"Z955","priority":2},{"id":"C793","priority":4},{"id":"R595","priority":8}]Checked answer{"ids": ["F692", "R595"]}wrong_schema_or_value
{"ids": ["M498", "Z955"]}17 tokens · 0.469sCorrect
{"ids":["F692","R595"]}16 tokens · 0.447sCorrect
{"ids":["F692","R595"]}16 tokens · 0.446sCorrect
{"ids":["F692","R595"]}16 tokens · 0.450sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=X239. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"W549","city":"Suva","units":1},{"id":"X239","city":"Kyoto","units":14},{"id":"N611","city":"Riga","units":14},{"id":"L210","city":"Lima","units":1},{"id":"O934","city":"Oslo","units":0}]Checked answer{"city": "Kyoto", "units": 14}wrong_schema_or_value
{"city":"Suva","units":1}11 tokens · 0.306sCorrect
{"city":"Kyoto","units":14}13 tokens · 0.366sCorrect
{"city":"Kyoto","units":14}13 tokens · 0.365sCorrect
{"city":"Kyoto","units":14}13 tokens · 0.367sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"G433","status":"pending"},{"id":"Q116","status":"closed"},{"id":"R724","status":"pending"},{"id":"U622","status":"ready"},{"id":"S193","status":"pending"}]Checked answer{"ids": ["U622"]}wrong_schema_or_value
{"ids": ["U622", "S193", "G433", "R724", "Q116"]}35 tokens · 0.959sCorrect
{"ids":["U622"]}11 tokens · 0.309sCorrect
{"ids":["U622"]}11 tokens · 0.308sCorrect
{"ids":["U622"]}11 tokens · 0.312sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"W969","units":16,"price":1},{"id":"J812","units":1,"price":1},{"id":"Y477","units":10,"price":3},{"id":"U316","units":8,"price":12},{"id":"U978","units":16,"price":16}]Checked answer{"total": 399}wrong_schema_or_value
{
"total": 31
}11 tokens · 0.306swrong_schema_or_value
{"total":312}8 tokens · 0.227swrong_schema_or_value
{"total":342}8 tokens · 0.227swrong_schema_or_value
{"total":341}8 tokens · 0.225sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"D371","priority":7},{"id":"W427","priority":9},{"id":"R134","priority":2},{"id":"Q856","priority":5},{"id":"G641","priority":1}]Checked answer{"ids": ["W427", "D371"]}invalid_json_object
[{"id":"W427","priority":9},{"id":"R134","priority":2},{"id":"Q856","priority":5},{"id":"D371","priority":7},{"id":"G641","priority":1}]62 tokens · 1.698sCorrect
{"ids":["W427","D371"]}16 tokens · 0.448sCorrect
{"ids":["W427","D371"]}16 tokens · 0.447sCorrect
{"ids":["W427","D371"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record R562. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"B409","city":"Kyoto","units":5},{"id":"J614","city":"Kyoto","units":8},{"id":"M625","city":"Perth","units":10},{"id":"X301","city":"Perth","units":3},{"id":"R562","city":"Oslo","units":14}]Checked answer{"city": "Oslo", "units": 14}wrong_schema_or_value
{
"city": "Kyoto",
"units": 5
}19 tokens · 0.525sCorrect
{"city":"Oslo","units":14}12 tokens · 0.337sCorrect
{"city":"Oslo","units":14}12 tokens · 0.339sCorrect
{"city":"Oslo","units":14}12 tokens · 0.339sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"A277","status":"closed"},{"id":"L175","status":"pending"},{"id":"I398","status":"closed"},{"id":"M384","status":"pending"},{"id":"Q198","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{"ids": ["A277", "L175", "I398", "M384", "Q198"]}35 tokens · 0.955sCorrect
{"ids":[]}6 tokens · 0.171swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.307sCorrect
{"ids":[]}6 tokens · 0.170sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"M205","units":17,"price":2},{"id":"K770","units":3,"price":4},{"id":"W794","units":7,"price":8},{"id":"S614","units":13,"price":6},{"id":"N412","units":6,"price":2}]Checked answer{"total": 192}wrong_schema_or_value
{"total": 100}9 tokens · 0.251swrong_schema_or_value
{"total":211}8 tokens · 0.226swrong_schema_or_value
{"total":232}8 tokens · 0.226swrong_schema_or_value
{"total":210}8 tokens · 0.223sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"M795","priority":6},{"id":"U245","priority":3},{"id":"E146","priority":2},{"id":"C652","priority":9},{"id":"N907","priority":5}]Checked answer{"ids": ["C652", "M795"]}wrong_schema_or_value
{"ids": ["U245", "M795", "N907", "E146", "C652"]}35 tokens · 0.957sCorrect
{"ids":["C652","M795"]}16 tokens · 0.444sCorrect
{"ids":["C652","M795"]}16 tokens · 0.446sCorrect
{"ids":["C652","M795"]}16 tokens · 0.442sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id J906 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"J569","city":"Suva","units":6},{"id":"O394","city":"Oslo","units":9},{"id":"B502","city":"Kyoto","units":2},{"id":"G295","city":"Oslo","units":12},{"id":"J906","city":"Kyoto","units":14}]Checked answer{"city": "Kyoto", "units": 14}wrong_schema_or_value
{"city":"Suva","units":6}11 tokens · 0.306sCorrect
{"city":"Kyoto","units":14}13 tokens · 0.366sCorrect
{"city":"Kyoto","units":14}13 tokens · 0.368sCorrect
{"city":"Kyoto","units":14}13 tokens · 0.367sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"U655","status":"pending"},{"id":"L771","status":"pending"},{"id":"G401","status":"pending"},{"id":"W747","status":"closed"},{"id":"A333","status":"pending"}]Checked answer{"ids": []}wrong_schema_or_value
{
"ids": ["U655", "L771", "G401", "W747", "A333"]
}38 tokens · 1.041sCorrect
{"ids":[]}6 tokens · 0.172swrong_schema_or_value
{"ids":["Z000"]}11 tokens · 0.312sCorrect
{"ids":[]}6 tokens · 0.170sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"O149","units":11,"price":11},{"id":"E241","units":19,"price":9},{"id":"Q809","units":9,"price":9},{"id":"N856","units":17,"price":3},{"id":"J890","units":8,"price":18}]Checked answer{"total": 568}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":431}8 tokens · 0.224swrong_schema_or_value
{"total":342}8 tokens · 0.228swrong_schema_or_value
{"total":471}8 tokens · 0.225sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"J625","priority":3},{"id":"D924","priority":5},{"id":"M254","priority":3},{"id":"Y263","priority":4},{"id":"B351","priority":5}]Checked answer{"ids": ["B351", "D924"]}invalid_json_object
[{"id":"M254","priority":3},{"id":"J625","priority":3},{"id":"D924","priority":5},{"id":"Y263","priority":4},{"id":"B351","priority":5}]62 tokens · 1.694sCorrect
{"ids":["B351","D924"]}16 tokens · 0.447swrong_schema_or_value
{"ids":["D924","B351"]}16 tokens · 0.449sCorrect
{"ids":["B351","D924"]}16 tokens · 0.445sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to P554? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"E112","city":"Kyoto","units":5},{"id":"I531","city":"Suva","units":3},{"id":"P554","city":"Lima","units":9},{"id":"P704","city":"Perth","units":7},{"id":"W171","city":"Accra","units":16}]Checked answer{"city": "Lima", "units": 9}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.416sCorrect
{"city":"Lima","units":9}11 tokens · 0.311sCorrect
{"city":"Lima","units":9}11 tokens · 0.307sCorrect
{"city":"Lima","units":9}11 tokens · 0.313sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"Z257","status":"closed"},{"id":"H299","status":"pending"},{"id":"W415","status":"pending"},{"id":"Y973","status":"closed"},{"id":"B547","status":"ready"}]Checked answer{"ids": ["B547"]}invalid_json_object
[{"id":"B547","status":"ready"}]15 tokens · 0.416sCorrect
{"ids":["B547"]}11 tokens · 0.309sCorrect
{"ids":["B547"]}11 tokens · 0.311sCorrect
{"ids":["B547"]}11 tokens · 0.312sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"C421","units":19,"price":1},{"id":"U251","units":15,"price":18},{"id":"U706","units":1,"price":7},{"id":"A471","units":12,"price":4},{"id":"O370","units":14,"price":13}]Checked answer{"total": 526}wrong_schema_or_value
{"total": 100}9 tokens · 0.253swrong_schema_or_value
{"total":371}8 tokens · 0.226swrong_schema_or_value
{"total":332}8 tokens · 0.225swrong_schema_or_value
{"total":341}8 tokens · 0.224sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"K925","priority":1},{"id":"R227","priority":8},{"id":"B116","priority":7},{"id":"P415","priority":2},{"id":"N973","priority":3}]Checked answer{"ids": ["R227", "B116"]}wrong_schema_or_value
{"ids": ["K925", "R227", "B116", "P415", "N973"]}35 tokens · 0.962sCorrect
{"ids":["R227","B116"]}16 tokens · 0.448sCorrect
{"ids":["R227","B116"]}16 tokens · 0.448sCorrect
{"ids":["R227","B116"]}16 tokens · 0.442sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=Z000. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"A256","city":"Riga","units":16},{"id":"G882","city":"Oslo","units":6},{"id":"N629","city":"Accra","units":6},{"id":"T573","city":"Oslo","units":13},{"id":"R903","city":"Lima","units":19}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{"city":"Riga","units":16}12 tokens · 0.335sCorrect
{"city":null,"units":null}10 tokens · 0.281sCorrect
{"city":null,"units":null}10 tokens · 0.282sCorrect
{"city":null,"units":null}10 tokens · 0.282sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"N855","status":"closed"},{"id":"E271","status":"closed"},{"id":"T876","status":"closed"},{"id":"Q863","status":"ready"},{"id":"N960","status":"ready"}]Checked answer{"ids": ["N960", "Q863"]}wrong_schema_or_value
{"ids": ["Q863", "N855", "E271", "T876", "N960"]}35 tokens · 0.963sCorrect
{"ids":["N960","Q863"]}16 tokens · 0.448sCorrect
{"ids":["N960","Q863"]}16 tokens · 0.443sCorrect
{"ids":["N960","Q863"]}16 tokens · 0.442sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"G132","units":13,"price":9},{"id":"Z475","units":5,"price":18},{"id":"Y505","units":14,"price":6},{"id":"J279","units":4,"price":15},{"id":"M597","units":13,"price":15}]Checked answer{"total": 546}wrong_schema_or_value
{
"total": 338
}12 tokens · 0.335swrong_schema_or_value
{"total":341}8 tokens · 0.226swrong_schema_or_value
{"total":342}8 tokens · 0.227swrong_schema_or_value
{"total":431}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"K805","priority":1},{"id":"U211","priority":1},{"id":"L107","priority":2},{"id":"I523","priority":3},{"id":"G470","priority":4}]Checked answer{"ids": ["G470", "I523"]}invalid_json_object
[{"id":"K805"},{"id":"U211"},{"id":"L107"},{"id":"I523"},{"id":"G470"}]43 tokens · 1.179swrong_schema_or_value
{"ids":["G470","L107"]}16 tokens · 0.444swrong_schema_or_value
{"ids":["G470","L107"]}16 tokens · 0.447swrong_schema_or_value
{"ids":["G470","L107"]}16 tokens · 0.452sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record B652. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"Z986","city":"Lima","units":18},{"id":"B652","city":"Riga","units":4},{"id":"T337","city":"Suva","units":12},{"id":"L480","city":"Accra","units":8},{"id":"I703","city":"Oslo","units":12}]Checked answer{"city": "Riga", "units": 4}invalid_json_object
[{"city":"Riga","units":4}]12 tokens · 0.335sCorrect
{"city":"Riga","units":4}11 tokens · 0.313sCorrect
{"city":"Riga","units":4}11 tokens · 0.311sCorrect
{"city":"Riga","units":4}11 tokens · 0.312sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"L393","status":"pending"},{"id":"J319","status":"ready"},{"id":"Y240","status":"pending"},{"id":"M843","status":"pending"},{"id":"I161","status":"ready"}]Checked answer{"ids": ["I161", "J319"]}wrong_schema_or_value
{"ids": ["I161", "J319", "M843", "L393"]}29 tokens · 0.801sCorrect
{"ids":["I161","J319"]}16 tokens · 0.450sCorrect
{"ids":["I161","J319"]}16 tokens · 0.448sCorrect
{"ids":["I161","J319"]}16 tokens · 0.451sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Y128","units":7,"price":1},{"id":"C757","units":18,"price":5},{"id":"F440","units":14,"price":15},{"id":"B585","units":18,"price":4},{"id":"L919","units":16,"price":16}]Checked answer{"total": 635}wrong_schema_or_value
{"total": 100}9 tokens · 0.253swrong_schema_or_value
{"total":441}8 tokens · 0.225swrong_schema_or_value
{"total":432}8 tokens · 0.223swrong_schema_or_value
{"total":471}8 tokens · 0.226sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"Z383","priority":1},{"id":"N849","priority":8},{"id":"D362","priority":4},{"id":"B186","priority":7},{"id":"F590","priority":1}]Checked answer{"ids": ["N849", "B186"]}wrong_schema_or_value
{"ids": ["N849", "D362", "B186", "F590", "Z383"]}35 tokens · 0.957sCorrect
{"ids":["N849","B186"]}16 tokens · 0.447sCorrect
{"ids":["N849","B186"]}16 tokens · 0.451sCorrect
{"ids":["N849","B186"]}16 tokens · 0.446sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id Q639 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"I598","city":"Lima","units":16},{"id":"O649","city":"Suva","units":6},{"id":"Q639","city":"Riga","units":11},{"id":"Z225","city":"Oslo","units":14},{"id":"S232","city":"Oslo","units":18}]Checked answer{"city": "Riga", "units": 11}Correct
{"city":"Riga","units":11}12 tokens · 0.332sCorrect
{"city":"Riga","units":11}12 tokens · 0.339sCorrect
{"city":"Riga","units":11}12 tokens · 0.336sCorrect
{"city":"Riga","units":11}12 tokens · 0.334sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"F396","status":"pending"},{"id":"Z208","status":"ready"},{"id":"P860","status":"pending"},{"id":"K674","status":"ready"},{"id":"U383","status":"ready"}]Checked answer{"ids": ["K674", "U383", "Z208"]}wrong_schema_or_value
{
"ids": ["F396", "Z208", "P860", "K674", "U383"]
}38 tokens · 1.040sCorrect
{"ids":["K674","U383","Z208"]}21 tokens · 0.586swrong_schema_or_value
{"ids":["K674","F396","U383"]}21 tokens · 0.580sCorrect
{"ids":["K674","U383","Z208"]}21 tokens · 0.577sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"J903","units":6,"price":11},{"id":"P320","units":11,"price":19},{"id":"V832","units":15,"price":6},{"id":"P774","units":7,"price":15},{"id":"J279","units":9,"price":12}]Checked answer{"total": 578}wrong_schema_or_value
{"total": 103}9 tokens · 0.251swrong_schema_or_value
{"total":351}8 tokens · 0.226swrong_schema_or_value
{"total":342}8 tokens · 0.224swrong_schema_or_value
{"total":371}8 tokens · 0.224sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"A661","priority":3},{"id":"N349","priority":5},{"id":"Y802","priority":7},{"id":"N689","priority":6},{"id":"J873","priority":2}]Checked answer{"ids": ["Y802", "N689"]}invalid_json_object
[{"id":"A661","priority":3},{"id":"N689","priority":6},{"id":"Y802","priority":7}]38 tokens · 1.039swrong_schema_or_value
{"ids":["Y802","N349"]}16 tokens · 0.446swrong_schema_or_value
{"ids":["Y802","N349"]}16 tokens · 0.441swrong_schema_or_value
{"ids":["Y802","N349"]}16 tokens · 0.444sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to N442? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"N442","city":"Lima","units":0},{"id":"T545","city":"Bern","units":0},{"id":"N222","city":"Lima","units":13},{"id":"O454","city":"Riga","units":14},{"id":"I395","city":"Bern","units":9}]Checked answer{"city": "Lima", "units": 0}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.415swrong_schema_or_value
{"city":null,"units":null}10 tokens · 0.282swrong_schema_or_value
{"city":null,"units":null}10 tokens · 0.281swrong_schema_or_value
{"city":null,"units":null}10 tokens · 0.283sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"P528","status":"ready"},{"id":"I154","status":"closed"},{"id":"T972","status":"closed"},{"id":"E978","status":"closed"},{"id":"O324","status":"closed"}]Checked answer{"ids": ["P528"]}invalid_json_object
[{"id":"P528"},{"id":"I154"},{"id":"T972"},{"id":"E978"},{"id":"O324"}]43 tokens · 1.179sCorrect
{"ids":["P528"]}11 tokens · 0.306sCorrect
{"ids":["P528"]}11 tokens · 0.305sCorrect
{"ids":["P528"]}11 tokens · 0.310sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"Y498","units":12,"price":14},{"id":"N361","units":2,"price":16},{"id":"B407","units":15,"price":17},{"id":"F599","units":14,"price":3},{"id":"G422","units":18,"price":5}]Checked answer{"total": 587}wrong_schema_or_value
{"total": 100}9 tokens · 0.253swrong_schema_or_value
{"total":431}8 tokens · 0.227swrong_schema_or_value
{"total":432}8 tokens · 0.225swrong_schema_or_value
{"total":471}8 tokens · 0.224sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"M598","priority":4},{"id":"T597","priority":4},{"id":"E473","priority":1},{"id":"U978","priority":2},{"id":"J233","priority":1}]Checked answer{"ids": ["M598", "T597"]}wrong_schema_or_value
{"ids": ["M598", "T597", "E473", "U978", "J233"]}35 tokens · 0.960sCorrect
{"ids":["M598","T597"]}16 tokens · 0.442swrong_schema_or_value
{"ids":["T597","M598"]}16 tokens · 0.444sCorrect
{"ids":["M598","T597"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=T987. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"T987","city":"Perth","units":7},{"id":"G787","city":"Oslo","units":6},{"id":"Y276","city":"Accra","units":5},{"id":"D153","city":"Riga","units":17},{"id":"Y564","city":"Accra","units":1}]Checked answer{"city": "Perth", "units": 7}Correct
{"city": "Perth", "units": 7}14 tokens · 0.390sCorrect
{"city":"Perth","units":7}11 tokens · 0.311sCorrect
{"city":"Perth","units":7}11 tokens · 0.309sCorrect
{"city":"Perth","units":7}11 tokens · 0.305sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"X215","status":"pending"},{"id":"A743","status":"closed"},{"id":"I329","status":"closed"},{"id":"A901","status":"ready"},{"id":"N600","status":"pending"}]Checked answer{"ids": ["A901"]}wrong_schema_or_value
{"ids": ["A743", "A901", "I329", "N600", "X215"]}35 tokens · 0.959sCorrect
{"ids":["A901"]}11 tokens · 0.309sCorrect
{"ids":["A901"]}11 tokens · 0.309sCorrect
{"ids":["A901"]}11 tokens · 0.305sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"U599","units":6,"price":15},{"id":"V597","units":9,"price":7},{"id":"G303","units":4,"price":9},{"id":"P497","units":8,"price":7},{"id":"G404","units":12,"price":7}]Checked answer{"total": 329}wrong_schema_or_value
{
"total": 108
}12 tokens · 0.331swrong_schema_or_value
{"total":310}8 tokens · 0.228swrong_schema_or_value
{"total":342}8 tokens · 0.229swrong_schema_or_value
{"total":241}8 tokens · 0.223sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"N929","priority":6},{"id":"F236","priority":3},{"id":"F146","priority":9},{"id":"P238","priority":5},{"id":"N844","priority":4}]Checked answer{"ids": ["F146", "N929"]}invalid_json_object
[{"id":"N929","priority":6},{"id":"F236","priority":3},{"id":"F146","priority":9},{"id":"P238","priority":5},{"id":"N844","priority":4}]62 tokens · 1.694swrong_schema_or_value
{"ids":["F146","N844"]}16 tokens · 0.447sCorrect
{"ids":["F146","N929"]}16 tokens · 0.446swrong_schema_or_value
{"ids":["F146","N844"]}16 tokens · 0.439sUse only the records below. Return a single JSON object, without Markdown or commentary.
Find record Z000. Return its city and units as JSON with exactly keys "city" and "units". Use null for both if the record is absent.
Records:
[{"id":"I405","city":"Lima","units":15},{"id":"A276","city":"Oslo","units":5},{"id":"B480","city":"Perth","units":10},{"id":"K853","city":"Bern","units":11},{"id":"T371","city":"Oslo","units":17}]Checked answer{"city": null, "units": null}wrong_schema_or_value
{
"city": "Lima",
"units": 15
}19 tokens · 0.525sCorrect
{"city":null,"units":null}10 tokens · 0.283sCorrect
{"city":null,"units":null}10 tokens · 0.282sCorrect
{"city":null,"units":null}10 tokens · 0.277sUse only the records below. Return a single JSON object, without Markdown or commentary.
Select records whose status is exactly "ready". Return {"ids": [...]} with their ids in ascending alphabetical order.
Records:
[{"id":"R502","status":"ready"},{"id":"O411","status":"pending"},{"id":"M868","status":"pending"},{"id":"V659","status":"ready"},{"id":"B963","status":"closed"}]Checked answer{"ids": ["R502", "V659"]}wrong_schema_or_value
{"ids": ["O411", "M868", "R502", "V659", "B963"]}35 tokens · 0.956sCorrect
{"ids":["R502","V659"]}16 tokens · 0.440sCorrect
{"ids":["R502","V659"]}16 tokens · 0.445sCorrect
{"ids":["R502","V659"]}16 tokens · 0.439sUse only the records below. Return a single JSON object, without Markdown or commentary.
For each record multiply units by price, then add the products across all records. Return only {"total": integer}.
Records:
[{"id":"Y132","units":1,"price":16},{"id":"K147","units":5,"price":7},{"id":"T950","units":18,"price":13},{"id":"G567","units":12,"price":15},{"id":"Q798","units":16,"price":17}]Checked answer{"total": 737}invalid_json_object
{"total": 16 + 7 + 13 + 15 + 17 = 69}27 tokens · 0.743swrong_schema_or_value
{"total":471}8 tokens · 0.223swrong_schema_or_value
{"total":432}8 tokens · 0.224swrong_schema_or_value
{"total":471}8 tokens · 0.224sUse only the records below. Return a single JSON object, without Markdown or commentary.
Choose the two records with highest priority. Larger priority numbers rank first; break ties by ascending id. Return their ids in ranking order as {"ids": [...]}.
Records:
[{"id":"E972","priority":1},{"id":"B525","priority":3},{"id":"F536","priority":4},{"id":"J276","priority":3},{"id":"Y324","priority":4}]Checked answer{"ids": ["F536", "Y324"]}wrong_schema_or_value
{"ids": ["B525", "F536", "J276", "Y324", "E972"]}35 tokens · 0.960swrong_schema_or_value
{"ids":["F536","B525"]}16 tokens · 0.444sCorrect
{"ids":["F536","Y324"]}16 tokens · 0.447swrong_schema_or_value
{"ids":["F536","B525"]}16 tokens · 0.445sUse only the records below. Return a single JSON object, without Markdown or commentary.
Look up id K954 in the supplied records. Reply with {"city": city, "units": units} only; absent ids require two null values.
Records:
[{"id":"Y723","city":"Bern","units":0},{"id":"W107","city":"Lima","units":18},{"id":"O318","city":"Kyoto","units":4},{"id":"M719","city":"Riga","units":1},{"id":"K954","city":"Perth","units":0}]Checked answer{"city": "Perth", "units": 0}Correct
{"city":"Perth","units":0}11 tokens · 0.307sCorrect
{"city":"Perth","units":0}11 tokens · 0.308sCorrect
{"city":"Perth","units":0}11 tokens · 0.312sCorrect
{"city":"Perth","units":0}11 tokens · 0.311sUse only the records below. Return a single JSON object, without Markdown or commentary.
Reply with one JSON key "ids": the alphabetically sorted ids of all ready records. Exclude pending and closed records.
Records:
[{"id":"R114","status":"pending"},{"id":"C431","status":"pending"},{"id":"V835","status":"pending"},{"id":"H469","status":"pending"},{"id":"E964","status":"ready"}]Checked answer{"ids": ["E964"]}wrong_schema_or_value
{
"ids": ["E964", "H469", "R114", "V835", "C431"]
}38 tokens · 1.044sCorrect
{"ids":["E964"]}11 tokens · 0.311sCorrect
{"ids":["E964"]}11 tokens · 0.312sCorrect
{"ids":["E964"]}11 tokens · 0.306sUse only the records below. Return a single JSON object, without Markdown or commentary.
Calculate the total invoice value: sum(units * price) for every entry, irrespective of status. Answer as JSON with exactly one integer key "total".
Records:
[{"id":"B844","units":8,"price":10},{"id":"B439","units":8,"price":11},{"id":"T402","units":12,"price":15},{"id":"G494","units":3,"price":14},{"id":"P260","units":7,"price":11}]Checked answer{"total": 467}wrong_schema_or_value
{"total": 100}9 tokens · 0.252swrong_schema_or_value
{"total":341}8 tokens · 0.227swrong_schema_or_value
{"total":342}8 tokens · 0.229swrong_schema_or_value
{"total":341}8 tokens · 0.227sUse only the records below. Return a single JSON object, without Markdown or commentary.
Rank rows by descending priority, then ascending identifier for equal priorities. Output the first two ids under the JSON key "ids".
Records:
[{"id":"F270","priority":3},{"id":"I528","priority":7},{"id":"L682","priority":3},{"id":"E942","priority":3},{"id":"B496","priority":5}]Checked answer{"ids": ["I528", "B496"]}invalid_json_object
[{"id":"F270","priority":3},{"id":"L682","priority":3},{"id":"E942","priority":3},{"id":"B496","priority":5}]50 tokens · 1.371sCorrect
{"ids":["I528","B496"]}16 tokens · 0.447sCorrect
{"ids":["I528","B496"]}16 tokens · 0.450sCorrect
{"ids":["I528","B496"]}16 tokens · 0.441sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which city and unit count belong to M688? Produce only a JSON object with keys "city" and "units"; both must be null if no id matches.
Records:
[{"id":"Y501","city":"Perth","units":1},{"id":"X352","city":"Kyoto","units":12},{"id":"U279","city":"Oslo","units":13},{"id":"M688","city":"Perth","units":8},{"id":"G212","city":"Suva","units":1}]Checked answer{"city": "Perth", "units": 8}wrong_schema_or_value
{
"city": null,
"units": null
}15 tokens · 0.416sCorrect
{"city":"Perth","units":8}11 tokens · 0.307sCorrect
{"city":"Perth","units":8}11 tokens · 0.309sCorrect
{"city":"Perth","units":8}11 tokens · 0.305sUse only the records below. Return a single JSON object, without Markdown or commentary.
Keep only entries marked ready, then alphabetize their identifiers. Output a JSON object containing only the array "ids".
Records:
[{"id":"S900","status":"pending"},{"id":"S359","status":"ready"},{"id":"I490","status":"closed"},{"id":"D654","status":"pending"},{"id":"Z860","status":"pending"}]Checked answer{"ids": ["S359"]}invalid_json_object
[{"id":"S359","status":"ready"},{"id":"I490","status":"closed"},{"id":"D654","status":"pending"},{"id":"Z860","status":"pending"}]51 tokens · 1.396sCorrect
{"ids":["S359"]}11 tokens · 0.310swrong_schema_or_value
{"ids":["D654","S359"]}16 tokens · 0.447sCorrect
{"ids":["S359"]}11 tokens · 0.306sUse only the records below. Return a single JSON object, without Markdown or commentary.
What is the combined value of all entries? Each contributes its unit count times its price. Output {"total": integer} without explanation.
Records:
[{"id":"I134","units":3,"price":11},{"id":"W560","units":15,"price":15},{"id":"U101","units":2,"price":8},{"id":"X430","units":8,"price":10},{"id":"I360","units":14,"price":17}]Checked answer{"total": 592}wrong_schema_or_value
{"total": 120}9 tokens · 0.251swrong_schema_or_value
{"total":331}8 tokens · 0.228swrong_schema_or_value
{"total":432}8 tokens · 0.225swrong_schema_or_value
{"total":431}8 tokens · 0.225sUse only the records below. Return a single JSON object, without Markdown or commentary.
Which two identifiers lead when priority is sorted highest first and tied ids alphabetically? Return {"ids": [...]} in that order.
Records:
[{"id":"A213","priority":5},{"id":"T122","priority":7},{"id":"E628","priority":9},{"id":"J981","priority":7},{"id":"B341","priority":5}]Checked answer{"ids": ["E628", "J981"]}wrong_schema_or_value
{"ids": ["A213", "T122", "E628", "J981", "B341"]}35 tokens · 0.956swrong_schema_or_value
{"ids":["E628","T122"]}16 tokens · 0.449sCorrect
{"ids":["E628","J981"]}16 tokens · 0.448swrong_schema_or_value
{"ids":["E628","T122"]}16 tokens · 0.442sUse only the records below. Return a single JSON object, without Markdown or commentary.
Extract city and units for id=H187. Required output schema: {"city": string or null, "units": integer or null}. If missing, return null in both fields.
Records:
[{"id":"H187","city":"Bern","units":1},{"id":"T805","city":"Lima","units":16},{"id":"N350","city":"Lima","units":8},{"id":"X625","city":"Perth","units":13},{"id":"Z263","city":"Kyoto","units":8}]Checked answer{"city": "Bern", "units": 1}Correct
{"city": "Bern", "units": 1}13 tokens · 0.360sCorrect
{"city":"Bern","units":1}10 tokens · 0.284sCorrect
{"city":"Bern","units":1}10 tokens · 0.279sCorrect
{"city":"Bern","units":1}10 tokens · 0.283sUse only the records below. Return a single JSON object, without Markdown or commentary.
List every ready id, sorted lexicographically ascending. The answer must have the shape {"ids": [...]}.
Records:
[{"id":"M560","status":"pending"},{"id":"L509","status":"pending"},{"id":"R446","status":"pending"},{"id":"M596","status":"ready"},{"id":"N314","status":"ready"}]Checked answer{"ids": ["M596", "N314"]}wrong_schema_or_value
{"ids": ["M560", "L509", "R446", "M596", "N314"]}35 tokens · 0.960swrong_schema_or_value
{"ids":["N314","M596"]}16 tokens · 0.447swrong_schema_or_value
{"ids":["N314","M596"]}16 tokens · 0.440swrong_schema_or_value
{"ids":["N314","M596"]}16 tokens · 0.447sUse only the records below. Return a single JSON object, without Markdown or commentary.
Compute the sum of quantity-times-price using units as quantity for all listed rows. Use only the JSON field "total", with an integer value.
Records:
[{"id":"U270","units":7,"price":4},{"id":"S189","units":18,"price":7},{"id":"H792","units":19,"price":19},{"id":"T290","units":13,"price":19},{"id":"J625","units":2,"price":15}]Checked answer{"total": 792}wrong_schema_or_value
{
"total": 100
}12 tokens · 0.332swrong_schema_or_value
{"total":471}8 tokens · 0.228swrong_schema_or_value
{"total":342}8 tokens · 0.222swrong_schema_or_value
{"total":531}8 tokens · 0.229sUse only the records below. Return a single JSON object, without Markdown or commentary.
Take the top two records using priority descending and id ascending as the tie-break. Answer with only a JSON object containing their ordered "ids".
Records:
[{"id":"L799","priority":6},{"id":"G601","priority":2},{"id":"Z184","priority":3},{"id":"V411","priority":6},{"id":"D563","priority":8}]Checked answer{"ids": ["D563", "L799"]}invalid_json_object
[{"id":"L799"},{"id":"G601"},{"id":"Z184"},{"id":"V411"},{"id":"D563"}]43 tokens · 1.172sCorrect
{"ids":["D563","L799"]}16 tokens · 0.440sCorrect
{"ids":["D563","L799"]}16 tokens · 0.440sCorrect
{"ids":["D563","L799"]}16 tokens · 0.445sLZ · Apache-2.0 · Full raw evaluations, predetermined final candidates, no serving promotion.