{"data":{"kind":"file","path":"README.md","version_id":"s95v7lrna7j6idxeied10u23","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":4300,"modified_at":"2026-10-28T05:20:17.780000","content_hash":"7480fe4aa5d22ac2d2fde4a5c00cef2ceac15d64d948d1de7bded14cacb72bf9"},"entries":[],"content":"# us-sut-reconciliation-v1\n\nA US **state Sales & Use Tax (SUT)** analogue of `au-bas-reconciliation-v1`, built on the v1\n`verifiers` stack (`import verifiers.v1 as vf`). Each task is a synthetic quarterly ledger with\ninjected bookkeeping faults. The model prepares the quarterly SUT return and audits every row.\n\n## Concept mapping (GST/BAS → US SUT)\n\n| Australian GST / BAS | US equivalent | Implemented as |\n|---|---|---|\n| BAS filing instrument | Quarterly state sales & use tax return | One ledger per task, random `year`/`Q1–Q4` filing period |\n| G1 total sales | Gross sales | `gross_sales`: sum of included sale lines |\n| GST-free / input-taxed supplies | Exempt sales | `exempt_sales`: exempt categories + certified resales |\n| 1A GST on sales | Sales tax due | `sales_tax_due`: Σ round(gross × destination combined rate) |\n| 1B GST on purchases | Vendor use tax | `use_tax_due`: untaxed taxable purchases × business-ZIP rate |\n| Net (1A − 1B) | Net remittance | `net_remittance = sales_tax_due + use_tax_due` |\n| ABN threshold ($82.50) | Resale/exemption certificate threshold | Exemption claims above **$200** need a cert ID |\n\nThe jurisdiction is an **illustrative** single-state rulebook (\"State of Lone Pine\": 6.25% state\nrate plus a ZIP-level local rate). Exempt categories are `unprepared_grocery`, `prescription_drug`,\n`medical_device` and `consulting_service`. The full rate table and rules go in every prompt, so the\ntask is closed-book and needs no statutory knowledge. The rates are stylized and are not current law.\n\n## Fault taxonomy\n\n| Code | Rule | Deterministic condition |\n|---|---|---|\n| `tax_on_exempt_item` | Tax collected on an exempt good/service | Exempt-category sale with recorded tax > 0 |\n| `tax_miscalculated` | Tax ≠ base × jurisdiction rate | Recorded ≠ correct on a taxable sale. Generated as a state-only rate (local omitted), an internal extraction `g·r/(1+r)`, or a keying error |\n| `duplicate_invoice` | Repeated invoice number | Second (later) occurrence of an `invoice_number`, excluded from the return |\n| `outside_period` | Dated outside the filing quarter | Excluded from the return |\n| `missing_exemption_cert` | Exempt/resale claim without a certificate | `tax_exempt_claimed` on a taxable category, gross > $200, empty cert ID. The row is treated as taxable (sales) or use-taxable (purchases) |\n| `nexus_misallocation` | Origin vs. destination sourcing | Recorded tax = gross × **origin** (business ZIP) rate instead of the delivery-ZIP rate |\n\nGround truth comes from `ledger.reconcile()`, using `Decimal` with half-up per-line rounding.\n\n## Schemas\n\nTask input: the rulebook plus a CSV ledger with the columns `transaction_id, invoice_number, date,\ntype (sale|purchase), category, gross_amount, tax_collected_or_paid, jurisdiction_zip,\ntax_exempt_claimed, customer_or_vendor_cert_id`.\n\nModel output: a final `<answer>` block holding JSON:\n\n```json\n{\"gross_sales\": 128450.00, \"exempt_sales\": 24100.00, \"taxable_sales\": 104350.00,\n \"sales_tax_due\": 8608.88, \"use_tax_due\": 412.50, \"net_remittance\": 9021.38,\n \"faults\": {\"txn_014\": [\"tax_on_exempt_item\"], \"txn_038\": [\"missing_exemption_cert\"]}}\n```\n\n## Rewards\n\n| Reward | Weight | Definition |\n|---|---|---|\n| `return_correct` | 0.6 | 1.0 iff `gross_sales`, `taxable_sales`, `sales_tax_due`, `use_tax_due`, `net_remittance` all match to the cent |\n| `fault_audit` | 0.4 | Micro-F1 over `(transaction_id, fault_code)` pairs (1.0 when both sets are empty) |\n\nMetrics (unweighted): `answer_parsed`, `fields_correct` (fraction of the 5 figures), `fault_precision`, `fault_recall`.\n\n## Config\n\n`--env.taskset.num-tasks` (default 50), `--env.taskset.num-rows` (30), `--env.taskset.fault-rate` (0.25),\n`--env.taskset.seed` (20260701; task *i* uses seed + *i*).\n\n## Run\n\n```bash\nuv pip install -e environments/us_sut_reconciliation_v1\nuv run vf-eval us-sut-reconciliation-v1 -m <model> -n 30 --env.agent.harness.id null \\\n  --env.agent.runtime.type subprocess --client.base-url <openai-compatible-url> --client.api-key-var <VAR>\npython scripts/summarize.py outputs/<run-dir> ...\n```\n\n`--env.agent.harness.id null` makes it a plain chat task (no tools). `--env.agent.runtime.type subprocess`\nruns locally instead of in a Prime sandbox. Results are in [`results/`](../../results/README.md).\n","encoding":"utf-8","truncated":false,"total_bytes":4300},"status":null}