{"data":{"kind":"file","path":"README.md","version_id":"wltjd6ju0fj0dlgo6qcmrz2g","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":17971,"modified_at":"2026-11-05T14:47:18.538000","content_hash":"8f4e5daf804a69193ed81bf5890b57795bd2cbdc6783cd51d3ab8487dfead777"},"entries":[],"content":"# paie-fr — French monthly payslip calculation (2026)\n\n**Language of tasks:** French. **Type:** single-turn, answer-only (no tools, no sandbox).\n\n## What it trains\n\nThe model acts as a French payroll clerk. Each task describes a fictitious employee and\nemployer (status cadre / non-cadre, base salary, overtime at +25 % / +50 %, bonuses,\nunpaid absence, benefit in kind, mandatory health insurance, meal vouchers, headcount\nbracket, work-accident rate, transport levy rate, personal withholding-tax rate) and gives\n**the full 2026 contribution schedule and the calculation rules in the prompt**. The model\nmust produce six amounts, exact to the cent:\n\n| key | meaning |\n|---|---|\n| `brut` | gross salary |\n| `total_cotisations_salariales` | total employee contributions (incl. CSG/CRDS, health insurance share, minus overtime reduction) |\n| `net_imposable` | taxable net |\n| `prelevement_source` | income-tax withholding |\n| `net_a_payer` | net pay |\n| `cout_employeur` | total employer cost |\n\nThis exercises long, rule-following multi-step arithmetic: ~20 contribution lines on\ndifferent bases (total gross, tranche 1 / tranche 2 around the social-security ceiling,\ngross capped at 4 PMSS), conditional lines (cadres only, headcount thresholds), CSG/CRDS\non 98.25 % of gross plus employer health/death-cover contributions, deductible vs.\nnon-deductible splits, overtime-specific rules, (level 3) the 2026 degressive\nemployer-contribution relief (RGDU) with a fractional exponent, and (level 4) the\n\"incomplete month\" workflow of real payroll: hiring or leaving mid-month, part-time\ncontracts, prorated social-security ceiling and RGDU reference SMIC, and the annual cap on\nthe income-tax exemption of overtime. One slip anywhere breaks the result.\n\n## Tasks and difficulty\n\nTasks are generated deterministically from a seed (`graine`). Every task has a\n`difficulte` field.\n\n| level | content |\n|---|---|\n| 1 | non-cadre below the ceiling, base salary + optional bonus; schedule reduced to the applicable lines; RGDU coefficient given |\n| 2 | cadre or non-cadre up to ~4 700 €; overtime, health insurance, meal vouchers possible; full schedule with conditions column; RGDU coefficient given |\n| 3 | up to ~7 500 € (tranche 2, CET); + unpaid absence, benefit in kind; RGDU coefficient must be computed from the written formula |\n| 4 | level 3 rules + incomplete month and/or part-time (see below); no unpaid absence |\n\nConfig (`--env.taskset.*`, or `--taskset.*` for `vf-validate`): `nb_taches` (default 400),\n`graine` (default 0), `difficulte` (0 = mix, or 1, 2, 3, 4), `task.tout_ou_rien`\n(default false). Any number of tasks can be generated; the default 400 = 100 per level.\n\n**Mix order.** In mixed mode (`difficulte = 0`) levels follow the cycle **1, 4, 2, 3**, so\ntask `i` has level `(1, 4, 2, 3)[i % 4]` and id `paie-<level>-<i>`. The first 6 tasks (the\nones a `-n 6` calibration run sees) are `paie-1-0`, `paie-4-1`, `paie-2-2`, `paie-3-3`,\n`paie-1-4`, `paie-4-5`: every level, two of level 1 (entry rung for small models) and two\nof level 4 (headroom for strong models). Levels 1 to 3 are generated exactly as before\nlevel 4 existed (a test pins a hash of their prompts and answers); in mixed mode the\n*sequence* of tasks differs from the previous 1/2/3 cycle because the seed's random stream\nis shared.\n\n### Level 4 (added after a first calibration where the strongest model solved 6/6 tasks of levels 1-3)\n\nEvery rule is written in the prompt; the fiche gives the contract hours, the entry or exit\ndate and, with overtime, the overtime pay already tax-exempted since January. Profiles\n(drawn per task): full-time entry or exit mid-month (40 %, half of them with overtime, then\nalways an exit), part-time over the whole month (30 %), part-time with entry or exit\n(10 %), full-time whole month with overtime (20 %). Cadres 50 %.\n\n- **Base salary of the month** (entry / exit): monthly base × calendar days of presence /\n  calendar days of the month, computed in one go and rounded once to the cent (the daily\n  rate is not rounded; \"calendar days\" method). Days of\n  presence count the entry day and the last day of the contract; the prompt states the\n  number of days of the month. Bonuses, benefits in kind, health insurance and meal\n  vouchers are not prorated.\n- **Prorated ceiling**: `plafond = 4 005 € × (contract hours / 151.67) × (days of presence /\n  days of the month)`, computed in one go then rounded to the cent (ratios not rounded).\n  The contract hours are the h/month value printed in the fiche, used as is (the prompt\n  says so and repeats the value; e.g. 86.67, not 20/35 nor an unrounded 20 × 52 / 12).\n  Tranche 1, tranche 2, the \"gross capped at 4 × ceiling of the month\" base (unemployment,\n  AGS, APEC; labelled that way in the level-4 rate table, \"4 PMSS\" in levels 1-3) and the\n  CET base all use the rounded ceiling, so a part-time cadre often has a tranche 2 while\n  earning less than the full PMSS (≈ 29 % of level-4 tasks; tranche 2 overall ≈ 64 %).\n- **RGDU reference SMIC**: `1 823.03 € × (contract hours / 151.67) × (days of presence /\n  days of the month)` with the same hours and days as the ceiling, computed in one go and\n  rounded to the cent; coefficient formula as in level 3.\n- **Overtime**: always both +25 % and +50 % hours (full-time only), August to December. The\n  income-tax exemption is capped at 7 500 € per year: exempt part of the month =\n  min(overtime pay ; 7 500 € − overtime pay already exempted, given), never negative; the\n  rest stays in taxable net. CSG/CRDS on overtime and the employee-contribution reduction\n  still apply to the whole overtime pay. Generated cases: cap reached this month (≈ 55 %),\n  already reached (≈ 20 %), not reached (≈ 25 %).\n- Part-time hours: 20, 24, 28, 30 or 32 h/week = 86.67, 104.00, 121.33, 130.00 or\n  138.67 h/month (× 52 / 12).\n- Tasks where the exact prorated ceiling, reference SMIC or prorated base lies within\n  1e-9 of a half-cent (or the RGDU coefficient within 1e-6 of a rounding boundary) are\n  discarded, so float and decimal arithmetic agree.\n\nLevel-4 prompts are about 10k characters. None of the prorated amounts (ceiling,\nreference SMIC, base of the month, exempt overtime) is written in the prompt (tested).\n\n### Rounding rules (made explicit after calibration)\n\nRule 1 of every prompt: each amount (a contribution = base × rate, a pay item, a share) is\ncomputed in one go and rounded **once** to the cent (half-up); intermediate results of the\nsame computation (overtime hourly rate × 1.25, daily salary, hours or days ratios) are never\nrounded unless a rule says so explicitly; totals are sums of already-rounded amounts. Each\nrule restates its own rounding: overtime = hours × hourly rate × 1.25 (or × 1.5), rounded\nonce (with a fictitious worked example: 3.5 h × 15.47 € × 1.25 = 67.68125 → 67.68 €, not\n3.5 × 19.34 = 67.69 €); health-insurance and meal-voucher employer shares rounded to the\ncent (meal vouchers on the total value, not voucher by voucher); overtime reduction on the\nsum of the two rounded overtime amounts; RGDU = gross × rounded coefficient, rounded;\nwithholding tax rounded. Calibration showed the strong model reading the old\nwording (\"one amount rounded per overtime rate\") as \"round the increased hourly rate first\";\nthat reading changed the overtime pay in ≈ 40 % of level-2/3 tasks and ≈ 29 % of level-4\ntasks (79, 83 and 58 tasks out of 200 per level, seed 11). Rounding the daily salary first\nwould have changed 84/200 level-4 tasks; meal vouchers voucher by voucher, 5-9/200. The\nprompt now rules out each of these readings explicitly (tested: every affected task contains\nthe sentence, and the alternative answer scores 0). These were wording changes only: the\nreference amounts of all tasks are unchanged (fingerprint test over 1 600 tasks).\n\n## Data\n\nAll employees and companies are **fictitious** (random combinations of common first\nnames, surnames and job titles; no real person, no personal data). Rates and thresholds\ncome from public 2026 regulatory sources (see \"Sources and licences\"; service-public.fr\ncontent is reused under the Licence Ouverte / Etalab 2.0). No dataset is downloaded.\n\n**Code licence:** MIT (see `LICENSE`).\n\n## Verifier\n\nAll scoring lives in `paie_fr/scoring.py` (pure functions, no `verifiers` import); the\nreference calculation lives in `paie_fr/calcul.py` (Python `Decimal`, every line rounded\nto the cent with half-up rounding, exactly as written in the prompt).\n\n**Extraction (strict):** the reply must contain exactly one `<bulletin>…</bulletin>` tag\n(case-insensitive; an optional ```json fence inside is tolerated) whose content is a single\nJSON object with exactly the six keys. Values may be JSON numbers or strings holding one\namount (`\"1 234,56 €\"` is normalised); at most two decimals. Rejected (score 0): no tag,\nseveral tags or opening tags, invalid JSON, duplicate / missing / extra keys, lists,\nbooleans, null, NaN/Infinity, expressions, more than two decimals, absurd amounts\n(|value| > 10^9 €, including out-of-range exponents such as `1e1000000000000000000`).\nAny unexpected exception while reading the reply also scores 0 (the reward never crashes);\nthe gold solution is checked without this safety net, so a broken reference fails loudly.\n\n**Reward:** exact comparison to the cent, no tolerance (every rounding rule is written in\nthe prompt, and generated tasks whose RGDU coefficient lies within 1e-6 of a rounding\nboundary are discarded).\n- all six amounts correct → **1.0**;\n- otherwise, if `brut` is correct → partial credit = 0.5 × (0.35 net à payer +\n  0.25 coût employeur + 0.15 cotisations salariales + 0.15 net imposable +\n  0.10 prélèvement à la source [only if net imposable is also correct]);\n  so any wrong answer scores **at most 0.45**, a wrong net pay at most 0.325;\n- wrong `brut` → 0 (the gross is often copyable from the prompt, so it earns nothing by\n  itself and gates everything else);\n- `task.tout_ou_rien=true` → 0 unless all six amounts are correct.\n\nThe expected amounts are stored in `TaskData.attendu` and never appear in the prompt (a\ntest checks that none of the computed amounts, in any common format, occurs in any\nprompt). Each task's `validate()` checks that the reference answer scores 1.0.\n\n## Sources and licences\n\nOnly regulatory figures (rates, ceilings, formula parameters set by laws, decrees and\norders) are used; no text is copied. Full list with values: `sources/paie-fr-bareme-2026.md`\nat the repo root.\n\n- service-public.fr (entreprendre), F24542 \"Réduction générale dégressive unique\" —\n  Licence Ouverte / Etalab 2.0 — RGDU formula and parameters (Tmin 0.02, Tdelta 0.3781 /\n  0.3821, P = 1.75, 3 × SMIC, SMIC frozen at its 1 Jan 2026 value 1 823.03 €).\n- Urssaf \"Taux de cotisations secteur privé\" (official reference) — cross-checked through:\n- CCI Paris Île-de-France, \"Les charges sociales au 1er janvier 2026\" (full 2026 table,\n  PMSS 4 005 €, SMIC 1 823.03 €).\n- LégiSocial, \"Taux de cotisations sociales Urssaf 2026\".\n- info.gouv.fr, SMIC revaluation of 1 June 2026 (12.31 €/h) — not used in the RGDU formula.\n\nLevel 4 (regulatory rules restated in the prompt, no text copied):\n- Prorating the social-security ceiling (part-time: PMSS × contract hours / legal hours;\n  entry or exit: × calendar days of the employment period / calendar days of the month;\n  both in turn when combined): net-entreprises.fr DSN documentation\n  (`dsn-rubrique-unite-de-mesure-valeur40.pdf`); LégiSocial, \"Temps partiel et plafond de\n  sécurité sociale : informations du BOSS\" and \"Proratisation du plafond de sécurité\n  sociale\"; Éditions Tissot (double proration, part-time with entry/exit).\n- Salary of an incomplete month, calendar-days method: Éditions Tissot, \"Le calcul du\n  salaire en cas de mois incomplet\".\n- RGDU reference SMIC prorated for part-time and incomplete months: service-public.fr\n  F24542 (Licence Ouverte 2.0).\n- Overtime income-tax exemption capped at 7 500 € per year (art. 81 quater CGI):\n  justice.fr page reproducing Service-Public content; LégiSocial; Assemblée nationale\n  written answer.\nLinks and values: `sources/paie-fr-bareme-2026.md`.\n\n## Simplifications (the prompt states its own rules; they are authoritative)\n\n- RGDU reference SMIC is **not increased by the overtime hours paid** (real rule: the\n  reference SMIC is SMIC × hours paid, overtime and extra hours counted without their\n  premium). The prompt gives the formula without this increase and states that it is\n  authoritative, so the reference amounts follow the prompt. Tasks concerned (overtime\n  and a non-zero RGDU), measured with seed 0: 75 of 200 level-3 tasks and 48 of 200\n  level-4 tasks.\n- RGDU computed on the month (real rule: annualised, with monthly progressive\n  regularisation); reference SMIC prorated for unpaid absence hours (levels 1-3) or by the\n  same hours × days fraction as the ceiling (level 4; real rule: contract hours over the\n  employment period).\n- Overtime: employee-contribution reduction applied as a flat 11.31 % of overtime pay\n  (real rule: capped at the old-age contributions actually due); the whole gross overtime\n  pay is removed from taxable net (real rule: net amount, 7 500 €/year cap). Levels 1-3:\n  cap assumed not reached. Level 4: cap applied to the gross overtime pay (real rule: net\n  taxable amount), and all CSG/CRDS on overtime stays non-deductible even for the taxable\n  part (real rule: the taxable part bears deductible CSG).\n- Ceiling: levels 1-3 never prorated (full-time, whole month); level 4 prorated in one\n  computation rounded once to the cent (real payroll software may round differently; some\n  references describe a \"thirtieths\" variant — the prompt fixes the calendar-days method).\n- Level 4: no \"heures complémentaires\" for part-timers (part-time tasks have no extra\n  hours); the employer's option to keep the full ceiling for part-timers is not used;\n  entry and exit never in the same month; overtime never on an entry month (a new employer\n  does not know the cumulative exempt amount).\n- Health insurance: flat monthly premium; employee share fully deductible; employer share\n  added to taxable net. Death cover for cadres: 1.50 % employer-only on tranche 1, not\n  added to taxable net. Forfait social 8 % on both for ≥ 11 employees.\n- Meal vouchers: employer share assumed within the exemption limit (at most 7.20 € per\n  voucher in generated tasks).\n- Known limit (level 4): about 6 % of level-4 tasks (70 of 1 200 measured, seeds 0, 11,\n  5) are full-time, whole-month tasks whose overtime stays under the\n  remaining 7 500 € cap, so no level-4-specific rule changes the result (the solver must\n  still check the cap). `paie-4-5`, one of the six calibration tasks, is such a task.\n  Forcing every level-4 task to trigger a level-4 rule would change the generated tasks\n  (shared random stream) and their reference amounts, so it was not done.\n- Known limit (level 4): a few part-time tasks pay an hourly base rate below the SMIC\n  (about 0.3 % to 1 % of level-4 tasks, e.g. 12.01 €/h in June while the SMIC is\n  12.31 €/h from 1 June 2026). This is unrealistic but does not affect correctness (the\n  prompt gives the amounts). Generation will be fixed in v0.2 (changing it now would change\n  the tasks and their reference amounts).\n- Not covered: part-time extra hours (heures complémentaires), paid leave and sick leave, Alsace-Moselle, apprentices,\n  collective-agreement specifics, PEEC/apprenticeship-tax annual timing, taxe sur les\n  salaires, executives above 8 PMSS.\n\n## Running\n\n```bash\nuv pip install -e environments/paie-fr\nuv run pytest environments/paie-fr/tests\nuv run vf-validate paie-fr -n 10 --runtime.type subprocess   # gold must be 100 % valid, noop 100 % valid (reward 0)\nuv run vf-run paie-fr -m <model> -n 2 -r 1 -c 1 \\\n  --env.agent.harness.id null --env.agent.runtime.type subprocess \\\n  --client.base-url https://openrouter.ai/api/v1 --client.api-key-var OPENROUTER_API_KEY \\\n  -o outputs/essai/paie-fr\n# one level only: add --env.taskset.difficulte 4   (vf-validate: --taskset.difficulte 4)\nuv run vf-validate paie-fr -n 10 --runtime.type subprocess --taskset.difficulte 4\n```\n\nPrompts are about 3.9k characters (level 1), 7.6k (level 2), 8.1k (level 3) and 10k\n(level 4); models\nneed room to reason, so allow a generous completion budget.\n\n## Calibration\n\n**Preliminary, small sample.** Treat these numbers as a first signal only (uncertainty of\nroughly ±30 points per level). Free models via OpenRouter, harness `null` (answer only),\nbinary success = reward 1.0.\n\nRun A (levels 1-4, the first 6 tasks of seed 0: `paie-1-0`, `paie-4-1`, `paie-2-2`,\n`paie-3-3`, `paie-1-4`, `paie-4-5`; 18 episodes):\n\n| model | episodes | success | mean reward | level 1 | level 2 | level 3 | level 4 |\n|---|---|---|---|---|---|---|---|\n| nvidia/nemotron-3-ultra-550b-a55b (2 attempts per task) | 12 | 9/12 (75 %) | 0.792 | 4/4 | 2/2 | 0/2 | 3/4 |\n| nvidia/nemotron-3-nano-omni-30b-a3b-reasoning (1 attempt) | 6 | 2/6 (33 %) | 0.354 | 2/2 | 0/1 | 0/1 | 0/2 |\n\nRun B (earlier, levels 1-3 only, before level 4 existed; 6 tasks, 1 attempt; 17 episodes,\none lost to provider rate limits): liquid/lfm-2.5-2.6b 0/5, nvidia/nemotron-3-nano-omni-30b\n2/6 (level 1 only), nvidia/nemotron-3-ultra-550b 6/6.\n\nOrdering strong > medium > small is clear; per-level rates are not reliable. Both runs\npredate the rewording of the rounding rules (see \"Rounding rules\"); that rewording changed\nno reference amount (the expected amounts of the 6 calibration tasks are identical), but\nthe current prompt has not been re-calibrated. One of the strong model's failures came\nfrom the ambiguity that the rewording removed. A larger calibration (20-30 tasks, 3\nmodels, 2 attempts) is planned.\n\n## Contact\n\nQuestions, custom task sets (other French payroll cases, collective agreements, more volume) or licensing: **veltris.buisness@gmail.com**\n","encoding":"utf-8","truncated":false,"total_bytes":17971},"status":null}