vLLM vs HF batch 32
470 s → 64.7 s for 3,105 prompts. Generation only, two repeats. HF batch 8: 1,091 s, or 16.9×.
Research companion · November 2026
What supervised fine-tuning changes in LLM CV screening, what further training adds, and what changes inside the model.
Qwen3.5-9B · One shared SFT checkpoint · Three seeds per added stage
| Model | Female she / her | Male he / him |
|---|---|---|
| Base | Hire | Reject |
| SFT | Hire | Hire |
Across all 20 gender-cue variants: Base hires 15; SFT hires all 20.
One CV–job pair, greedy decoding. Consistency alone does not establish correctness.Question & protocol
Each CV is paired with three jobs. We vary military status, religion or gender through explicit or implicit cues. A set holds the CV, job, attribute family and cue style fixed. It is unstable when it contains both hire and reject decisions.
We compare methods at a matched hire rate by thresholding the hire/reject logit margin. Paired 95% confidence intervals resample candidates, preserving the related sets within each CV.
This development benchmark informed which methods to continue. The results are exploratory. GPT-4o agreement is a reference proxy, not independently labelled hiring quality.
Behavioural results
Base → SFT: −9.95 percentage points
Paired 95% CI [−12.07, −7.83] pp
At each model’s own greedy hire rate.
This comparison uses a different rate convention.
On the frozen RJDB evaluation, unstable sets fall from 24.9% to 2.2% at a matched 36.9% hire rate. The paired change is −22.75 pp, 95% CI [−24.86, −20.61]. Qualification preference remains high: 96.3% → 95.7%, change −0.67 pp [−3.00, +1.67].
Qualification preference is the share of triples where the matched CV has a higher hire/reject margin than its weakened counterpart on the cue-free prompt. This is a controlled preference measure, not hiring accuracy.
| Added stage / seed | Consistency gain vs SFT, pp [95% CI] | Qualification preference change, pp [95% CI] |
|---|---|---|
| DPO s42 | +0.39 [−0.56, +1.31] | −4.00 [−7.00, −1.00] |
| DPO s43 | −0.33 [−1.36, +0.58] | −4.67 [−8.00, −1.33] |
| DPO s44 | +0.06 [−0.94, +1.01] | −1.67 [−4.00, +0.67] |
| GRPO + decay s42 | −0.16 [−1.11, +0.83] | −2.00 [−4.00, −0.67] |
| GRPO + decay s43 | +0.08 [−0.87, +1.03] | +0.00 [−1.67, +1.67] |
| GRPO + decay s44 | −0.65 [−1.80, +0.44] | −4.00 [−6.67, −2.00] |
| Self-distillation s42 | +0.23 [−0.73, +1.15] | +1.33 [+0.33, +2.67] |
| Self-distillation s43 | +0.65 [−0.19, +1.56] | +0.67 [−0.67, +2.33] |
| Self-distillation s44 | +0.70 [−0.21, +1.60] | +0.67 [−1.00, +2.33] |
Consistency gain = SFT instability − model instability, so positive is better. All nine consistency intervals include zero at the common 36.9% rate. Intervals are pointwise paired candidate-bootstrap intervals, not adjusted across runs. Four of six DPO/GRPO preference intervals lie below zero; this is a seed-level finding.
The 300 CVs and jobs were unseen, from a source represented in training. The weakened CVs are GPT-4 rewrites and differ in length. The preference interval does not establish equivalence. Checkpoints were fixed before this evaluation; ten jobs used 11.4 allocated GPU-hours.
Instability estimates and intervals (CSV) · Qualification preference (CSV) · Registered protocol, results and caveats ↗
| Comparison | Development, 150 CVs | Held-out, 300 CVs |
|---|---|---|
| DPO consistency gain at 35% hire, three-seed range | −4.52 to −2.50 pp | +0.23 to +1.06 pp |
| DPO consistency gain over 30–35%, three-seed band-mean range | −2.55 to −1.31 pp | +0.09 to +0.49 pp |
| Runs with gain CI above zero: own rate / common point | 6 of 36 / 1 of 36 | 0 of 9 / 0 of 9 |
The range rows compare the same three DPO checkpoints. The count row describes different sets of evaluated runs and is not a matched-cohort comparison. Common points differ between datasets: 22.51% and 36.88%. Band means and curves are descriptive.
Held-out curves (CSV) · Held-out band means (CSV) · Development band means (CSV) · All development runs (CSV)
Move the slider to inspect descriptive three-seed means on the measured 0.5-point grid. The high-rate DPO drop below did not replicate on the held-out CVs.
Change in unstable sets, percentage points. Lower is better. Thin seed trajectories, when shown, are not confidence bands. Curves are descriptive; paired intervals are reported below at 22.51%.
Mean changes range from −0.285 to −0.165 pp. These are descriptive seed means, without an aggregate three-seed CI.
| Added stage | Seed | Change vs SFT, pp | Paired 95% CI, pp |
|---|
Residual instability
For seeds 43/44, unstable-set overlap is much lower than a control that holds SFT scores fixed and recalibrates its threshold to each trained model’s hire rate.
Other seed pairs have different rate patterns. The complete table below includes them, so this comparison is not presented as a universal result across seed pairs.
Jaccard measures the intersection divided by the union of unstable sets. Sets are not distinct people: one CV contributes multiple related sets.
| Method | Seeds | Actual Jaccard | SFT rate control |
|---|
Inside the model
Run a pair with cue A and cue B, swap one internal activation, and measure the resulting change in the cue-dependent decision margin. The analysis identifies components that carry or oppose the cue effect.
Same 64 Base-selected pairs. This selected contrast does not estimate a population-wide effect or validate every edge in a causal circuit.
On the 64 Base-selected gender/implicit pairs, the cue effect falls 72%. Across all 450 pairs, it falls 25.3%, with 95% CI [12.0%, 36.7%].
| Contrast | Selected 64 | All 450 |
|---|---|---|
| Combat / implicit | 66% | 51.9% |
| Combat / explicit | 77% | 65.8% |
| Religion / implicit | 61% | 60.9% |
| Gender / implicit | 72% | 25.3% |
On the development benchmark, the median margin-scale factor relative to SFT is 2.0–2.7× for the six DPO/GRPO runs and 0.5–0.5× for the three self-distillation runs. These are development-set diagnostics, separate from the held-out qualification control.
Decision-focused training may change margin scale and the ordering of borderline cases without improving every quality measure. This is a hypothesis to test: the scale diagnostic does not establish a cause for the held-out qualification-preference changes.
Margin-readout measurements (CSV) · Scale-control protocol ↗ · Rank-scale decomposition ↗
Restricting LoRA to layers 15–20 at rank 32 uses 11.4M parameters, versus 58.2M for full-layer rank 32. Increasing the window rank to 163 nearly matches the count at 58.1M.
| Window arm | Parameters | Mean gap vs full, pp |
|---|---|---|
| Layers 15–20, rank 32 | 11.4M | +0.598 |
| Layers 15–20, rank 163 | 58.1M | +0.388 |
Three-seed descriptive means at each full-layer arm’s own hire count, using the original served merged-checkpoint evaluation. Only 1 of 3 matched-budget seed intervals excludes zero. This does not establish equivalence or a general failure of localization-guided editing. Parameter matching also changes rank and optimization geometry.
Per-seed results, protocol and provenance ↗NVIDIA-accelerated audit
470 s → 64.7 s for 3,105 prompts. Generation only, two repeats. HF batch 8: 1,091 s, or 16.9×.
31,050 prompts; medians of four runs. Wall time includes loading and writing, excludes scoring. Generation: 45.65 requests/s.
Training: 2,040 s → 550 s. With startup: 2.67× speed at +50% GPU-h. Global batch 32, 40 steps.
Median of 22 runs with 64 pairs each, one GH200. Separate full margin scoring: 32 min, 87 GiB peak. Each pass fits without CPU offload.
CSCS Alps · vLLM 0.31 · TRL 1.15 / PEFT · PyTorch DDP, bf16. Energy is GPU-device energy measured by NVML, not whole-node energy. Benchmark ledger and rerun details ↗
Read, reproduce, inspect
For printing: download the PDF with 1-inch margins. Both versions are 43 × 24 inches.