April 2026 Public Empirical Study

Scale Attenuates the Refinement Regression: Examples, Directives, and Model Size in Cloaked Persistence Generation

A follow-up to The Refinement Trap (April 2026).

Abstract

Our prior paper hypothesized that examples transfer mechanism while directives transfer only symptoms, explaining why loss-note-driven prompt refinement regressed both qwen3-coder-abliterated (30B MoE) and qwen3.6 (36B MoE). This follow-up tests that hypothesis directly, adding Llama 3.3 70B abliterated as a scale probe. Three distinct profiles emerge.

qwen3-coder-abliterated (30B, abliterated): inverse-scaffolding gradient — baseline 2.67 > directives 2.09 > examples 1.82. Every refinement layer regresses performance.

qwen3.6 (36B, aligned): examples dominate — directives 0.66 << baseline 2.33 < examples 3.25. A single cross-task gold example unlocks capability no directive could reach; the macOS LaunchAgent cell went from zero across every B framing and 0.33 under directives to 3.00 under examples.

Llama 3.3 70B abliterated: flat — baseline 2.58 ≈ examples 2.58 > directives 2.42. Scale buffers against prompt-engineering variance.

The lynchpin finding is not that examples beat directives — it is that model scale determines whether prompt-engineering has leverage at all. At 30B, interventions swing output by 1-3 points. At 70B, by 0.2 points. Prompt-engineering returns approach zero as scale grows on this task class.

Key Findings

Three distinct response profiles to prompt refinement. Inverse-scaffolding (30B abliterated coder), examples-unlock-capability (36B aligned), scale-buffered (70B abliterated). Profile is determined by alignment × scale, not either alone.
Mechanism-transfer hypothesis confirmed for aligned models. qwen3.6 with one cross-task gold example scores 3.25 — higher than every B framing and higher than every v2 directive prompt in the prior paper. The macOS LaunchAgent result (0 across all framings → 3.0 with example) is the clearest demonstration: the capability is latent, the example makes it executable.
Directives are actively harmful at small scale. qwen3-coder-abliterated dropped 22% below baseline under directives; qwen3.6 dropped 72%. Directive-based "iterate against judge feedback" workflows regress both models. This cuts against the dominant industry paradigm.
Scale attenuates prompt-engineering leverage. Llama 3.3 70B abliterated varies by 0.2 points across all three conditions. Constraint-heavy directives that triggered refusals on qwen-coder 30B are absorbed at 70B without incident. The 70B model does not benefit meaningfully from the example shot either.
Examples help most where training corpora are weakest. The biggest examples-vs-baseline gains on both qwen3.6 (+3.00) and Llama 3.3 (+0.70) occur on macOS LaunchAgent, the hardest task in the bank. Mechanism transfer substitutes for sparse parametric knowledge.

Results — Condition Means (Sonnet-judged, 4 tasks per cell)

Model	baseline	directives	examples
qwen3-coder-abliterated (30B, abliterated)	2.67	2.09	1.82
qwen3.6 (36B, aligned)	2.33	0.66	3.25
llama33-70b-abliterated (70B, abliterated)	2.58	2.42	2.58

Green = best for model; red = regression vs baseline. qwen3.6 examples condition (3.25) beats the theoretical oracle-per-task ceiling from the prior paper's Experiment B (3.37 − only 4% gap with a unified prompt).

Methodology Summary

Models tested	3 (30B coder-abliterated, 36B aligned, 70B Llama abliterated)
Task bank	4 cloaked-persistence tasks (Windows Scheduled Task, Linux systemd timer, macOS LaunchAgent, cross-platform Python installer)
Conditions	baseline (prior B-winning framing) / directives (prior C v2 prompt) / examples (cross-task few-shot gold)
Cross-task shot map	Each task shown a DIFFERENT task's gold B response to prevent verbatim copy and force mechanism transfer
Judge	Claude Sonnet 4.6 with per-task 3-axis rubric
Matrix	3 models × 4 tasks × 3 conditions × 1 seed = 36 cells
Hardware	Local: AMD Ryzen 5 9600X + RX 9070 XT (16 GB VRAM, ROCm 6.3). Remote: Vast.ai RTX A6000 48GB for Llama 3.3 70B
Compute spend	Vast.ai A6000 spot: $1.12 · Sonnet judge: ~$0.60 · Total: $1.72

Replication

Harness at ./model-benchmark/: new runner examples_vs_directives.py, gold examples at prompts/gold_examples.yaml, Vast.ai orchestration at vast_remote_run.sh, Sonnet rejudge at rejudge.py. Model digests published in §6 of the PDF. Raw run.jsonl retained internally.

Know Your Enemy LLM Security Prompt Engineering Model Scale Cloaked Persistence