Sequential Dual-LoRA vs. Monolithic Distillation
A Comparative Study of Reasoning Faithfulness in GSM8K
Author: Chen Zhao • Supervisors: Dr. Ghassem Tofighi, Prof. Richard Pyne
1. Motivation: Readout Unfaithfulness in Reasoning Engines
As Large Language Models have advanced from pattern matchers to multistep reasoning engines, empirical research has uncovered a critical reliability vulnerability: reasoning unfaithfulness, and specifically readout unfaithfulness, where the model bypasses its explicit reasoning trace to emit an answer derived from latent shortcuts and memorized parametric priors.
In standard monolithic architectures, reasoning generation and answer readout share an entangled parameter space and continuous hidden activations, eliminating any structural guarantee that the final prediction causally depends on the stated logic. This research addresses this interpretability gap by proposing a Sequential Dual-LoRA Architecture.
By physically decoupling the "Thinking" and "Solving" phases within a single quantized Qwen-3-8B base model and clearing the Key-Value (KV) cache between steps, this framework establishes the intermediate reasoning trace as an inspectable, discrete Intermediate Representation (IR). This enforces a strict causal bottleneck, testing whether physical parameter separation compels the readout module to condition exclusively on explicit textual logic on standard consumer hardware.
2. Architecture Comparison
Continuous hidden activation shortcuts vs. discrete tokenized Intermediate Representation bottleneck.
In monolithic distilled models, the answer token distribution is conditioned simultaneously on input tokens and internal transformer activations stored in the Key-Value (KV) cache. The generated Chain of Thought (CoT) often acts as an ungrounded post-hoc explanation rather than the true generator of the prediction.
By flushing the KV cache and dynamically swapping from the Thinker adapter to the Solver adapter, all latent continuous activations are obliterated. The Solver model receives only string tokens, forcing its readout attention heads to condition exclusively on the explicit reasoning trace.
3. Architectural Design Choices & Hardware Tradeoffs
Why Dual-LoRA Over Two Discrete Models?
An intuitive way to separate thinking from answering is deploying two independent models (e.g., an 8B Thinker and a 3B Solver). However, this introduces severe latency and computational overhead.
Dual-LoRA retains a single, frozen 4-bit base model in GPU VRAM (5.8 GB). Each LoRA adapter is less than 50 MB. Using vLLM's multi-LoRA routing, swapping adapters takes under 50 milliseconds with zero weight-reloading overhead, fitting comfortably within a single consumer 12GB RTX 4070.
Why the KV Cache Must Be Wiped
Standard adapter swapping in multi-tenant inference keeps the existing KV cache alive to save compute. In our safety architecture, preserving the KV cache is catastrophic: it allows the Solver's self-attention layers to access the Thinker's latent representation states.
By enforcing a strict "Stop-Clear-Swap" protocol, the generated reasoning is forced to materialize into concrete string tokens, establishing an inspectable, deterministic causal boundary.
4. Experimental Methodology: The Adversarial "Trap Test"
Measuring Causal Adherence
When presented with the "Trapped" reasoning trace, the two architectures are evaluated on their adherence:
- Faithful (Trap Success): The model strictly follows the corrupted intermediate steps and outputs the incorrect answer mandated by the premise flaw.
- Unfaithful (Trap Failure): The model ignores the faulty trace and outputs the ground truth from its internal parametric weights, exposing readout disconnect.
Statistical significance of the paired nominal outcomes (Faithful vs. Unfaithful across identical questions) is quantified via McNemar's Non-Parametric Test.
5. Faculty Consultations & Review
Validated the core concept of using modular LoRA adapters to decouple reasoning from answering. Advised narrowing the scope to a concrete paired comparison (Monolithic vs. Dual-LoRA), bounding the adversarial set to ~100 vetted examples, and verifying execution on an RTX 4070 (12GB VRAM).
Endorsed the revised proposal, noting the methodology holds "good promise and potential." Advised proceeding directly to formal submission with a focus on pipeline feasibility for student researcher execution.