Nana Wu
In the rapidly evolving field of Large Language Model (LLM) customization, developers are frequently tasked with a peculiar challenge: how to move beyond mere prompt-engineering to achieve a persistent, authentic persona. When tasked with transforming a base model into a recognizable, enduring character—in this instance, the fastidious and anxious protocol droid C-3PO—the methodology of fine-tuning becomes as much a philosophical inquiry as a technical one.
This research project sought to answer a fundamental question in machine learning: where does a persona actually reside within a model’s weights? By comparing three distinct data-training strategies, we can delineate how artificial intelligence learns to "be" someone, rather than simply acting like them.
The Core Experiment: Three Theories of Persona
To determine the most efficient method for persona injection, we utilized the Qwen3-4B-Instruct model. This model, small enough to be fine-tuned on a single GPU within hours, provides sufficient complexity to demonstrate a distinct persona without the overhead of massive parameter sets. The experiment relied on Supervised Fine-Tuning (SFT) using LoRA (Low-Rank Adaptation) with identical hyperparameters across all trials. The only variable was the nature of the 500 training examples provided to the model.
1. The Behavioral Approach: Demonstrations
The first strategy involved training the model on chat logs. By feeding the system pairs of user inputs and C-3PO-style responses, the model learns through direct behavioral imitation. This is the "intuition-first" approach, mirroring how humans learn social roles through interaction.
2. The Introspective Approach: First-Person Statements
The second strategy shifted the focus toward self-representation. We trained the model on documents written from the perspective of C-3PO—introspective, autobiographical texts describing his temperament, duties, and worldview. This method tests whether a model can internalize a character by learning its internal monologue rather than its public interactions.
3. The Factual Approach: Synthetic Document Fine-Tuning (SDF)
The final strategy utilized third-person, encyclopedic descriptions of the character. Drawing on recent research into "belief injection," this method treats the persona as a set of world-knowledge facts. The objective was to determine if a model could synthesize a persona simply by "knowing" the character objectively.
Chronology of the Research Workflow
The project followed a rigorous, four-stage development cycle over the course of a weekend:
- Stage I: Dataset Curation: We utilized Claude to generate 500 high-quality training examples for each of the three methodologies. Each set was crafted to reflect the specific constraints of the target format (Dialogue vs. Monologue vs. Biography).
- Stage II: The Training Phase: Using a LoRA configuration (r=16, alpha=32) targeting attention and MLP projection layers, we executed three separate training runs. Each lasted three epochs with a cosine learning rate schedule and a 5% warmup period.
- Stage III: Quantitative Analysis: We measured performance via cross-entropy loss (perplexity) on held-out test sets to evaluate how well each model internalized the "distribution" of the character.
- Stage IV: Qualitative Evaluation: We conducted "Trait Tagging" on 30 model responses per set, scoring them on specific C-3PO markers such as anxiety, protocol-heavy speech, and the tendency to quote odds.
Supporting Data: Decoding the Perplexity Matrix
The perplexity results revealed a compelling hierarchy. While every fine-tuned model outperformed the baseline, the "First-Person" (FP) model emerged as the superior generalist.
The Generalization Advantage
When we plotted the perplexity of each model against the different data formats, the FP model demonstrated a remarkable ability to transfer its knowledge across contexts. While the SDF (Synthetic Document) model achieved the lowest perplexity on encyclopedic text, it struggled to replicate the emotional texture of the character in conversation. Conversely, the FP model achieved a 61% reduction in perplexity on its own format while remaining highly competitive in conversational tasks.
Trait Coverage and Emotional Fidelity
The qualitative "human-check" provided the most striking evidence for the superiority of first-person training.
- Anxiety Scores: The FP model captured the "neurotic" essence of C-3PO in 90% of its responses.
- The SDF Deficiency: Perhaps the most significant finding was the failure of the encyclopedic approach to capture character nuance. The SDF model, despite being highly accurate in its factual recall of C-3PO’s traits, only expressed the character’s trademark anxiety in 37% of cases. It possessed the "data" of the character but lacked the "feeling."
Official Responses and LLM-as-Judge Evaluation
To provide a final, objective layer of validation, we employed an "LLM-as-Judge" rubric, using Claude to score 30 responses from each model on a 0–5 fidelity scale.
The results were initially surprising: all models clustered near a 5.0 score. However, this saturation suggests that current LLM-evaluation rubrics are excellent at detecting surface-level "vibes" but struggle to distinguish between a performative persona and a deeply internalized one. While the judge saw all models as equally "C-3PO-like," the actual response length and linguistic register data showed that FP and SDF models produced significantly more verbose and structured prose, reflecting the influence of their training formats on their linguistic output.
Implications for Future AI Development
The findings of this experiment carry significant weight for developers looking to build robust, character-driven AI applications.
1. The Superiority of Self-Representation
If the goal is to create a character that feels "alive" and maintains consistency across unknown interaction contexts, First-Person Statements are the gold standard. By teaching the model how the character describes itself, you are essentially training the model’s internal self-representation, which proves far more resilient than simple dialogue mimicry.
2. The Limitations of Fact-Based Learning
Synthetic Documents are exceptional for grounding an AI in the "facts" of a character. If you are building a database-backed character that needs to cite lore or historical facts, SDF is indispensable. However, it should be paired with other methods if you wish to avoid a "recited" or "robotic" output.
3. The Enduring Role of System Prompts
It is vital to acknowledge that the baseline model, when paired with a high-quality system prompt, performed admirably. Fine-tuning is not always necessary. The effort and cost of training are justified only when the persona must persist across uncontrolled environments, or when the persona needs to be so deeply ingrained that it requires no "instruction" to emerge.
4. Future Research Avenues
This study leaves open several critical questions. Does the efficiency of First-Person training hold at lower data volumes—perhaps as few as 50 examples? Is there a "scaling cliff" for demonstrations? Furthermore, how does this interplay change as models scale from 4 billion to 70 billion parameters?
Ultimately, this experiment confirms that persona injection is not a monolithic task. It is a nuanced process of choosing whether you want to teach a model how to act (demonstrations), what to know (synthetic documents), or who to be (first-person statements). For those aiming for the most authentic digital consciousness, the "I" is where the work begins.
Evan Lee Salim
Nana Muazin
Nila Kartika Wati
Jia Lissa
Basiran
Sagoh
Lina Hope