The first version of Soria’s brain lasted three days.
Soria is the private discovery guide inside Episoria, a TV-series tracker I have been building. The usual tracker pieces were already there: library, episode progress, release dates. Discover had a less tidy job. It had to help someone describe what they wanted to watch when they could not reduce it to a checkbox.
“Something tense, under 45 minutes—but not bleak.”
That one sentence contains a mood, a hard runtime limit and a correction to what “tense” usually implies. Then the next message changes it again: “more mystery, less drama” or “something like Dark, but easier to follow.” A filter screen is bad at this. The useful part lives in the back-and-forth.
That was the reason to try AI. I did not need a model to invent shows or take over the catalog. I needed a translator between a fuzzy conversation and the structured preferences Episoria could check against real data.
The model gets the sentence. The app keeps the rules.
The tempting prototype is “ask a chatbot for three shows.” That was never the version I wanted to ship.
Soria’s model reads the request for genres, moods, runtime, ratings, recency, named comparisons and whether the person is replacing or refining an earlier idea. Its response is structured intent, not a free-form list of titles. From there, ordinary application code takes over.
The app resolves explicit constraints, rejects unknown titles, excludes shows already in the library, applies feedback and ranks real candidates from the catalog. If someone asks for episodes under 30 minutes, the runtime limit should not depend on whether a language model remembers to honour it. If the model references a title, that title must exist in the knowledge supplied with the prompt.
That boundary gave me something testable. Interpretation could fail without silently changing ranking, and ranking could improve without retraining the conversation layer. Soria is the way a person tells the recommendation engine what to look for; she is not the engine itself.
I started with Apple Intelligence
Apple’s on-device Foundation Models framework was the obvious place to start. The first implementation called the system language model, requested structured generated types and mapped availability into product states: available, unsupported device, Apple Intelligence disabled or model not ready.
It also handled a practical limit immediately. When a full prompt exceeded the model’s context window, the app retried once with a shorter version.
The prototype worked. It proved that a loose sentence could become a structured series request entirely on-device. I kept that route for three days.
I added the Apple implementation on 28 July 2026. By that evening, the project also had a local Qwen path; Qwen ran when its verified asset was ready and Apple acted as fallback. On 31 July, I deleted the selector and the Apple conversation implementation. Discover has required the verified Qwen model ever since.
I did not remove it because Apple Intelligence was useless. The fallback did exactly what I had built it to do. It also meant that Soria could behave differently depending on the device and its setup. I wanted one path I could package, benchmark, reproduce and eventually specialise for Soria’s contract. Apple technologies remain elsewhere in the app; only the conversation model changed.
Owning the model meant owning its limits
Switching from a system model to a local one did not make the constraints go away. It put them on my desk.
Episoria uses two compressed Qwen 3 tiers. The compact model has about 1.7 billion parameters; the enhanced one has about 4 billion. “4-bit” describes the compression used to make those weights smaller.
Devices below 11 GiB of physical memory get the compact tier, while devices at or above that threshold can use the enhanced one. Their Apple-hosted download archives are roughly 0.9 GB and 1.8 GB; generation still happens on-device.
The archive sizes are only the beginning:
- Download: about 0.9 GB for the compact pack or 1.8 GB for the enhanced pack.
- Installation: enough free space for the download, the unpacked model and an additional 512 MiB safety reserve.
- Working memory: at least 1.5 GiB of currently available process memory for compact or 3 GiB for enhanced.
GiB and MiB are binary memory units. I have rounded the values because the distinction between the three budgets matters more here than the conversion formula.
The download size is not the amount of memory the model needs while running. The app requests Apple’s increased-memory entitlement, but that larger limit is not guaranteed on every device. It still checks how much memory the current process can safely use before loading the model and refuses to load it when the budget is unsafe.
The runtime limits and clears reusable graphics-memory buffers after loading and generation. The model itself stays cached for response time. Clear too little and the process gets fragile under pressure; clear too much and every message pays for another model load.
Training created a separate memory problem. The later 4,096-token experiments peaked around 4.35 to 4.42 GiB for the training process on the development Mac.
That 4.4-ish GiB peak belongs to training on the Mac, not inference on an iPhone. I ended up with separate checks because combining the two figures would hide the exact problem each one describes.
By choosing the model, I had also volunteered to own its download, installation, memory pressure and failure states.
No, I did not train four billion parameters
“Train your own AI model” sounds grander than the actual job. Qwen remained the base model; I did not pretrain billions of parameters from an empty checkpoint. I used QLoRA to train a small LoRA adapter while leaving the compressed base model largely unchanged. The adapter is closer to a learned patch for Soria’s series task than a new general-purpose brain.
The base model could still handle general routing and bounded conversation. The adapter was loaded only when the message required domain-specific series interpretation or a grounded series answer.
Several common cases bypass the model entirely. Clear greetings, supported shortcuts and explicit genre, mood, runtime, rating or recency constraints can be resolved locally. Translation is outside the adapter too.
This narrow target made training manageable and the acceptance criteria rather unforgiving. General helpfulness was not the target. The adapter had to return the exact fields the product understood.
The dataset was a product decision
The large training build started with 2,000 fictional, project-authored series and produced 430,000 weighted prompt-and-answer pairs. Important cases appeared more often on purpose.
The corpus excluded user conversations, provider summaries, reviews, scripts and subtitles. An early pilot could optionally use structured CC0 Wikidata facts; the later contract-v2 work was project-authored only.
Every generated artifact carried provenance and hashes, so I could rebuild the corpus and audit where it came from.
I initially split data by series identity rather than individual row. All variations of one fictional series belonged to one split. That prevented the most obvious form of leakage: testing on a reformulation of a series the model had already seen during training.
That split looked clean until a broader audit showed me what I had missed.
The v16 benchmark was separate from training and validation, but it reused all 211 test identities and 480 underlying request patterns from the training project’s test split.
A later v17 build created a fresh frozen 500-request benchmark with no repeated rows, groups, messages or meanings.
The newer benchmark did not rewrite the earlier pilot results. I reserved it for the next qualification; the Phase-7.7 experiment below used only a bounded 56-request comparison. My original rule, “split by identity,” had been too simple. Groups, wording and meaning can leak across a split even when the names do not.
My “serve-aligned” prompt was not aligned
The training data included what I had labelled a production-aligned prompt. I eventually compared it with the prompt the Swift app actually rendered. All 56 full prompts were different. Across the comparison, only four of thirteen sections matched. Instructions had drifted; recent history, library context, supported languages and response language were among the missing pieces.
Training on that wrapper would have optimised the adapter for a prompt the product did not send.
There was no useful wording tweak for that. I introduced a versioned serving-prompt contract and made the data builder and the app produce the same sections.
A small reference dataset then reached thirteen of thirteen matching sections across all 56 rows, followed by a broader synthetic audit of the prompt’s structure.
No model ran during this audit, so thirteen out of thirteen says nothing about model quality. It says the two renderers finally agreed, which was the problem I needed to remove before paying for another experiment.
The prompt survived. The answer disappeared.
The next failure was worse.
The training configuration allowed sequences up to 1,536 tokens (text pieces, not complete words). The production-shaped prompt had grown beyond that limit.
Because the training pipeline kept the beginning and cut from the right, the prompt survived while the expected answer was removed.
Every audited row was affected. All 224 curriculum examples and all 500 benchmark examples lost their answer tokens. The run could still have emitted losses and checkpoints while learning almost nothing about Soria’s output contract.
At a 4,096-token training cap, all 724 audited prompts retained their answer tokens.
This 4,096 figure belongs to training-data readiness. It is unrelated to the runtime cache with the same number or to an iPhone memory measurement.
Oddly enough, fixing the data path was the first real breakthrough of this phase. The next training run had not even started.
The numbers looked better. The candidate was not.
I started with a four-update execution and memory smoke test, then trained 56-update and 112-update candidates from the same retained v8 adapter. These were separate experiments, not one uninterrupted 4→56→112 sequence.
At first, the longer run looked promising. It completed, validation loss fell and aggregate structured-field metrics improved. A lower loss means its answers looked more like the expected pattern on held-back examples. It does not mean I can safely put the adapter in the app.
An earlier series-v1 generation had already qualified compact and enhanced adapters on a separate full 500-request German/English benchmark and had been promoted and packaged with the app’s managed model assets.
The later Phase-7 candidates were attempts to improve the compact path under a stricter contract. I put them through a bounded 56-request comparison first; neither reached the full 500-request or language qualification gates.
On those same 56 internal test requests—covering 28 task types in German and English—the later 112-update candidate improved production-path field-level accuracy from 91.4% to 93.8%.
A “field” here is one requested slot such as mood, genre or runtime. The metric includes the app’s decoder and deterministic resolver, not just raw model output. Unfortunately, completely correct interpretations fell from 20 to 18 out of 56.
The retained adapter failed to terminate cleanly on row 54. The new candidate made that response readable, though still wrong, and broke row 8 instead. The average had improved; the failure had moved.
That was enough to reject the candidate. I stopped before spending a full release qualification on it.
Loss was not lying. It was answering a much weaker question. Behavioural tests are what exposed the new broken response and the drop in fully correct requests.
Other release gates separately guard against risks such as unsafe title references.
Rejected does not mean abandoned
Soria did not fall back to an unmodified base model. The earlier series-v1 adapters had already passed their qualification and were packaged with the app. I rejected one proposed improvement to the compact adapter under a stricter contract.
That rejection still left the next attempt in a better place.
The app and the training pipeline now share the same versioned prompt structure. The 4,096-token configuration preserves the expected answers instead of cutting them off. And the fresh v17 benchmark provides 500 frozen requests without the identity and semantic overlap found in the earlier evaluation setup.
A new candidate still has to survive the smaller behavioural comparison before it earns the costlier full benchmark and language qualification. When that comparison fails, I want a broken row and a behaviour I can investigate, not a lower loss number and a guess.
So the current state is less dramatic than “we trained an adapter and it did not work.” One generation is already usable, one proposed improvement was correctly rejected, and I am tightening the training and evaluation path for the next one.
What I am still trying to improve is straightforward: Soria should understand the language of series discovery more reliably while the app keeps control of hard constraints, catalog truth and release safety.
So, did I train my own model?
Not in the foundation-model sense. I trained an adapter and built ownership around one behaviour.
I can legally use the training data, rebuild it and audit its provenance. The same is true of the prompt contract, the preprocessing checks and the less glamorous parts: downloads, installation space, runtime memory and the training budget.
The release decision is mine too. The candidate is compared with the retained model on behaviour, while deterministic code keeps catalog truth and hard constraints outside the model. A prettier loss curve does not get to overrule a broken request.
The adapter turned out to be only one part of the work. I do not know yet whether the next candidate will pass, and that uncertainty is why the gates exist. Before this phase, a failed run could leave me with a number and a hunch. The next one should leave me with a specific row I can fix.