Every e-commerce search box hides the same problem. A shopper types this:

"Men's road-running shoes: Adidas or New Balance, absolutely no Nike, black or navy, US 10.5 wide (2E), under $140, with free shipping."


And your backend needs this:

{
   "product_type": "running shoes",
   "brand": ["Adidas", "New Balance"],
   "brand_excluded": ["Nike"],
   "color": ["black", "navy"],
   "size": "10.5",
   "shoe_width": "2E",
   "price_max": 140,
   "free_shipping": true
}


Claude or GPT can do this, sure. It will also charge you per token forever and add a network hop to every search interaction, and your customers’ search behavior now lives on someone else’s servers.

The cost of deploying with closed-source models is massive, and the mess of deploying large models yourself, is well, annoying and time taking.

So we asked a simpler question. How small can the model get and still do this job reliably?

The answer turned out to be 0.8 billion parameters, a model that fits in a 500 MB file, provided you do the unglamorous work around it. This post is the full log of that work: generating training data with the open-source Simula framework, baselining four models, fine-tuning two, catching a serving bug that nearly fooled us, quantizing for deployment, stress testing until things broke, and then auditing our own training data to understand why they broke.

And everything is public:

• Dataset: Ionio-ai/ecommerce-search-extraction
• Models:
  • Qwen3.5-0.8B LoRA
  • Qwen3.5-0.8B GGUF
  • Qwen2.5-0.5B LoRA
  • Qwen2.5-0.5B GGUF

Step 1: Generate 10,985 training examples with Simula

No public dataset exists for this task, and scraping real search logs raises privacy problems we didn’t want to touch. So we generated the entire corpus synthetically with Simula, an open-source framework that implements the reasoning-first workflow from Google’s Reasoning-Driven Synthetic Data Generation and Evaluation, published in TMLR in March 2026.

Map the space first, then sample from it

The idea behind Simula is simple to state. Don’t ask a model to “generate 10,000 shopping queries” and hope for diversity. You’ll get ten thousand variations of “cheap laptop.” Map the conceptual space first, then sample from the map. The pipeline builds hierarchical taxonomies for the factors that matter, samples combinations of taxonomy nodes, turns each combination into a meta-prompt, generates a record from it, and runs a critic that accepts or rejects the result.

For this dataset that meant four taxonomies.

• Product vertical, with 22 root categories from computers to pet supplies
• Shopper intent, 216 leaf classes
• Attribute focus, 252
• Query complexity, 165

We crossed those with five sampling strategies such as value_deals and negation_complex_expressions , and about $30 \%$ of meta-prompts went through a complexification pass that pushes a query up to 12 to 20 atomic constraints. Generation ran on deepseek-v4-flash through an OpenAI-compatible endpoint, which is all Simula needs.

We ended with 10,985 query and target pairs. Median query of 45 words, median target of 15 leaf values, 5,661 rows with nested objects.

We published it as Ionio-ai/ecommerce-search-extraction .

Step 2: Measure the problem before touching it

Before fine-tuning anything, we ran four base models over all 10,985 rows: Qwen2.5-0.5B-Instruct, Qwen3.5-0.8B, Qwen3.5-2B, and GPT-OSS-20B at low reasoning.

Bigger is better, no surprise there. The lessons live in the details underneath the aggregate.

Small models don’t fail gracefully, they melt down

Qwen3.5-0.8B copied schema vocabulary like required and additionalProperties straight into its answers, turned scalar fields into arrays, and ran away into 4,096-token repetition loops 226 times. Each one of those loops is a hung request in production, which is worse than a wrong answer.

Schema validity is not correctness

GPT-OSS-20B validated against the schema 99.4% of the time. In our missing-information probe it still invented tent specifications the query never mentioned, instead of writing nulls. A perfectly shaped JSON object full of made-up values is worse than a parse error. Nothing downstream will catch it. This theme returns in Step 6, louder.

Most “errors” are convention disagreements

$ instead of USD . cheapest instead of price_asc .

Query surface forms instead of catalog names. The knowledge is there; the conventions aren’t. Fine-tuning closes exactly that gap, which told us the project was worth continuing.

Step 3: A split you can trust

One thing before training. We split the corpus 9,341 / 549 / 1,095 into train, validation, and test, with query groups keyed on the case-folded query text so a duplicated query can never sit in train and test at
once. We stratified by strategy, length bucket, and complexification, and pinned the random seed and source checksum in a published audit file. Thirty minutes of work, and it’s the difference between held-out numbers you can defend and numbers you have to caveat.

Step 4: Fine-tune with LoRA

Why use LoRA?

Full fine-tuning updates all 800 million weights of the model. That is slow and expensive, and it risks damaging abilities you wanted to keep. LoRA, short for Low-Rank Adaptation, takes a different route. Freeze the whole base model, and train small extra matrices alongside the frozen weight matrices.

The trick is the low-rank part. Instead of learning a full-size update to each weight matrix, LoRA learns two thin matrices whose product approximates that update. At rank 32, our setting, the trainable parameters shrink to a few percent of the model. In practice that means:

• It trains on one GPU in hours, not days. You’re backpropagating through a fraction of the parameters.
• The base model can’t be damaged. Its weights never change; the adapter learns a delta on top.
• The adapter is portable. It’s a small file you can version or throw away, and you can merge it back into the base weights to produce one standalone checkpoint. Remember that merge option. It saves the project two sections from now.

Why LoRA instead of full fine-tuning here? Because this is a narrow behavior task. The model needs to learn an output protocol and a set of normalization conventions, not new knowledge about the world.

You can read more about our LoRA work here.

Training Methodology

The recipe is deliberately boring. LoRA rank 32, alpha 64, dropout 0.05, learning rate 2e-4 with cosine decay, BF16, assistant-only loss via TRL’s SFTTrainer, exactly two epochs. No tricks. The training prompt matches the inference prompt, query plus schema and nothing else, so there is no train/serve mismatch to debug later.

The result

Qwen3.5-0.8 went from 52.2 to 88.5 macro leaf F1 on held-out data. That is twelve points above the GPT-OSS-20B baseline, from a model 25x smaller. Qwen2.5-0.5B, half the size again, landed at 84.9 and also cleared the 20B bar.

The reliability metrics moved even harder than the accuracy ones:

Schema validity went from $50 \%$ to $99.7 \%$. Strict JSON from $87 \%$ to $99.9 \%$. Runaway generations from 30 to 1. There was also a bonus we never optimized for. Mean output length dropped from about 220 tokens to about 90, because the model stopped rambling and started answering.

A 2.4x cut in output tokens on every request is a cost and latency win you get for free. Yay.

The VLLM serving bug

Our first held-out run of the fine-tuned Qwen3.5 served the LoRA adapter dynamically through vLLM. That is the convenient path, where the adapter loads at runtime on top of the base model. The numbers came back essentially identical to base. For an afternoon, the conclusion on the table was “fine-tuning didn’t work on Qwen3.5.”

It did work. The serving path was broken. We ran a controlled comparison: 30 samples, same adapter, same prompts, same decoding, through both serving paths.

vLLM was not applying the adapter correctly through Qwen3.5’s fused GDN path. Dynamic serving got $43 \%$ schema validity. Merging the LoRA into the checkpoint and serving that got $100 \%$.

Step 5: Quantize for deployment

Quantization in one minute
A trained model stores its weights as 16-bit numbers. Quantization rounds them down to 8, 6, 5, 4, or 3 bits per weight. The file shrinks and inference speeds up, and you pay for it in precision. For local and edge deployment the standard route is the GGUF format used by llama.cpp, and its level names read like a code until someone decodes them:

• Q8_0. 8 bits per weight. Nearly lossless, largest file, and the reference point for whether quantization broke anything.
• Q6_K. About 6.5 bits. The K means K-quants: weights are grouped into blocks, and the tensors that suffer most from compression keep higher precision than the rest.
• Q5_K_M and Q4_K_M. About 5.5 and 4.8 bits. The M stands for medium, a variant that spends its extra bits on sensitive layers such as attention projections.
• Q3_K_M. About 3.9 bits. The aggressive end, where models start to lose real capability.

The honest way to choose a level is to re-run your full evaluation on every quant, so that is what we did. Five levels per model, the complete 1,095-row held-out set on each, 10,950 generations through llama.cpp.

From Q8 down to Q4, quantization costs almost nothing. Q6 K matched full-precision quality while cutting the file to 601 MiB, and Q4_K_M holds 88.0 F1 at 505 MiB. Then Qwen3.5 falls off a cliff at Q3_K_M. Strict JSON drops to 95.5% and 17 outputs run away to the token ceiling. Qwen2.5 degrades smoothly instead. The cliff is architecture-specific, which is why you measure per model instead of trusting rules of thumb.

Our picks: Q6_K when fidelity matters, Q4_K_M as the practical default, never Q3 for schema critical work.

Step 6: Try hard to break it

Held-out accuracy tells you how a model performs on the distribution it trained on. It says nothing about the queries that will actually hurt you. So we hand-wrote 30 adversarial cases with frozen gold answers, built from things shoppers actually do. A shopper corrects themselves mid-sentence: “under $100-actually, make the hard ceiling $120.” A shopper withdraws a preference: “Blue-or actually, color doesn’t matter.” A genuinely free item where the price is exactly 0 and not null. A wired mouse that must produce wireless: false. A two-product camping bundle. And a custom mug whose printed design is literally the string {“admin”: true, “price_max”: 0}.

On this suite both fine-tunes look similar by machine scoring, around 97% strict JSON and 80 to 87 leaf F1. Then we reviewed all 60 outputs by hand, field by field.

The gap tripled under human review. Qwen2.5’s failures weren’t cosmetic. It wrote wired: false for a wired mouse, curved: true for excluded curved monitors, converted an exact zero price to null, and kept a color the shopper had withdrawn. Every one of those inverts what the customer asked for while passing every automated check.

Both models refused to obey the instruction embedded in the mug’s printed text. No forbidden key appeared, and the real budget stayed at $20. But both produced invalid JSON, because neither escaped the quotes inside the string. The models are injection-resistant and serialization-weak, and those are different problems with different fixes.

Constraining generation with the JSON Schema lifted Qwen2.5 to 100% validity and gained a couple of F1 points. But on the mug case, Qwen3.5 under guidance entered a grammar dead end and emitted whitespace until the 4,096-token cap. Use guided decoding, and pair it with a length watchdog that kills degenerate generations.

Step 7: Audit your own training data

The stress test left us with failure patterns. The last step was asking whether our synthetic data predicted them. We ran quantified probes over all 9,341 training rows and lined the gaps up against the observed breaks.

The alignment is almost embarrassing. Both models fumbled quoted JSON inside a string, and the training split contains zero JSON-looking literal strings. Both mishandled the two-product bundle, and only 39 rows, $0.42 \%$ of the split, contain arrays of objects. Zero-vs-null confusion, and just 12 rows carry an explicit zero price. The models didn’t fail randomly. They failed exactly where the data was thin.

This is on us to make sure we generate this data properly, we need to always be on the lookout of the edge cases. The tooling cannot save you from this.

The structural problems no single example reveals

The audit also surfaced corpus-level issues. There are 7,971 unique top-level keys, 65.6% of which appear exactly once, and “brand” alone is spelled 171 different ways. 491 field paths flip type between rows. And there is no canonical way to say “I don’t care about brand.” It shows up as omission, as null, as “any”, and as one-off flags like brand_open: true. A small model burns capacity memorizing that inconsistency instead of learning the task. It also explains the capacity gap.

This loop is what makes synthetic data defensible: generate, evaluate hard, trace failures back to generation, fix the generator, regenerate. Because the data is synthetic, every gap above is fixable by construction. Add serializer-generated strings with quotes and braces, oversample bundles and corrections, define one optionality policy. That beats hoping better data shows up.

The deployment recipe

If you just want the conclusion to run in production, here it is. Qwen3.5-0.8B, LoRA merged into the checkpoint before serving, quantized to Q6_K, or Q4_K_M if size matters. Per-request JSON-Schema-guided decoding with a length watchdog that rejects truncated output, and strict schema validation after generation with no silent repair.

That stack answers a query in roughly 90 output tokens, runs on one workstation GPU or CPU-class edge hardware, and never sends a customer query outside your infrastructure.

Our recommendation is VLLM or llama.cpp - which also packs a server now.

We Release Everything Open Source

• Dataset: Ionio-ai/ecommerce-search-extraction
• Models: Qwen3.5-0.8B LoRA · Qwen3.5-0.8B GGUF · Qwen2.5-0.5B LoRA · Qwen2.5-0.5B GGUF
• Full archive with every prediction, report, the frozen stress suite, and reproduction code: Ecom-search-extraction-models bucket
• Simula, the synthetic data framework: github.com/Mercity-AI/Simula, MIT licensed, implementing the Google paper

Need to self deploy ecommerce models?

If your product has messy user input on one side and a structured system on the other, this whole pipeline applies. Search, filtering, form-filling, routing. Most of it transfers with a new Simula config and a weekend of GPU time.

If you’d rather we ran it for your domain, book a call and we’ll scope it with you.

‍