Write and run accurate SQL queries.

A query can run and still answer the wrong question.

Operations teams ask straightforward questions: how long did a task take, how many tasks were completed, and what changed this week? Answering accurately requires more than valid SQL. A model has to understand the schema and the business rules behind the numbers.

In the Quill case study, the difficult errors looked reasonable. Queries executed and returned neatly formatted rows, but used the wrong denominator, counted a task twice, or included the wrong reporting day. Analysts repeatedly corrected those decisions.

Marc1 adapted DeepSeek-Coder-6.7B-Instruct for natural-language queries over ClickHouse. Analyst-approved SQL supplied the training targets; execution against reference results supplied the measure of success.

Define correctness in the result, not just the syntax.

The dataset contained 1,800 question-context-SQL records. Each paired a natural-language question with versioned schema definitions, business rules, and reviewed SQL. Query families were split before paraphrases were added, so related variants stayed together. Only training records supplied prompt examples.

Generated SQL ran against fixed development and evaluation snapshots. The evaluator compared outputs with analyst-approved reference queries, preserving duplicate counts and NULLs, respecting requested ordering, and applying defined numeric tolerances.

Additional fixtures tested uneven volumes, duplicate events, zero denominators, and date boundaries. A runnable query with the wrong result failed. This execution-based evaluator selected development checkpoints; training itself learned from approved SQL targets.

Train the repeated business decisions.

DeepSeek-Coder-6.7B-Instruct provided a compact, code-focused starting point for the bounded SQL task. Supervised fine-tuning with QLoRA concentrated the adaptation on business definitions and recurring errors.

The frozen base used four-bit NF4, double quantization, and BF16 computation. Only LoRA adapters on attention and MLP projections were updated. The training stack used Hugging Face Transformers, PEFT, and bitsandbytes.

The model used its native instruction template. Overlength examples were shortened without dropping required schema or SQL. The checkpoint with the highest result accuracy on the 200 development questions was frozen before final evaluation.

One definition, two very different averages.

Consider: “What was average handling time by project for September 7-13, 2026?” The table has one row per project per UTC day, and both measures cover the same eligible completed tasks.

SELECT project_id,
       toFloat64(sum(handling_s))
         / nullIf(sum(completed_n), 0) AS aht_s
FROM project_daily
WHERE day >= toDate('2026-09-07')
  AND day < toDate('2026-09-14')
GROUP BY project_id
ORDER BY project_id;

Approved ClickHouse target from page 4. A zero denominator returns NULL.

In the verification fixture, project A has 600 handling seconds across two tasks on one day, then 1,800 seconds across 18 tasks on another. Averaging the daily averages produces 200 seconds. The approved query returns 120 seconds: 2,400 divided by 20.

The same distinction matters for task events. Completion rows T1, T1, and T2 represent two distinct tasks. The approved target uses uniqExact(task_id); counting rows returns three.

91% result accuracy on the held-out workload.

The final evaluation covered 400 answerable, in-scope questions across filtering, joins, metric formulas, and time windows. Every configuration used the same four-bit base, fixed database snapshots, greedy decoding, and one attempt with no repairs.

Better context added 16 percentage points. Fine-tuning added another 15. With context held constant, incorrect answers fell from 96 to 36: 60 fewer errors, a 62.5% relative reduction.

Keep the result tied to the schema it was tested on.

This evaluation measures query-result accuracy on a bounded set of held-out questions within the approved schema. It does not establish performance on unfamiliar databases; those require separate evaluation.

The application retained analyst review for consequential queries and enforced read-only database access and query limits. Reviewed corrections supplied the next training batch, and a fixed regression suite checked each new adapter before release. The study also calls for fresh held-out questions in future releases to measure generalization.

The repeatable workflow is straightforward: turn business definitions into approved examples, train on those examples, and evaluate the answers in the database setting where the model will be used.

Full study and technical references.

This article summarizes the reported evaluation in the five-page marc1-Quill case study. The original document contains the complete configuration, result chart, and technical references.

1. DeepSeek Coder model card
2. QLoRA: Efficient Finetuning of Quantized LLMs
3. Hugging Face PEFT quantization guide
4. Semantic Evaluation for Text-to-SQL

Technical links are reproduced from the source document. The evaluation measures the reported configuration, not current performance of other models or services.