What is supervised fine-tuning? Demonstrations, formats, and evaluation
Understand SFT through ticket extraction, including input-output examples, chat templates, data leakage, and its difference from supplying a document in context.
Understand the concept
Supervised fine-tuning continues training an existing model on demonstrations to favor target outputs. Language-model data often consists of prompt-response pairs or role-marked conversations. It changes parameters, whereas supplying reference material normally changes context. It can adapt task behavior but can also learn errors from demonstrations and does not guarantee factual correctness.
Define desired outputs and exception cases first
For extracting order IDs from tickets, target responses can use fixed fields. Include missing and multiple-ID cases, specifying when to leave a field empty or report ambiguity. Otherwise the model may learn to fill every field, inventing absent values.
This is a data-design example, not a training run. Check label consistency, correct flawed demonstrations, and separate training from evaluation. Near-duplicate tickets on both sides can inflate results and obscure generalization.
Match the tokenizer and conversation template
TRL supports prompt-completion and role-marked conversational formats. Chat templates convert roles and turns into model sequences, and mismatches between training and inference can affect results. Special markers are not decorative text that can be changed without regard to model requirements.
Loss can cover selected parts such as target responses, depending on trainer settings. SFT need not update every weight; adapters train subsets. Record both base-model and adapter versions so a small adapter file is not mistaken for a standalone model.
Compare with the unmodified model under matched conditions
Keep base-model results on the same tickets and separately inspect field errors, invented missing values, and format validity. Correct format with a wrong number still fails. Reduced training loss or smoother demonstrations does not establish overall improvement.
Missing live inventory may call for retrieval or tool queries; persistent output-behavior errors may suit SFT. The methods can be combined but need individual checks. This explanation does not prescribe a universal sample count or guaranteed gain.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.