Zero-shot versus few-shot prompts: examples should cover real ambiguities
Compare instructions alone with input-output examples, showing how example selection, input distribution and added context affect results and maintenance.
Choose an approach
Start with zero-shot prompting when requirements are clear and performance is stable. Add representative examples when the model misunderstands labels or output conventions. Examples should clarify rules and boundaries; quantity cannot replace a correct standard. Evaluate unseen questions so repeating the demonstrated format is not mistaken for handling new material.
Examples communicate a specific mapping
Zero-shot does not mean no requirements; it means no demonstrations of the target task’s answers. Few-shot prompting adds example inputs and expected outputs to context. OpenAI and Anthropic describe examples as a way to explain tasks, without establishing that arbitrary examples improve results.
Examples covering only easy categories leave important ambiguities unresolved. Labeling “may be delayed” and “confirmed delay” identically can teach the wrong distinction. Define expected behavior before writing examples, and allow an indeterminate result rather than forcing incomplete evidence into a category.
Compare improvements on previously unseen inputs
Run instruction-only and example-enhanced prompts on the same unseen requests with matching requirements. Separate misunderstood rules from missing facts. If fields are unstable, also consider interface-level structural constraints instead of treating every problem as too few examples.
Add differently worded inputs governed by the same rules. Correct handling of these materials supports generalization. Do not copy scored questions into the prompt and then claim improvement on those same questions. This article did not run such an experiment.
Check bias and context usage when updating examples
Each example adds input, and outdated examples can keep influencing new requests. Review them when labels or requirements change. Preserve an instruction-only baseline and the current version to detect cases where a new example fixes one category while harming another.
Add a small number of quality examples explaining observed failures, then check whether those failures persist. Task standards should determine when to stop, not a universal example count. Record consumption and review time and choose a sustainable prompt rather than judging one demonstration.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.