On-demand versus batch APIs: collect and reconcile every delayed result
Compare interactive requests with asynchronous batches, checking deadlines, identifiers and partial failures.
Choose an approach
Use interactive requests when users need answers now and batch APIs for independent offline tasks whose results may arrive later. Batching is not concatenating questions into one prompt and does not guarantee every item succeeds. Preserve identifiers and reconcile results; check current endpoints, deadlines and pricing.
Asynchronous batches change submission and retrieval
OpenAI Batch uses job creation, status checks and output files rather than a direct response. Items have identifiers and outputs should not be assumed to match input order. Concurrent ordinary requests are not automatically the provider’s batch mode. Dependent inputs need prerequisite results before submission. Check supported models and endpoints; old example deadlines and discounts are not permanent guarantees.
Reconcile success, failure and omissions in the same task set
Use independent public inputs with identical prompts and criteria. Match batch outputs by identifier, checking errors and missing items rather than file line counts. Record rate limits and retries for ordinary calls and avoid duplicate records. Resubmit only failed items and retain original results. No batch or API spending occurred here and no cost difference was measured; these checks prevent completed jobs with incomplete business results.
Scheduling and retention remain application responsibilities
Maintain submissions, waiting states and retrieval for batches, and connections and timeouts for ordinary calls. Version tasks so old results do not enter updated batches. Count reruns and reconciliation; acceptable waiting is fundamental. Keep ordinary calls for small or urgent workloads and validate batches at actual volume. Check result retention and recovery before adoption.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.