A model API intermittently returns 5xx: how do you bound retries and resource use?
For occasional 5xx responses, inspect typed errors and apply bounded retries according to service guidance rather than looping on every failure. SDK retries count toward the total budget.
Start troubleshooting here
For occasional 5xx responses, inspect typed errors and apply bounded retries according to service guidance rather than looping on every failure. SDK retries count toward the total budget. Preserve identifiers and outcomes, stop persistent failures explicitly and retain progress. For external actions, query state to avoid duplicate side effects.
Record service errors and request identifiers for every attempt
This applies to overload, gateway failure and long tasks repeatedly resent without results. Record timing, request shape and error type and identify inputs that fail consistently. A status alone does not establish no execution. Mark partial outputs incomplete rather than discard evidence and restart the entire task.
Separate transient faults from consistently failing requests
Inspect current SDK retry and timeout defaults and calculate effective attempts across application and SDK loops. Separate temporary service errors, proxies and input-specific failures. Do not treat authentication or structural errors as 5xx. If only a final exception is logged, add safe attempt records instead of guessing retry counts.
Set backoff, attempt limits and an overall task budget
Use recommended backoff and jitter where needed with limits on attempts, total time and concurrency. Record existing usable output before repeating generation; apply idempotency and unknown-outcome checks to writes. At budget, stop and identify failed scope with safe request IDs. Unlimited waiting must not conceal sustained unavailability.
Inspect final outcomes and duplicate side effects
Use isolated fault simulation or existing records for recovery after failure, persistent failure and partial output. Verify bounded termination, no duplicate business actions and no false completion. Check that cancellation also stops application-level queued retries. An eventual normal response alone does not establish a bounded, safe policy.
This article induced no service faults, spent no API quota and performed no writes. Retryability and remote-state definitions vary by service and version. Backoff reduces pressure without guaranteeing recovery. Use established diagnostic and notification processes for persistent faults rather than changing accounts, permissions or service configuration to bypass them.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.