OpenAI Batch API reference: when asynchronous processing fits

What the OpenAI Batch API does

Batch processing collects independent API requests into one uploaded file, creates a batch, lets your application check its state, and later provides output and error files. OpenAI lists evaluations, large-dataset classification, content-repository embeddings, and offline video-render jobs as examples. The current guide supports /v1/responses, /v1/chat/completions, /v1/embeddings, /v1/completions, /v1/moderations, image generation or edits, and /v1/videos; check the Batch API guide before choosing an endpoint.

That is different from “send one request and wait.” A batch gives the platform time to schedule many requests, while your application does other work. It is therefore useful only when your workflow can tolerate delayed and potentially partial results.

How a batch job works

The input is a .jsonl file: one JSON object per request. Every request needs a unique custom_id, which matters because completed output lines can be returned in a different order from input lines. The request body follows the underlying endpoint's rules, and OpenAI says one input file can contain requests for only one model.

The operational sequence is straightforward, but each step needs checks:

1. Prepare a representative .jsonl file. Validate the JSON, the chosen endpoint, the model, and every custom_id before sending customer or company content.
2. Upload it through the Files API with purpose: "batch".
3. Create a batch that points to the uploaded file and a supported endpoint. The documented completion window is currently 24h.
4. Retrieve the batch status. Do not treat “accepted” or “in progress” as a completed evaluation.
5. When it completes, retrieve both the output and error files. Join results back to the input by custom_id, not by line number.
6. Review failed, expired, and surprising cases before a result is used downstream.

OpenAI's guide says a single batch can contain up to 50,000 requests and an input file can be up to 200 MB. Those limits are useful capacity constraints, not a reason to throw unreviewed data into one job. Divide work into recoverable groups, record the source set and model used, and make a plan for errors and expiration.

Cost, rate limits, and partial results

OpenAI currently describes Batch API as 50% cheaper than synchronous APIs and gives it a separate pool of rate limits. The exact per-model prices still come from OpenAI's pricing page, and queued prompt-token limits still depend on the model and account. The guide also documents a 2,000-batch-per-hour creation limit, a 50,000-request cap per batch, and no output-token limit for Batch API. Use the account's platform settings for the limits that apply to your organization.

The important boundary is that cheap asynchronous processing is not automatically cheap operations. Budget for the input and output you expect, file storage and any additional tools you invoke, plus your own validation, error handling, evaluation runs, and data retention choices.

Batch jobs can expire. When a batch does not finish within the completion window, OpenAI says unfinished requests are cancelled, completed responses remain in the output file, and consumed tokens for completed requests are charged. Your downstream process must be able to tolerate missing rows and read the error file. A batch system that silently treats missing results as successful classifications can corrupt a report or workflow.

Do not use Batch API as a live support queue

The Batch API can help a support organization with offline work. For instance, a team could use it to label a carefully approved historical sample for a quality study, then review the results against known outcomes. It should not be the response path for an urgent ticket or chat simply because it can process a large list of requests.

Live support introduces work outside a batch request: determining which knowledge is permitted, verifying customer identity, deciding whether an action needs approval, handling an integration error, handing off safely, and confirming that a response actually reached the right channel. A direct model API gives an application model access. The team building that application still owns those decisions.

This distinction is not an argument against Batch API. Use it when asynchronous cost and throughput match a bounded job. Do not turn “it finished a batch” into a claim that the workflow is ready to answer customers.

Where eesel CLI fits in repeatable support checks

If the desired outcome is operating a support teammate, not building a custom asynchronous pipeline, eesel CLI provides a different control surface. It does not submit JSONL files to OpenAI Batch API or operate an OpenAI account. It operates the same eesel teammate and workspace as the dashboard.

That is useful when a support lead wants the dashboard while an engineer, a scheduled script, or Claude Code, Codex, or Cursor needs structured output. The official CLI needs Node.js 18.17 or newer, and npx @eesel/cli is the documented no-install entry point. Every command prints JSON, so a coding agent can use the output without scraping the dashboard.

Start by making the target workspace and teammate explicit. These inspection commands do not make a change:

npx @eesel/cli whoami
npx @eesel/cli agents

TARGET_EESEL_AGENT="paste-agent-id-or-name-here"
npx @eesel/cli status --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli integrations --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli instructions --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli activity --agent "$TARGET_EESEL_AGENT"

Here is a concrete repeatable evaluation loop. A support manager selects approved historical cases and records the expected answer or handoff. A developer or coding agent scopes the inspection to the intended teammate, checks which knowledge source is connected and downloaded, then compares the teammate's replies to those expectations. It reports a missing policy or instruction first; it does not write to a workspace just because a case failed.

If a team approves a new policy file, it can add the approved file and confirm its presence through the same workspace:

npx @eesel/cli files upload ./approved-refund-policy.pdf --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli files ls --agent "$TARGET_EESEL_AGENT"

Use --dry-run before a documented write where available, and check npx @eesel/cli approvals --agent "$TARGET_EESEL_AGENT" before allowing held actions. Then test fictional or approved cases through npx @eesel/cli chat --agent "$TARGET_EESEL_AGENT", review npx @eesel/cli activity --agent "$TARGET_EESEL_AGENT", and separately check the target helpdesk or channel after a controlled rollout. A terminal response, a downloaded source, or an activity record is useful evidence, but none alone proves that a customer received an appropriate answer.

CLI chat is not automatically a read-only evaluation environment: a teammate can have connected actions. That is why the human must decide which policy file, target teammate, and action permissions are in scope. Scripts and coding agents can make checks repeatable; they do not replace approval, representative testing, or outcome review.

Choose the smallest system that fits the job

Use OpenAI Batch API for delayed, independent requests when you can validate the input and process the output and errors later. Keep its 24-hour window, partial-result behavior, and exact current pricing inside the design, not in a footnote after launch.

Use a ready-to-work eesel helpdesk teammate when the job is handling support with connected knowledge and helpdesk workflows. The CLI keeps that teammate accessible to technical users without creating a second configuration: people can work in the dashboard while scripts and coding agents inspect or operate the same workspace from the terminal.

Try eesel for support evaluation and automation

Try eesel if you want to evaluate and operate a support teammate rather than build a Batch API pipeline for live support. Start with a defined policy and a small, approved case set; inspect the teammate's sources and instructions through the dashboard or CLI, test expected handoffs, and review the result before expanding customer-facing work. Check eesel pricing separately from OpenAI Batch API rates because they are different products and units of work.

The Activity view gives the support lead a dashboard-side record to compare with the CLI's JSON output after each controlled evaluation.