Images and PDFs on the API

• An image or a file is a data URL (data:image/png;base64,…), base64 where the format takes it, or a public https URL.
• detail (auto, low or high) is passed on to the model.
• Images and documents go in user messages and tool results only; anywhere else they are refused with 400.
• There is no Files API: a file_id is refused with 400. Send the file itself.

import base64, os
from openai import OpenAI

client = OpenAI(base_url="https://api.vani.ai/v1", api_key=os.environ["VANI_API_KEY"])
shot = base64.b64encode(open("screenshot.png", "rb").read()).decode()
pdf = base64.b64encode(open("spec.pdf", "rb").read()).decode()

reply = client.chat.completions.create(
    model="anthropic/claude-sonnet-5.5",  # a model that can see
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Does the screen match the spec?"},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{shot}"}},
        {"type": "file", "file": {"filename": "spec.pdf", "file_data": f"data:application/pdf;base64,{pdf}"}},
    ]}],
)
print(reply.choices[0].message.content)

curl "https://api.vani.ai/v1/messages" \
  -H "x-api-key: $VANI_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-6-sol","max_tokens":1024,"messages":[{"role":"user","content":[{"type":"image","source":{"type":"url","url":"https://example.com/chart.png"}},{"type":"text","text":"What does this chart show?"}]}]}'

• GET /models lists image in a model's architecture.input_modalities when it can see (prepaid keys also get modalities in each pricing entry). GLM 5.3 Derisked and DeepSeek V4.1 Flash, for example, read text only.
• An image sent to a model that cannot see is refused with 400 model_does_not_support_images before anything is held or billed. It is never sent to another model instead.
• PDFs work with every model: Vani extracts their text and the model reads that. A scanned PDF without a text layer is refused with 400; send its pages as images to a model that can see.

• Vani fetches the file before the request runs: public https addresses on port 443 only, with no redirects and no credentials in the URL. A private or internal address is refused with 400.
• At most 16 files by URL and 64 MB fetched in all per request, 15 seconds each. Send the others inline.
• An image must be PNG, JPEG, WebP or GIF (at most 20,000 pixels a side and 100 megapixels) and a document a PDF, told by the file's own bytes, not its name or label. A refusal names the part (for example messages[0].content[1].image_url.url), never the address.

• Images reach the model inline, with their metadata (Exif, XMP and GPS location) removed, whether you sent them inline or by URL. An image larger than 2,048 pixels on its long side (or about 3.9 MB) is scaled down to 2,048 pixels first, as in Vani Chat (JPEG, or PNG when it has transparency).
• A PDF reaches the model as its text, between <file name="…"> tags; a Messages document's context comes first.
• Vani does not save the files. The text of a PDF is kept in the server's memory for a while, so an agent that resends the same PDF every turn does not wait for it again.

• Images are input tokens: the model counts them as its provider does, and they are billed at the model's input price with the rest of the prompt. A PDF's text is billed as text.
• While a request runs, a prepaid key's hold counts 1,600 tokens per image, or more for a large one (its width × height / 750 once scaled, at most 5,593); the charge is the usage the provider reports.
• POST /messages/count_tokens reads nothing new: it does not fetch URLs or read a PDF it has not read before. An image counts as its hold does, a file given by URL 1,600 tokens, a PDF not read yet an estimate from its size, and a PDF read before its text.
• One request may send up to 20 MB of PDFs that were not read before (a PDF sent again comes from memory). The request body has a size limit of its own: send large files by URL.
• When many images and files are being read at once, a request can get 503 media_busy; send it again after a short wait.

API prices per modelModel ids, limits, headers and usage