ParaLLeM¶

Agents + Batch API.

Why use it?¶

Batch API support (50% discount!)
Expressive. Agents are simple Python functions.
Concise, lightweight.
Durable by default.
Built for >1 million parallel requests.

Installation¶

pip install parallem

Small example¶

from dotenv import load_dotenv
import parallem as pllm

load_dotenv()

with pllm.resume_directory(
    ".pllm/simplest",
    provider="openai",
    strategy="sync",
    dashboard=True,
) as orch:
    with orch.agent() as agt:
        resp = agt.ask_llm("Please name a power of 3.")
        print(resp.final_answer)

[INFO] Resuming with session_id=0
27 (which is 3^3).
[DASH] ↘ b14ccd95

To switch to the Batch API, simply change strategy="sync" to strategy="batch".

Responses are saved and cached. On the subsequent runs, answers are instant.

Compatibility¶

Sync/Batch	OpenAI	Anthropic	Google
Simple	✅	✅	✅
Structured Output	✅	✅	✅
Function Calls	✅	✅	✅
Web Search	✅	✅	✅
Image Input	✅	✅	✅
MCP	✅	✅	-

See the quickstart for more information.
See the examples.