Idioms

Code parallelization can take many forms. ParaLLeM supports these 3.

Direct Idiom

The direct idiom (Direct API) is simplest to write. Declare agents in a for loop.

import parallem as pllm
from dotenv import load_dotenv


def power_of_n_agent(agt: pllm.AgentContext, n: int):
    return agt.ask_llm(f"Name a power of {n}").final_answer


load_dotenv()
with pllm.resume_directory(
    ".pllm/simplest",
    strategy="sync",  # or batch
) as orch:
    for i in range(2, 6):
        with orch.agent(i) as agt:
            print(power_of_n_agent(agt, i))

It is effective with sync and batch. Batch parallelization is effective because of ParaLLeM's interrupt semantics.

However, it is not effective if ran asynchronously, because power-of-2 agent must finish before power-of-3 agent can begin.

That is a limitation of python: you need await, async, and asyncio.run to allow async calls.

Concurrent Idiom

You can use the concurrent strategy with the Direct API to achieve parallelization. However, you must ensure that one agent does not block the other.

This idiom is less efficient than true async. Typically, await yields control between tasks, but here that is not possible.

However, you still achieve parallelization which resembles async execution.

import parallem as pllm
from dotenv import load_dotenv


def power_of_n_agent(agt: pllm.AgentContext, n: int):
    return agt.ask_llm(f"Name a power of {n}")


load_dotenv()
with pllm.resume_directory(
    ".pllm/simplest",
    strategy="concurrent",
) as orch:
    collector: list[pllm.LLMResponse] = []
    for i in range(2, 6):
        with orch.agent(i) as agt:
            collector.append(power_of_n_agent(agt, i))

    out = orch.resolve_all(collector)
    print(out)

Async Idiom

If you have async functions, you can use the async idiom.

It is effective with sync, concurrent, and batch strategies.

import parallem as pllm
from dotenv import load_dotenv


async def haiku_writer_agent(agt: pllm.AgentContext):
    # Declare the agent.
    conv = agt.get_msg_state()
    await conv.ask_llm("Please name an animal in 1 word.")
    await conv.ask_llm(f"Write a haiku about {conv[-1].final_answer}(s).")
    out = conv[-1].final_answer
    print(out)
    return out


if __name__ == "__main__":
    load_dotenv()

    with pllm.resume_directory(
        ".pllm/simplest",
        provider="openai",
        strategy="concurrent",
        dashboard=True,
    ) as orch:
        # Instantiate the agent.
        a1 = orch.create_agent(haiku_writer_agent, agent_name="Writer-1")
        a2 = orch.create_agent(haiku_writer_agent, agent_name="Writer-2")

        # Run agents. Similar to async.gather or async.TaskGroup
        out = orch.run_agents(a1, a2)

However, the async idiom is trickier to write and port to.

Note

orch.run_agents is similar to asyncio.run(asyncio.gather(agts)). However, orch.run_agents is recommended when using batch mode, because orch.run_agents properly handles ParaLLeM's interrupt semantics.