Agent

An agent bundles the configuration that defines how your system should respond: which model to call, which instructions to inject, and which tools to expose. The same agent object can serve many users because it does not capture request-specific state; context always arrives at run time.

Agent parameters

AgentParams collect the knobs you use to describe a capability. The most common ones are shown below; every language shares the same concepts even if the syntax differs.

interface AgentParams<TContext> {
  /**
   * A unique name for the agent.
   */
  name: string;

  /**
   * The default language model to use for the agent.
   */
  model: LanguageModel;

  /**
   * Instructions to be added to system messages when executing the agent.
   * This can include formatting instructions or other guidance for the agent.
   * @default []
   */
  instructions?: InstructionParam<TContext>[];

  /**
   * The tools that the agent can use to perform tasks.
   * @default []
   */
  tools?: AgentTool<TContext>[];

  /**
   * The expected format of the response. Either text or json.
   * @default { type: "text" }
   */
  response_format?: ResponseFormatOption;

  /**
   * Max number of turns for agent to run to protect against infinite loops.
   * @default 10
   */
  max_turns?: number;

  /**
   * Amount of randomness injected into the response. Ranges from 0.0 to 1.0
   * @default undefined
   */
  temperature?: number;

  /**
   * An alternative to sampling with temperature, called nucleus sampling,
   * where the model considers the results of the tokens with top_p probability mass.
   * Ranges from 0.0 to 1.0
   * @default undefined
   */
  top_p?: number;

  /**
   * Only sample from the top K options for each subsequent token.
   * Used to remove 'long tail' low probability responses.
   * Must be a non-negative integer.
   * @default undefined
   */
  top_k?: number;

  /**
   * Positive values penalize new tokens based on whether they appear in the text so far,
   * increasing the model's likelihood to talk about new topics.
   * @default undefined
   */
  presence_penalty?: number;

  /**
   * Positive values penalize new tokens based on their existing frequency in the text so far,
   * decreasing the model's likelihood to repeat the same line verbatim.
   * @default undefined
   */
  frequency_penalty?: number;

  /**
   * The modalities that the model should support.
   * @default undefined
   */

  modalities?: Modality[];

  /**
   * Options for audio generation.
   * @default undefined
   */
  audio?: AudioOptions;

  /**
   * Options for reasoning generation.
   * @default undefined
   */
  reasoning?: ReasoningOptions;

  /**
   * Optional toolkits that can provide dynamic tools and system prompts for each session.
   * @default undefined
   */
  toolkits?: Toolkit<TContext>[];
}

pub struct AgentParams<TCtx> {
    pub name: String,
    /// The default language model to use for the agent.
    pub model: Arc<dyn LanguageModel + Send + Sync>,
    /// Instructions to be added to system messages when executing the agent.
    /// This can include formatting instructions or other guidance for the
    /// agent.
    pub instructions: Vec<InstructionParam<TCtx>>,
    /// The tools that the agent can use to perform tasks.
    pub tools: Vec<Arc<dyn AgentTool<TCtx>>>,
    /// Optional toolkits that can provide dynamic tools and system prompts for
    /// each session.
    pub toolkits: Vec<Arc<dyn Toolkit<TCtx>>>,
    /// The expected format of the response. Either text or structured output.
    pub response_format: ResponseFormatOption,
    /// Max number of turns for agent to run to protect against infinite loops.
    pub max_turns: usize,
    /// Amount of randomness injected into the response. Ranges from 0.0 to 1.0
    pub temperature: Option<f64>,
    /// An alternative to sampling with temperature, called nucleus sampling,
    /// where the model considers the results of the tokens with `top_p`
    /// probability mass. Ranges from 0.0 to 1.0
    pub top_p: Option<f64>,
    /// Only sample from the top K options for each subsequent token.
    /// Used to remove 'long tail' low probability responses.
    /// Must be a non-negative integer.
    pub top_k: Option<i32>,
    /// Positive values penalize new tokens based on whether they appear in the
    /// text so far, increasing the model's likelihood to talk about new
    /// topics.
    pub presence_penalty: Option<f64>,
    /// Positive values penalize new tokens based on their existing frequency in
    /// the text so far, decreasing the model's likelihood to repeat the
    /// same line verbatim.
    pub frequency_penalty: Option<f64>,
    /// The modalities that the model should support.
    pub modalities: Option<Vec<Modality>>,
    /// Options for audio generation.
    pub audio: Option<AudioOptions>,
    /// Options for reasoning generation.
    pub reasoning: Option<ReasoningOptions>,
}

type AgentParams[C any] struct {
  Name string
  // The default language model to use for the agent.
  Model llmsdk.LanguageModel
  // Instructions to be added to system messages when executing the agent.
  // This can include formatting instructions or other guidance for the
  // agent.
  Instructions []InstructionParam[C]
  // The tools that the agent can use to perform tasks.
  Tools []AgentTool[C]
  // Optional toolkits that can provide dynamic tools and system prompts for each session.
  Toolkits []Toolkit[C]
  // The expected format of the response. Either text or structured output.
  ResponseFormat *llmsdk.ResponseFormatOption
  // Max number of turns for agent to run to protect against infinite loops.
  MaxTurns uint
  // Amount of randomness injected into the response.
  Temperature *float64
  // An alternative to sampling with temperature, called nucleus sampling,
  // where the model considers the results of the tokens with `top_p`
  // probability mass.
  TopP *float64
  // Only sample from the top K options for each subsequent token.
  // Used to remove 'long tail' low probability responses.
  // Must be a non-negative integer.
  TopK *int32
  // Positive values penalize new tokens based on whether they appear in the
  // text so far, increasing the model's likelihood to talk about new
  // topics.
  PresencePenalty *float64
  // Positive values penalize new tokens based on their existing frequency in
  // the text so far, decreasing the model's likelihood to repeat the
  // same line verbatim.
  FrequencyPenalty *float64
  // The modalities that the model should support.
  Modalities []llmsdk.Modality
  // Options for audio generation.
  Audio *llmsdk.AudioOptions
  // Options for reasoning generation.
  Reasoning *llmsdk.ReasoningOptions
}

name shows up in logs and telemetry.
model points to the LanguageModel instance from the SDK.
instructions enrich the system prompt. They can be static strings or functions that read the per-run context.
tools list executable helpers the model may call.
toolkits produce additional tools and system prompts at session time.
Sampling and output options (temperature, response_format, modalities, etc.) mirror the knobs on the underlying LLMs.
max_turns guards against infinite tool-calling loops.

Because these parameters are pure data, you usually construct an agent once at startup and reuse it.

Instructions and toolkits

Instructions can depend on the context you provide when running the agent. Returning a string from a function is enough; the library aggregates them into a single system prompt before each run.

Toolkits let you derive tools and extra prompt snippets dynamically. Each toolkit generates a per-session object that stays alive until you close the run session, which is useful for holding onto database connections or scoped caches.

interface Toolkit<TContext> {
  /**
   * Create a new toolkit session for the supplied context value.
   * The function should also intialize the ToolkitSession instance with instructions and tools.
   */
  createSession(context: TContext): Promise<ToolkitSession<TContext>>;
}

interface ToolkitSession<TContext> {
  /**
   * Retrieve the current system prompt for the session.
   */
  getSystemPrompt(): string | undefined;
  /**
   * Retrieve the current set of tools that should be available to the session.
   */
  getTools(): AgentTool<TContext>[];
  /**
   * Release any resources that were allocated for the session.
   */
  close(): Promise<void> | void;
}

pub trait Toolkit<TCtx>: Send + Sync {
    /// Create a new toolkit session for the supplied context value.
    /// Implementations should also initialize the session with instructions and
    /// tools.
    async fn create_session(
        &self,
        context: &TCtx,
    ) -> Result<Box<dyn ToolkitSession<TCtx> + Send + Sync>, BoxedError>;
}

pub trait ToolkitSession<TCtx>: Send + Sync {
    /// Retrieve the current system prompt for the session, if available.
    fn system_prompt(&self) -> Option<String>;
    /// Retrieve the current set of tools that should be available to the
    /// session.
    fn tools(&self) -> Vec<Arc<dyn AgentTool<TCtx>>>;
    /// Release any resources that were allocated for the session.
    async fn close(self: Box<Self>) -> Result<(), BoxedError>;
}

type Toolkit[C any] interface {
  // CreateSession creates a new toolkit session for the supplied context value.
  // Implementations should also initialize the session with any instructions or tools.
  CreateSession(ctx context.Context, contextVal C) (ToolkitSession[C], error)
}

type ToolkitSession[C any] interface {
  // SystemPrompt returns the current system prompt for the session if available.
  SystemPrompt() *string
  // Tools returns the current set of tools that should be available to the session.
  Tools() []AgentTool[C]
  // Close releases any resources that were allocated for the session.
  Close(ctx context.Context) error
}

Use toolkits sparingly for capabilities that need per-user initialisation. For simple helper functions that do not keep state, define them directly in tools instead.

Lifecycle with run sessions

Agents remain stateless. When you call agent.run, agent.run_stream, or agent.create_session, the library creates a RunSession that binds the agent parameters to the context you pass in. That session resolves dynamic instructions, opens toolkit sessions, and keeps them around for the duration of the run. When you close the run session, those resources are released.

See Agent vs Run session for a deeper dive into how the pieces fit together and when to choose each entry point.