Function & Tool Calling

A tool definition tells the model what capabilities are available. The schema usually includes a stable name, a clear description, and parameters expressed as JSON Schema: object type, properties, required fields, allowed enums, descriptions, and sometimes additionalProperties restrictions.

The schema is not just documentation. It shapes the model choice of whether to call a tool and what arguments to produce. Good schemas use precise names, narrow fields, explicit allowed values, and descriptions that say when the tool should and should not be used. Overbroad tools make the model guess; narrow tools make failures easier to validate and repair.

When the model decides a tool is needed, it returns a structured assistant message rather than a final user-facing answer. In OpenAI-style APIs, the response contains tool_calls and commonly ends with finish_reason equal to tool_calls. Each call has an id, a function name, and JSON arguments.

That response is a request for your application to do work. It is not proof that the call is allowed, safe, or valid. The application should parse the JSON, validate it against the tool schema, authorize the operation for the current user, execute the mapped server-side function, and record the result.

The full loop has four phases. First, the app sends messages and tool schemas to the provider. Second, the assistant returns one or more tool calls with JSON arguments. Third, the app executes the requested tools and appends tool result messages tied to the original call ids. Fourth, the app calls the model again so it can use the tool results to produce a final answer or request another tool.

This loop can repeat. A simple support answer may need one account lookup. An agentic workflow may observe a result, decide the next action, call another tool, and continue until it reaches a stop condition, budget limit, or final answer.

Some providers allow the assistant to request multiple independent tools in one response. For example, a support assistant might ask for customer profile, recent invoices, and policy search at the same time. If the calls do not depend on each other and are safe to run concurrently, the app can execute them in parallel to reduce latency.

The important rule is correlation. Each returned tool result must reference the matching tool call id or provider-specific block id, and the app should append all results before asking the model to continue. If one call fails, return a structured error result for that call or retry according to policy instead of silently dropping it.

Models can invent tool names, omit required fields, pass the wrong type, choose impossible enum values, or include arguments that look plausible but violate business rules. JSON Schema validation catches structural issues, but it does not prove that the user is authorized or that the requested action is safe.

A robust implementation validates in layers: tool allowlist, JSON parse, schema validation, business validation, authorization, idempotency checks, and safe execution. Invalid arguments can trigger a repair turn, a user clarification, or a refusal. Sensitive actions should not be fixed by guessing missing values.

Most providers expose a way to control tool use. auto lets the model decide whether to call tools. required forces at least one tool call. A specific tool choice forces one named tool. none disables tools for a turn, which is useful for pure summarization or when rendering a final answer from already gathered evidence.

The names differ by provider. OpenAI uses tools and tool_calls. Anthropic represents tool requests as tool_use content blocks and results as tool_result content blocks. Gemini uses functionCall and functionResponse parts. The message dialect changes, but the application-owned loop and security boundary stay the same.

A tool should represent one clear capability. Names should be action-oriented, descriptions should include when to use the tool, and schemas should encode constraints with required fields, enums, length limits, and additionalProperties set to false when possible.

Avoid generic tools such as execute_query or call_api unless they are heavily constrained by server-side policy. Broad tools give the model too much room to invent parameters and make it harder to reason about access control. A good tool surface looks like a small product API, not a bag of arbitrary powers.

Parallel tool calls are useful when the model needs several observations that do not depend on one another. Running customer lookup, policy search, and invoice retrieval concurrently can save a full round trip compared with asking the model to request them one by one.

Do not parallelize calls that have ordering constraints, shared mutable state, or side effects that must happen conditionally. For mutating actions, prefer explicit planning, user confirmation, idempotency keys, and sequential execution so the app can stop safely after a failed or denied step.

A failed tool call can be returned to the model as a structured result saying the lookup timed out, the id was not found, or the user is not authorized. That lets the model explain the issue, ask for clarification, or choose another allowed tool.

The app should still distinguish retryable infrastructure failures from permanent validation failures. Retry transient network errors with backoff. Do not retry unsafe mutations blindly. Do not expose raw stack traces, secrets, internal table names, or policy details in tool result content.

Tool calling usually requires at least two model calls: one to request the tool and one to answer after receiving results. Agentic loops can require many more. Each loop adds provider latency, token cost, tool latency, and operational failure modes.

Reduce cost by exposing only relevant tools, keeping result payloads compact, summarizing large tool outputs before sending them back, caching read-only observations when safe, and setting a maximum step budget. A tool loop should be measured as an end-to-end transaction, not just a model call.

OpenAI commonly represents tools as a top-level tools array and tool calls as assistant message fields. Anthropic represents tool use and tool result as content blocks inside messages. Gemini represents function calls and function responses as parts. Some providers return arguments as JSON strings, while others expose structured objects.

Build an internal abstraction that normalizes provider events into tool requested, tool result appended, final answer, and error. That keeps business logic independent from provider syntax while still preserving provider-specific ids needed to correlate results.

Tool calling is the primitive. An agent is a policy loop that repeatedly decides what to do, calls tools, observes results, updates state, and continues toward a goal. Without tools, most agents are just chat prompts with memory. With tools, they can inspect live state and take bounded actions.

MCP, the Model Context Protocol, standardizes how applications expose tools, resources, and prompts to AI clients. MCP can make tool discovery and integration cleaner, but it does not remove the need for application-level authorization, validation, observability, and least-privilege tool design.

Treat tools as real capabilities. Keep an allowlist of exposed tools, map tool names to server-owned functions, authorize every call server-side, and never trust model-provided arguments for sensitive actions. User confirmation is appropriate for irreversible writes, purchases, emails, deletes, permission changes, and financial operations.

Log the model request id, tool schema version, tool_choice mode, selected tool name, validated arguments, execution outcome, latency, token usage, and final answer. For sensitive domains, record who authorized the action and what user-visible confirmation was shown.

Read-only tools can often be retried after transient failures. Mutating tools need idempotency keys, deduplication, timeouts, and clear rollback or compensation behavior. The app should cap loop depth so a model cannot spin forever calling tools after repeated failures.

Return only the fields the model needs to answer. Large raw records increase tokens, leak unnecessary data, and invite reasoning over irrelevant details. Shape tool results into small, explicit objects with safe error messages and redact secrets before appending them to the conversation.

This example shows the core loop without calling a real provider. The fake provider first returns a tool call, the app validates and executes it, then the fake provider returns a final answer after seeing the tool result.

When the assistant requests independent tools in one turn, execute them concurrently and append one result message per call id before continuing the model loop.