Learn

Tool calling: how a model asks your code to act

Tool calling, also called function calling, is a model's ability to reply with a structured request (a tool name plus JSON arguments) that your program executes, then feed the result back to the model. The model never runs anything itself: it asks, your code acts, and the answer goes back into the conversation. Every coding agent and MCP integration is built on it.

At a glance

What it is
A structured reply: tool name plus JSON arguments, not prose
Who runs the tool
Your program; the model only asks
Where it breaks
Wrong JSON, invented tools, or a server parser that does not match the model
Why it matters here
Every agent lane in a fleet lives or dies by it
Flow

One tool-calling round

The model proposes, your code decides and executes, the result goes back.

1
Model replies with a tool name and JSON arguments
2
Your code validates the call, runs the tool, captures the result
3
Model again reads the result and answers or calls the next tool

Why does a model need a special format to use tools?

Because your program has to understand the request without guessing. A sentence like “I would now read the file config.py” is ambiguous; a call with the name read_file and the argument {"path": "config.py"} is not. Models are trained to emit that structure when you list tools in the request, and the serving stack (vLLM, SGLang, an API) extracts it into a tool_calls field. When the model’s format and the server’s parser disagree, tool calls silently arrive as plain text and an agent looks broken even though the model is fine.

Why is tool calling a security question?

Because the model decides what to ask for, and anything it reads can try to steer that decision. A web page or a file can contain instructions aimed at the model. Your code is the only place that can say no: validate arguments, give each tool the least power that works, and run agents that execute commands inside a sandbox, so a bad call cannot reach what the task never needed.

Check it yourself

curl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "your-model",
  "messages": [{"role": "user", "content": "What is 2+2? Use the tool."}],
  "tools": [{"type": "function", "function": {"name": "add",
    "parameters": {"type": "object", "properties": {"a": {"type": "number"}, "b": {"type": "number"}}}}}]
}' | jq '.choices[0].message.tool_calls'

A model and server that support tool calling return a tool_calls array here; null or plain text means the parser or the model is not doing it.

Tool calling gives you

  • A model that can read, search and edit instead of only talking
  • Machine-checkable output: a call either parses or it does not
  • One interface for many tools, from a calculator to a shell

It asks of you

  • A serving stack whose tool parser matches the model's format
  • Validation of every argument before anything runs
  • Limits on what a tool may touch, because the model can be wrong or manipulated

Related terms

Go deeper

← All terms Reviewed: October 2026