What Is Function Calling in LLMs?
Function calling lets an LLM ask your code to run a tool and use the result. Learn the step-by-step flow, a runnable example and the safety rules.
Short answer: function calling (also called tool calling) is a way for a large language model (LLM) to ask your program to run a function instead of only writing text. You describe the available tools, the model replies with the tool name and arguments in JSON, your code runs it, and you send the result back so the model can answer. The model never runs anything itself.
Why Do LLMs Need Function Calling?
An LLM only knows its training data and cannot act on its own. It cannot check today's weather, query your database or send an email. Function calling lets it request those actions while your code stays in control. It is the basic building block of AI agents.
How Does Function Calling Work, Step by Step?
Describe the tools. For each function, you send a name, a short description and the arguments it accepts (usually as a JSON schema).
The user asks something. “What's the weather in Rivertown?”
The model decides. If a tool would help, it replies with a tool call, not a final answer: the tool name and arguments as JSON.
Your code runs the tool. You check the request and call the real function.
You return the result to the model in the next message.
The model writes the final answer using that result.
Exact field names differ between providers, so use your provider's documentation for the real format. The idea is the same everywhere.
What Does It Look Like in Code?
This runnable sketch simulates only the middle part: reading a tool call and running it safely. The model's reply is hard-coded here so it needs no account.
import json
def get_weather(city):
data = {"Rivertown": "18C, cloudy", "Lakeside": "25C, sunny"}
return data.get(city, "unknown")
TOOLS = {"get_weather": get_weather} # only tools we allow
# What a model might return when it decides to call a tool
model_reply = '{"tool": "get_weather", "arguments": {"city": "Lakeside"}}'
call = json.loads(model_reply)
func = TOOLS[call["tool"]]
result = func(**call["arguments"])
print(result) # 25C, sunny (sent back to the model)
Function Calling vs RAG vs MCP
Idea | What it does |
|---|---|
Function calling | The model requests an action; your code runs it. |
Finds relevant documents and adds them to the prompt. | |
A shared standard for plugging tools into AI apps. |
Retrieval can itself be offered as a tool, so the ideas combine.
Safety Rules
Run only tools from an allow-list, as in the code above.
Validate arguments; the model can send wrong or unexpected values.
Ask a person to confirm risky actions such as payments, deletions or emails.
Give each tool the least permission it needs, and treat text coming back from tools as untrusted.
Common Mistakes
Vague tool descriptions, so the model picks the wrong tool.
Believing the model executes the function. Your code does.
No error handling when a tool fails or returns nothing.
Too many tools at once, which makes choosing harder.
Interview tip: describe the loop (describe, call, run, return, answer) and mention validation and permissions. The free chapter Tool contracts and execution covers it in depth.
Key Takeaway
Function calling lets an LLM request actions as structured JSON while your program decides what actually runs. Clear tool descriptions, validation and limited permissions make it safe and reliable.
Ready to start? Build LLM apps step by step, including tools, context, cost and error handling. Begin with the free course AI Engineering with LLMs: Beginner.
Related articles
More in AI Engineering
How to Get an LLM API Key (OpenAI and Gemini) and Understand the Cost
Where to create an OpenAI or Gemini API key, how token-based billing works, how to estimate cost and how to keep your key safe.
4 min readNov 11, 2026What Is LangChain? When to Use It (and When Not To)
LangChain is a framework for building LLM apps. Learn its core building blocks, a short sketch, and when plain code is the better choice.
4 min readNov 4, 2026How Do You Evaluate an AI or LLM System?
Building an AI system is not enough—you need to measure how accurately, reliably, and safely it performs. Learn the key metrics and methods used to evaluate modern AI and LLM applications.
2 min readSep 6, 20267 AI Project Ideas for Your Portfolio
Seven small, finishable AI project ideas for a beginner portfolio, with the skill each one teaches and how to present it so others trust your work.
4 min readNov 14, 2026What Is Prompt Injection?
Prompt injection tricks an AI app into following an attacker's instructions. Learn the types, why it is dangerous and the layers of defence that reduce the risk.
4 min readNov 12, 2026What Is a System Prompt and Why Does It Matter?
A system prompt is the standing instruction that sets an AI model's role, rules and format. Learn how it differs from a user prompt, what to put in it and its limits.
3 min readNov 10, 2026
Comments
Sign in to join the discussion.



