Function calling for support bots

The mechanics

You pass a list of function definitions alongside the messages. Each has a name, a description the model uses to decide relevance, and a JSON Schema for its parameters. If the model decides a call is warranted, the response contains a tool call rather than prose.

You then execute it, append the result to the conversation, and call the model again to compose the customer-facing answer. Two round trips per answered tool question.

{
  "type": "function",
  "function": {
    "name": "lookup_order",
    "description": "Look up the status of an order for the current customer.",
    "parameters": {
      "type": "object",
      "properties": {
        "order_id": { "type": "string" }
      },
      "required": ["order_id"]
    }
  }
}

The model is not an authorisation layer

The single most important design decision: never rely on the prompt to stop a tool being misused. A system prompt saying "only look up orders belonging to this customer" is a suggestion. The context contains crawled web content and customer messages, either of which may be adversarial.

Authorisation belongs outside the model, at the point where tools are selected. AskBrew resolves which tools a visitor may use before the model sees anything: identity-gated tools are filtered out entirely for an unverified visitor. The model is never told they exist, so there is nothing for an injection to talk its way into calling.

This is the difference between a guardrail and a boundary. A guardrail asks the model not to. A boundary means the capability is absent.

Proving who the visitor is

A widget runs in the browser, so anything it claims about the visitor is a claim, not proof. "customer_id: 12345" from client JavaScript is trivially editable, and using it to fetch order data means anyone can read anyone's orders by incrementing a number.

Proof has to come from the customer's server. AskBrew accepts a signed JWT, or an HMAC hash of the customer ID, generated server-side with a shared secret. The signature is verified before any identity-gated tool is offered. An unverified identity is still accepted for cosmetic personalisation — greeting someone by name — but it never unlocks a data lookup.

Private claims from the token are kept separate from anything that enters the prompt, so information used for authorisation is not incidentally exposed to the model or echoed into an answer.

Scope the parameters, not just the call

Even with a verified visitor, a tool taking an arbitrary order ID is a lookup for any order in the system. The customer identity should come from the verified context on the server side, not from an argument the model filled in.

The practical rule: model-supplied arguments are for disambiguation within what the visitor may already see. Anything that determines whose data is returned comes from the verified session.

Tool output is untrusted too

The result comes back from an external API and goes straight into the prompt. If a field contains text that reads as an instruction — a product name, a customer-entered delivery note — a naive implementation will let it act as one.

Delimit tool results the same way as retrieved content, and say in the system prompt that everything inside the delimiters is data. AskBrew wraps results in an explicit tag with the call status attached.

<tool_result status="ok">
{ "order_id": "1042", "status": "shipped", "eta": "2026-07-29" }
</tool_result>

Operational limits worth setting early

• —One tool call per answer. Chained calls make cost and latency unpredictable and debugging much harder.
• —A timeout per tool. A customer's slow endpoint should degrade to a normal answer plus a handover offer, not a hung request.
• —Bill completed calls, not failed ones. A timeout is your customer's infrastructure failing, and charging for it is indefensible.
• —Log every invocation with arguments, status, duration and HTTP code. This is the first thing you will want when a lookup misbehaves.
• —Encrypt stored credentials at rest, and never let them reach the model.

When not to bother

Function calling adds a round trip, a failure mode, and an authorisation surface. If your support volume is genuinely dominated by questions your documentation answers, a well-tuned content assistant will deflect more tickets per unit of effort.

Add tools when you can point at a specific high-volume question that content cannot answer. For most ecommerce teams that question is order status, and it is usually worth it. For a documentation-heavy SaaS product, often it is not.