How to Write a Prompt for an AI Voice Agent
A voice-agent prompt is not a chat prompt. The eight sections to write, how to phrase rules for the ear, a template, and how to test it on real phone calls.
Réponse rapide
Write a voice-agent prompt in eight short sections: role and objective, tone, context, pronunciations, tools, rules, conversation flow, and safety and escalation. Phrase rules as plain sentences written for the ear, keep facts and prices out of the prompt, and test it by placing real phone calls, not by reading a chat transcript. Version every change so you can compare and roll back.
A prompt that works in a chat window will often fail on a phone line. Callers interrupt, mishear, read out numbers and expect an answer in seconds. This guide is for the person who configures the agent: it covers what to write, how to phrase it, a template you can copy, and how to test and improve it. The structure follows the sections recommended in the OpenAI Realtime Prompting Guide (checked November 7, 2026), applied here to how you build an agent in DialerBee.
Why a phone prompt differs from a chat prompt
- There is no screen. The caller cannot scroll back, skim a list or click a link. Everything must be said once, clearly.
- People interrupt. A long answer gets cut off halfway. Short turns survive interruption; long ones do not.
- Numbers are spoken. Phone numbers, dates, times and amounts have to be said in a way that sounds natural. See numbers and dates in Arabic voice agents for the Arabic side.
- Hearing is imperfect. Noise and accents cause mishearing, so the agent must confirm what it heard before it acts.
- Silence feels long. A pause that is fine in text feels like a dropped call on the phone.
The eight sections of a voice-agent prompt
| Section | What goes in it | Keep out |
|---|---|---|
| 1. Role and objective | Who the agent is, for which business, and what a successful call looks like | Long company history |
| 2. Tone | Pace, warmth, formality, sentence length, one example of the voice | Adjective lists with no example |
| 3. Context | Today's situation: inbound or outbound, what the system already knows about the caller | Facts that belong in the knowledge base |
| 4. Pronunciations | Brand names, product names and tricky words, with how to say them | Words the voice already says well |
| 5. Tools | What each tool does, when to use it, what to say while it runs | Tools the agent should not use |
| 6. Rules | Do and do-not statements in plain sentences | Rules that repeat each other |
| 7. Conversation flow | The stages of the call, with a goal and exit condition for each | Scripts for every possible reply |
| 8. Safety and escalation | When to hand over to a person, what to refuse, what to say when unsure | Vague warnings |
In Agent Studio, you set the persona and role first because it decides how everything else reads. Sections 1 and 2 describe the persona. Facts belong in the knowledge base, and flow profiles decide how the conversation is structured. Keep the eight sections as the written brief behind those settings, so every change has a reason you can read back.
Write rules as plain sentences, not IF/THEN code
Rules written like code read badly and are brittle. Compare:
- Brittle: "IF intent == booking AND date == null THEN ASK date."
- Better: "If the caller wants to book and has not given a day, ask which day suits them."
The same applies to thresholds and symbols: spell them out in words. Write "more than two failed attempts" rather than a symbol. Prefer short bullets over paragraphs, because it is easier to see when two rules conflict. The OpenAI guide makes the same point (clear, short bullets outperform long paragraphs), and also notes that ambiguous or conflicting instructions degrade performance. Its own example of converting a symbol rule is to write "more than three failures" instead of "x > 3". Read your rules aloud once: if two of them could pull the agent in opposite directions, fix that before you test.
Keep facts out of the prompt
The prompt should say how to behave. The facts the agent speaks belong elsewhere:
- Hours, services, policies, locations: the knowledge base. The agent finds the right passage by semantic search. See how to build a knowledge base for a voice agent.
- Prices: the service catalogue. The price-truth guard means only catalogue prices can be spoken.
- Appointments: the connected calendar, checked on the call.
If a fact lives in the prompt and in a document, one of them will go stale. Single source, one owner.
Write for the ear
- Short turns. One or two sentences per turn, then stop and let the caller speak.
- One question at a time. "What day suits you, and which service, and what is your name?" fails. Ask them in sequence.
- Confirm before acting. Read back names, numbers, dates and times before booking or changing anything.
- Say numbers in groups. Phone numbers in short chunks, times as a person would say them.
- Vary the wording. Add a rule not to repeat the same acknowledgement every turn, or the agent sounds mechanical. The OpenAI guide calls this a variety rule.
- Handle unclear audio. Tell the agent to ask the caller to repeat rather than guess. The OpenAI guide has a section on instructions for unclear audio.
Conversation flow as states with clear exits
Describe the call as a short list of stages. For each stage, give a goal, what to collect, and the condition for moving on. For a clinic receptionist:
- Greet. Goal: say who is answering and ask how to help. Exit: the caller states a need.
- Understand. Goal: decide whether it is a booking, a change, or a question. Exit: intent is clear.
- Collect. Goal: the minimum details for that intent, one at a time. Exit: all details confirmed back.
- Act. Goal: use the tool, say what happened. Exit: the tool result is spoken.
- Close. Goal: ask if there is anything else, then say goodbye. Exit: caller says no.
Allow the caller to jump between stages. People change their mind; a flow that cannot go back feels like a phone tree.
Escalation, refusal and "I don't know"
Write these as exact lines, because they are where an agent is most likely to improvise badly.
- Unknown answer: "I don't have that in front of me. I can take your number and have a colleague call you back."
- Asked for a person: hand over without argument. See how human handoff works.
- Out of scope: say what you can help with, then offer the handoff.
- Asked if it is an AI: say yes, plainly. See AI call disclosure rules.
- Distress or emergency: name the exact emergency line to give, and hand over.
A copy-paste prompt template
This is an illustrative English template for a dental clinic receptionist. Replace everything in square brackets. It contains no prices on purpose.
ROLE AND OBJECTIVE
You are the virtual receptionist for [Clinic name]. You answer inbound calls.
A good call ends with the caller's need handled or passed to a person.
TONE
Warm, calm, and brief. One or two short sentences per turn.
Example: "Of course. Which day works best for you?"
CONTEXT
The caller is calling the clinic's main number. Use the knowledge base
for hours, services and policies. Use the service catalogue for any price.
PRONUNCIATIONS
Say "[Clinic name]" as "[how it sounds]". Say the doctor's title and name as
"[how it sounds]".
TOOLS
Calendar: check free times, then book, move or cancel.
Use the calendar before promising any time. While it runs, say "One moment."
RULES
- Ask one question at a time.
- Read back the name, day and time before you book.
- Only quote prices from the service catalogue. If a price is missing, say
a colleague will confirm it.
- Never give medical advice. Offer a call from the clinic team instead.
- If you are not sure what the caller said, ask them to repeat it.
- Do not repeat the same acknowledgement twice in a row.
CONVERSATION FLOW
1. Greet and ask how you can help.
2. Work out if it is a booking, a change, or a question.
3. Collect the details for that need, one at a time, and confirm them.
4. Use the tool and say what happened.
5. Ask if there is anything else, then say goodbye.
SAFETY AND ESCALATION
- If the caller asks for a person, or sounds upset, pass them to reception.
- If the caller describes an emergency, tell them to call [local emergency
number] and pass the call to reception.
- If asked whether you are an AI, say yes.
- If you do not know, say so and offer a callback. Test on real phone calls, not in a chat window
A transcript shows what the agent was meant to say. A phone call shows what it actually says, how fast, and how it handles an interruption. In Agent Studio, place a test call and listen. A good test set covers:
- the happy path for each intent;
- a caller who interrupts mid-sentence;
- a caller who changes their mind about the day;
- a question that is not in the knowledge base;
- a request for a person;
- a noisy line, or an unclear name.
For Arabic callers, use the Arabic voice agent testing checklist. Iterate in small steps: one change at a time, then listen again, because small wording changes can change behaviour a lot. The OpenAI guide puts it as "small wording changes can make or break behavior".
Version, compare and roll back
Every publish in Agent Studio is a version. Promote the one you want live, and roll back when today's change turned out worse than yesterday's. When you have two candidate prompts, run an A/B experiment against real traffic and read which one performed, instead of settling the argument by seniority. Keep a one-line note with each version saying what you changed and why.
Langues
The AI voice agent has run on live calls in Arabic and English. For any other language, test on a demo first. Pin the language the agent should speak, write the spoken example lines in that language, and say what to do if the caller switches. If your callers mix Arabic and English, read Arabic-English code-switching in voice AI. You can try an agent on the free trial, and the broader setup steps are in how to set up an AI receptionist. For inbound script ideas, see inbound call center scripts et le agent vocal IA page.
Foire aux questions
How long should a voice agent prompt be?
As short as it can be while covering the eight sections. A prompt the agent has to hold in mind on every turn works better as a few short bullets per section than as pages of prose. Move facts such as hours, policies and services into the knowledge base, and move prices into the catalogue, so the prompt carries behaviour only.
Should I write the prompt in the caller's language?
Write the rules in the language you can review most easily, and write the spoken example lines in the language the caller will hear. Pin the language the agent should speak, and say what to do if the caller switches. DialerBee's AI voice agent has run on live calls in Arabic and English. For other languages, test on a demo first.
Why does my agent sound fine in a chat window and wrong on the phone?
A chat window shows text. A phone call has timing, interruptions, mishearing and spoken numbers, so a reply that reads well can be too long to listen to. Place test calls and listen. Agent Studio test calls exist for exactly this reason: the ear is the acceptance test.
Should the prompt contain prices?
No. Put prices in the service catalogue. DialerBee's price-truth guard means only prices in your catalogue can be spoken, so the agent does not invent a number. If a price sits in the prompt as well, the two will drift apart the first time one is edited.
How do I know a new prompt is better than the old one?
Publish it as a new version, place test calls, and then compare against the previous version. Agent Studio keeps versions you can promote or roll back, and A/B experiments let you run two versions against real traffic and read the result, so you measure instead of arguing about wording.
What should the agent do when it does not know?
Say so plainly, offer a next step, and move on. A good rule is: if the answer is not in the knowledge base or the catalogue, say you will pass the question to a colleague, take the caller's details, and never guess. Write the exact refusal line into the prompt so it sounds natural.
Sources
- OpenAI Cookbook, Realtime Prompting Guide (section structure and prompting tips). developers.openai.com, Realtime Prompting Guide (consulté le 7 novembre 2026)
Lectures complémentaires
- Agent Studio
- How to Set Up an AI Receptionist
- AI Voice Agent Human Handoff
- Inbound Call Center Scripts
- Building a Knowledge Base for a Voice Agent
À lire aussi : AI Voice Agent Security: Can Callers Trick It? · Les appelants raccrochent-ils quand une IA dit qu'elle est une IA ?
Articles connexes
Arabic Customer Service Phrases for Agents
10 min de lecture
GuidesArabic Voice Agent Testing Checklist Before You Go Live
12 min de lecture
RecouvrementArabic Debt Collection Call Script (With English Notes)
9 min de lecture
TechnologieArabic-English Code-Switching in Voice AI and Transcription
9 min de lecture
Prêt à voir DialerBee en action ?
Demandez une démo en direct de 15 minutes, ou démarrez un essai gratuit de 14 jours — sans diaporama, sans engagement. Les appels démarrent dès que votre propre trunk SIP ou PBX est connecté et approuvé.
Essai gratuit de 14 jours · crédit IA gratuit · sans carte · 11 langues · BYOC · contrôles qui contribuent à la conformité