How to A/B Test Outbound Call Campaigns
How to A/B test outbound call campaigns: what to test, one metric per test, randomising the list, sample size before you start and reading results honestly.
Kurze Antwort
To A/B test call scripts, change one thing, pick one metric, split the list at random into two equal groups and run both at the same times with the same agents. Work out the sample size before you start, do not stop early, then compare the two proportions with a z test. Keep every variant compliant.
Many outbound teams change a script, watch results for a week and decide it "feels better". The trouble is that call results swing for reasons unrelated to the script: a different list, a different day, a stronger team on shift. Without a proper split, you cannot tell a real improvement from noise.
A/B testing a call campaign is not complicated, but it has rules, and calling has traps that email tests do not. This guide covers what you can test, how to choose a single metric, how to randomise, how to size the test before dialling, how to read the result in plain words and what compliance expects while you experiment.
What you can test on a call campaign
Each of these is a candidate for a test, and each affects a different part of the funnel.
| What you change | Metric it should move | Anmerkungen |
|---|---|---|
| Opener (first 10 seconds) | Conversation continues, then conversion per contact | See the script templates |
| Offer or price framing | Conversion per contact | Only with approved, accurate offers |
| Voicemail message | Callbacks or later contact | Sehen voicemail drop practices |
| Calling window | Kontaktrate | Stay inside permitted hours |
| Dialing mode or pacing | Agent talk time, contacts per hour | Watch abandon limits; see the Power Dialer |
For ideas that raise the chance of an answer in the first place, see Wie man die Quote ausgehender Kontakte verbessert, and for the wider habits around testing and coaching, the outbound calling best practices playbook.
One metric per test: contact rate or conversion per contact, not both
Choose the metric before the test starts and write it down. Two metrics matter most, and they answer different questions.
- Kontaktrate: contacts divided by dials. It tells you whether people answer. It is affected by timing, caller ID, list quality and answering-machine handling, and hardly at all by what an agent says after the greeting. See the glossary entry on Kontaktrate.
- Conversion per contact: sales, appointments or promises to pay divided by live conversations. It tells you whether the conversation worked. It is affected by the script, the offer and the agent, and not by whether the phone was answered.
Mixing them causes confusion. A new opener cannot change the contact rate, because the person has not heard it yet. A new calling window cannot sensibly be judged by conversion alone. If you measure per dial instead, you blend both effects and cannot tell which one changed. Definitions of related measures are in the KPI formulas guide.
The danger of looking at five metrics after the fact is real: with enough numbers, one will favour either version by chance. Name a primary metric. Treat the rest as context.
Randomise the list, not the agents' choice
Randomisation is what makes the comparison fair. If agents choose which script to use, they will use the new one on the leads they like. If you split by alphabet, region or upload date, you are comparing different kinds of people.
- Take the full list for the test and remove contacts that fail consent or do-not-call checks first, so both groups start clean.
- Add a column of random numbers (a spreadsheet RAND function is enough), sort by it and cut the list in half. Alternatively, assign contacts alternately after sorting by the random number.
- Check that the halves look alike on the things that matter: number of contacts, time zones, age of lead, source. If one half has most of the best leads, re-randomise.
- Upload each half as its own list, attached to its own campaign: Campaign A with script A, Campaign B with script B.
Keep records of which contact went to which group. You will need them to explain any odd result.
Work out the sample size before you start
This is the step call tests often skip, and it decides whether the test can ever answer your question. A small test can only detect a big difference; a real but modest improvement will look like noise.
The NIST/SEMATECH e-Handbook of Statistical Methods gives the method for sizing a test on a proportion by normal approximation, using the significance level, the power, the baseline rate and the difference you want to detect, with a continuity correction added (NIST/SEMATECH e-Handbook, section 7.2.4.2, checked November 7, 2026). The handbook's worked example is a single-process one, detecting a rise from 10 percent to 20 percent defective. Comparing two scripts is a two-group version of the same approach, and the figures below are our own computation using the standard two-group form of that normal approximation.
Illustrative example. Your baseline conversion is 10 percent of live conversations, and you want to be able to detect a rise to 14 percent. Use a 5 percent significance level (two-sided) and 80 percent power.
- Without correction: about 1,035 live conversations per script.
- With a continuity correction: about 1,085 per script.
- For a smaller rise, 10 to 12 percent, the need rises to roughly 3,840 to 3,940 per script.
- For a larger rise, 10 to 20 percent, about 200 to 220 per script.
Note the unit: these are conversations, not dials. If only 20 percent of dials reach a person, 1,085 conversations means about 5,400 dials per script. If that is more list than you have, test a bigger change, run the test longer, or accept that the test can only detect large effects. Writing this down first prevents the common mistake of treating a weak test as a negative result.
Running the test: same windows, same agents, no peeking
- Run both arms at the same time. Not script A this week and script B next week. Time-based differences, such as pay day, weather or a news event, would pollute the result.
- Use the same calling windows for both campaigns, so each sees the same hours.
- Use the same agents if you can. If agents are dedicated to one campaign each, agent skill becomes part of the result. Where your set-up allows it, let every agent work both campaigns, or swap the agents halfway.
- Do not change anything mid-test. If you must edit a script, restart.
- No peeking at winners. Look for operational problems if you need to, but do not stop because version B is ahead on day two. Early leads can shrink or reverse. Stop at the planned number.
Reading the result: the two-proportion test in plain words
When the planned sample is in, you have two proportions. The question is whether the gap between them is bigger than chance would typically produce. The NIST handbook sets this out as a z test: pool the two groups to get an overall rate, work out how much random variation to expect from groups of your sizes, and divide the observed gap by that expected variation. Reject "the two rates are equal" when the absolute z value exceeds the critical value for your significance level (NIST/SEMATECH e-Handbook, section 7.3.3 on testing proportions, checked November 7, 2026). For a two-sided 5 percent test that critical value is about 1.96, from the standard normal table (NIST/SEMATECH e-Handbook, section 1.3.6.7.1, checked November 7, 2026).
Worked example, illustrative. Script A: 60 conversions from 500 conversations (12.0 percent). Script B: 85 from 500 (17.0 percent). The pooled rate is 145 out of 1,000, or 14.5 percent. The z statistic is about 2.25, which is above 1.96, so the gap is unlikely to be chance at the 5 percent level.
Second example. Script A: 70 from 500 (14.0 percent). Script B: 92 from 500 (18.4 percent). The pooled rate is 16.2 percent and z is about 1.89, below 1.96. B looks better, but the test does not reach the bar. That is not proof that the scripts are equal; it means you have not shown a difference. Either run more contacts or accept that the effect, if any, is small.
The handbook uses the normal approximation when the samples are reasonably large; for small samples it points to Fisher's exact test, which works on the two-by-two table of outcomes (NIST/SEMATECH e-Handbook, section 7.3.3). A spreadsheet or statistics package will calculate both. Also report the size of the difference, not only whether it passed: a statistically significant gain of half a point may not repay a script change.
Confounders that are specific to calling
| Confounder | How it spoils a test | Bewachen |
|---|---|---|
| Agent skill | The strong agents end up on one script | Same agents on both, or swap halfway |
| Time of day | One arm called in better hours | Run in parallel, same windows |
| List age and source | Fresher contacts answer and buy more | Randomise and check balance |
| Anrufbeantwortererkennung | Different handling of machines changes who counts as a contact | Identical AMD settings in both campaigns |
| Abandon caps and pacing | Different pacing changes wait time and caller mood | Same dialing mode and limits |
| Retry rules | More attempts on one arm raises its contact rate | Same attempts and intervals |
If you are testing a dialing mode or pacing change deliberately, then of course those settings are the variable, and everything else should be held fixed. The playbook explains the trade-offs between modes.
Compliance guardrails during a test
- Every variant is an approved script. A test is not a licence to improvise claims. Both versions need the same review.
- Required disclosures stay in. Do not test a version that leaves out identification, recording notices or other mandatory statements.
- Consent, DNC and calling hours apply equally. Scrub the whole list before randomising, and apply the same permitted-hours rules to both arms. The calling hours, DNC and consent guide covers the settings.
- Honest offers only. Testing two prices or promises means both must be real and available.
- Keep records. Store the script versions, dates and groups, so you can show what was said to whom.
Running a test with two campaigns and the reports in DialerBee
The product pages do not describe a built-in split-test feature, so the method here uses what they do describe: separate campaigns and separate lists, and reporting. Create two campaigns with identical settings and dialing mode, upload one randomised half of the list to each and load script A and script B. Then compare them in the Berichte: the Campaign Summary report covers how a campaign performed from dials to connections to dispositions, with the Agent Performance and Lead Conversion reports beside it, and each exports as CSV, Excel or PDF. The Analysen page adds campaign comparison, agent metrics and scheduled CSV exports, which suit a test that runs for days.
Use the dispositions you already track as the conversion event, so both arms are measured the same way. Export both campaigns' final numbers at the planned sample size, apply the test above and note the result with the date, the scripts and the sample.
Häufig gestellte Fragen
How do you A/B test a cold call script?
Split one list at random into two equal groups, give each group a different script, run both under the same calling windows with the same pool of agents, and compare one pre-chosen metric once the planned number of contacts is reached. Decide the metric, the sample size and the stopping point before the first dial.
How many calls do I need for a script test?
It depends on your baseline rate and the smallest improvement you care about. As an illustration, detecting a rise from a 10 percent to a 14 percent conversion per contact with a 5 percent significance level and 80 percent power needs roughly 1,035 contacts per script, or about 1,085 with a continuity correction. Smaller differences need far more.
Which metric should I use?
Choose one. If you are testing something that affects whether people pick up, such as the calling window or the caller ID, use contact rate. If you are testing what is said once someone answers, such as the opener or the offer, use conversion per contact. Do not pick the winner afterwards from whichever number looks best.
Can I look at the results while the test is running?
You can monitor for problems, but do not stop the test the first time one script is ahead. Repeated early looks make a chance difference appear real. Fix the sample size in advance and read the result when it is reached.
Does DialerBee have a built-in split test feature?
The product pages do not describe one. You run a test with two campaigns and two lists, split at random before upload, and compare them using the campaign, list and agent reports and CSV exports.
Is it compliant to test different scripts?
Testing does not change your obligations. Every variant must be an approved script, every contact must still pass consent, do-not-call and calling-hour checks, and required disclosures must appear in both. This is general information, not legal advice.
Quellen
- NIST/SEMATECH e-Handbook of Statistical Methods, section 7.3.3 (z test for two proportions; Fisher's exact test for small samples). itl.nist.gov, section 7.3.3 (abgerufen am 7. November 2026)
- NIST/SEMATECH e-Handbook of Statistical Methods, section 7.2.4.2, sample sizes required for a proportion (normal approximation with continuity correction). itl.nist.gov, section 7.2.4.2 (abgerufen am 7. November 2026)
- NIST/SEMATECH e-Handbook of Statistical Methods, section 1.3.6.7.1, cumulative distribution function of the standard normal distribution (critical value 1.960). itl.nist.gov, section 1.3.6.7.1 (abgerufen am 7. November 2026)
- Sample size and z statistic figures in the examples were computed by DialerBee for this article and are illustrative only.
Weiterführende Lektüre
- Outbound call script templates
- Outbound calling best practices playbook
- How to improve outbound contact rates
- Voicemail drop best practices
- Call center KPIs and formulas
- Berichte und Erkenntnisse
Haftungsausschluss: This article is general information, not legal advice or statistical consultancy. DialerBee does not provide legal advice or guarantee regulatory compliance. Calling-hour, consent and disclosure rules differ by country, and a test must follow them like any other campaign. Confirm requirements with qualified counsel.
Verwandte Artikel
Bereit, DialerBee in Aktion zu sehen?
Fordern Sie eine 15-minütige Live-Demo an oder starten Sie eine 14-tägige kostenlose Testphase – ohne Folien, ohne Verpflichtung. Das Telefonieren beginnt, sobald Ihr eigener SIP-Trunk oder Ihre TK-Anlage verbunden und freigegeben ist.
14-tägige kostenlose Testphase · kostenloses KI-Guthaben · keine Kreditkarte · 11 Sprachen · BYOC · Kontrollen zur Unterstützung der Compliance