Arabic-English Code-Switching in Voice AI and Transcription
Gulf callers mix Arabic and English mid-sentence. How code-switching affects speech recognition and voice agents, which research datasets exist, what to test.
Kurze Antwort
Arabic-English code-switching means callers move between the two languages mid-sentence. It is hard for speech recognition because vocabulary, sounds and often script change within one utterance, and research on Emirati-English speech reports weak results from pre-trained systems. Public datasets such as Mixat and ZAEBUC-Spoken exist, but you should test with your own callers before launch.
Ask a Gulf contact centre what language its callers speak and the honest answer is often "both". A caller starts in Arabic, gives an account number in English, names a product in English, and finishes with an Arabic confirmation. For people this is natural. For speech recognition and voice agents it is one of the most common ways a good demo turns into a disappointing live call. This guide explains what code-switching looks like, why it causes trouble, which research resources exist, and what to test before you put a voice agent on real calls.
What code-switching looks like on real Gulf calls
The examples below are written for illustration. They are not quotes from any customer or any recording.
| Pattern | Illustrative example | Warum es wichtig ist |
|---|---|---|
| Brand and product names | An Arabic sentence that contains an English plan name or app name | The name must be recognised as one unit and matched to your catalogue |
| Numbers and dates | A reference number read digit by digit in English inside an Arabic sentence | One wrong digit breaks a lookup or a booking |
| Technical and business terms | English words such as "invoice", "booking" or "upgrade" inside Arabic | Intent detection depends on catching the key word |
| Email addresses and codes | Spelled-out letters in English during an Arabic call | Needs Latin characters in the transcript |
| Confirmations and fillers | An English sentence that ends with an Arabic "yes" or a short phrase of thanks | A short word decides whether the caller agreed |
| Arabic written in Latin letters | Customers who type Arabic with numerals and Latin characters in chat or SMS, a style often called Arabizi | Affects any text channel that sits next to the voice channel |
Why it breaks speech recognition
Four things change at once when a speaker switches, and each is a place to test.
- Language identification. A system that decides the language once at the start of a call, or once per utterance, will misjudge a call that changes language several times. The switch can happen inside a phrase, not only between sentences.
- Vocabulary. The set of likely words changes. English words said with an Arabic accent, or Arabic words said with an English accent, sit outside the clean pronunciation a model learned from monolingual data.
- Dialect. Gulf, Levantine, Egyptian and Maghrebi Arabic differ, and a standard written form (Modern Standard Arabic) is spoken differently again. Code-switching stacks on top of dialect.
- Script. The output has to choose Arabic script or Latin script for each word, and a consistent choice is needed for search, analytics and downstream systems.
Research backs the difficulty. The authors of the Mixat dataset report that the pre-trained Arabic and multilingual speech recognition systems they evaluated struggle with this low-resource dialectal Arabic, with code-switching adding a further challenge. (Mixat paper, checked November 7, 2026.)
Research datasets worth knowing
You do not need to run academic benchmarks to run a contact centre, but the datasets show what the field has measured and give you a vocabulary for your own test.
| Resource | What it is | Reported scale and scope |
|---|---|---|
| Mixat | A dataset of bilingual Emirati-English speech for speech recognition | 15 hours, from two public podcasts with native Emirati speakers, with code-switching in formal and natural conversational contexts |
| ZAEBUC-Spoken | A multilingual, multidialectal Arabic-English speech corpus, accepted to LREC-COLING 2024 | About 12 hours of Zoom meetings in which speakers role-play a work situation, with Modern Standard, Gulf and Egyptian Arabic and English with a range of accents, plus transcription guidelines for code-switching and orthography |
| SAGE | A method that splices audio to generate synthetic code-switched training speech for Arabic-English recognition (IEEE MLSP 2025) | Abstract reports a 7.8% absolute word error rate improvement on Arabic-English code-switched benchmarks, and a WER of 31.1% on them |
Two points stand out. First, ZAEBUC-Spoken adds annotation for how much Arabic varieties mix, and publishes guidelines for transcribing code-switching, which is useful if you need to write your own conventions. Second, the SAGE result is a research figure on research benchmarks: a WER of 31.1% shows that even a strong research system leaves a large error rate on Arabic-English code-switched speech. Do not read it as a figure for any commercial product, including ours. (Mixat, ZAEBUC-Spoken and SAGE papers on arXiv, checked November 7, 2026.)
Effects on voice agents
A voice agent adds three steps after recognition, and code-switching affects each.
- Understanding. The agent has to extract intent and details such as names, dates and numbers from a mixed transcript. A misrecognised English product name inside an Arabic sentence is enough to send the agent down the wrong path.
- Replying in the caller's mix. A caller who mixes languages may find a pure-Arabic or pure-English reply stiff, while a reply that mixes badly sounds wrong. Decide whether the agent mirrors the caller or stays in one language, and test that choice with listeners.
- Speaking English words inside Arabic speech. Text-to-speech has to pronounce an English brand name or number inside an Arabic sentence in a way that sounds natural to a Gulf listener. This is a listening test, not something a log can prove. If a caller says it sounded wrong, it did.
Transcript conventions
Agree these before you analyse a single call, because inconsistent conventions make search and reporting unreliable.
- Script for English words: Latin for brand names, codes and email addresses; Arabic script for ordinary borrowed words, or Latin throughout. Choose one rule.
- Numbers: digits or words, and which numeral shapes.
- Spelled-out items: how to write letter-by-letter spelling.
- Unclear speech: a marker for unintelligible words, not a guess.
- Language tags: whether each switch is marked, so you can measure by language.
The test set to build before launch
You do not need thousands of hours. You need a small, honest sample that looks like your calls.
- List your call types (for example billing query, appointment, collections reminder, complaint).
- Write 10 to 20 utterances per call type that include the switching patterns in the table above. Include your own product names, plan names and the numbers and dates callers actually give.
- Cover your callers. Record speakers who match your caller base in dialect and accent. Include fast speech, background noise and a mobile line, because a clean studio recording is not a phone call.
- Include hard cases on purpose: a switch inside a phrase, a spelled email address, a reference number, an English word with an Arabic accent and an Arabic word with an English accent.
- Score the right things. Check whether the agent captured the right intent and the right details, not only whether the words match. Count how often a human would have had to correct it.
- Listen to the replies, especially English words inside Arabic speech.
- Repeat after every change to the agent's script, voice or knowledge base, and again on real calls once live. A rising score on a test set is not proof of a better live call.
Wie DialerBee passt
DialerBee's KI-Sprachagent has run on live calls in Arabic and English, and the Arabic AI voice agent page sets out what it does with Gulf and other Arabic dialects. For what the agent does in Arabic today, read AI voice agent: what it does today in Arabic. Any other language should be tested on a demo first. Call Transkription is a separate feature that turns calls into searchable text, with 11 languages listed on that page.
This page makes no accuracy claim for mixed Arabic and English speech. How well any system handles your callers depends on their dialect, accent, line quality and vocabulary, which is why the test set above is the right first step. If your operation is multilingual by design, the guide to multilingual AI outbound calling covers the wider picture, and a demo with your own scripts is the quickest way to see how the agent behaves.
Quellen
- Mixat: A Data Set of Bilingual Emirati-English Speech, arXiv:2405.02578. arxiv.org/abs/2405.02578 (abgerufen am 7. November 2026)
- ZAEBUC-Spoken: A Multilingual Multidialectal Arabic-English Speech Corpus, arXiv:2403.18182. arxiv.org/abs/2403.18182 (abgerufen am 7. November 2026)
- SAGE: Spliced-Audio Generated Data for Enhancing Foundational Models, arXiv:2506.22143. arxiv.org/abs/2506.22143 (abgerufen am 7. November 2026)
Häufig gestellte Fragen
What is Arabic-English code-switching?
It is switching between Arabic and English within a conversation, often within a single sentence. On Gulf calls it commonly shows up as English brand names, numbers, dates, email addresses and technical terms dropped into Arabic speech, or Arabic fillers and confirmations inside an otherwise English sentence.
Why is code-switching hard for speech recognition?
Many speech recognition systems are built and tuned for one language at a time, and the vocabulary, the sounds and sometimes the writing script change mid-utterance. The Mixat dataset paper reports that the pre-trained Arabic and multilingual recognition systems it tested struggled with Emirati-English speech, and that code-switching adds difficulty on top of a low-resource dialect.
Which public datasets cover Arabic-English code-switching?
Three useful starting points are Mixat, about 15 hours of Emirati-English speech from two public podcasts; ZAEBUC-Spoken, about 12 hours of Arabic and English Zoom meetings covering Modern Standard, Gulf and Egyptian Arabic; and the SAGE research on generating synthetic code-switched speech to improve Arabic-English recognition. All three are research resources, not product benchmarks.
Should English words in a transcript be written in Arabic or Latin script?
Pick one convention per use case and apply it consistently. Latin script keeps brand names, product codes and email addresses searchable and unambiguous. Arabic script keeps a sentence readable as one language. Whatever you choose, write it into the transcription guidelines so reviewers and downstream search treat both forms the same.
Can DialerBee's AI voice agent handle callers who mix Arabic and English?
The AI voice agent has run on live calls in Arabic and English. Handling of any particular mix depends on your callers, accents and vocabulary, so this page makes no accuracy claim. Run a demo with recordings or scripts that reflect your own callers before you commit, and treat any other language as something to test on a demo first.
How should I test a voice agent for code-switching before launch?
Build a small test set from your own call types: scripted utterances that include brand names, numbers, dates and common English terms inside Arabic sentences, spoken by people with the accents and dialects of your callers. Score what was understood and what the agent replied, and read transcripts as well as listening to the calls.
Weiterführende Lektüre
- Arabischer KI-Sprachagent
- KI-Sprachagent
- Anrufprotokoll
- AI voice agent: what it does today in Arabic
- Multilingual AI outbound calling
- Arabic speech recognition across dialects in call centres
- How to measure call transcription accuracy (WER)
- Arabic voice agent testing checklist before you go live
- Arabic TTS for numbers, dates and prices in voice agents
Weiterführende Lektüre: Writing Prompts for an Arabic Voice Agent
Verwandte Artikel
Bereit, DialerBee in Aktion zu sehen?
Fordern Sie eine 15-minütige Live-Demo an oder starten Sie eine 14-tägige kostenlose Testphase – ohne Folien, ohne Verpflichtung. Das Telefonieren beginnt, sobald Ihr eigener SIP-Trunk oder Ihre TK-Anlage verbunden und freigegeben ist.
14-tägige kostenlose Testphase · kostenloses KI-Guthaben · keine Kreditkarte · 11 Sprachen · BYOC · Kontrollen zur Unterstützung der Compliance