In a country as expansive as the United States it’s no surprise the nation’s politicians speak with a variety of accents, speech patterns, and pronunciation quirks. Now that ASR (automated speech recognition) is becoming part of our landscape those different speech patterns make a difference.
Trint is an A.I.-based software that uses machine learning to transcribe recorded audio and lets users verify and correct errors. With good audio and clear speakers Trint can return transcripts that are 98% correct - in just minutes at very low cost.
To see who works best with A.I. bots, Trint set up a challenge putting prominent U.S. politicians head-to-head in a speech recognition test.
Who did the bots understand better? Hillary Clinton, Donald Trump or Barack Obama?
How the Trint Index works
We picked 14 influential politicians -- and a fictional one from a hit television series. We took audio recordings of their speeches and put them through the Trint transcription AI.
Each of the recordings...
Was created with high-quality equipment
Included minimal audience noise
Was 4–5 minutes in length
Our bots transcribed the speeches and gave us the most accurate transcription they could. We analyzed the results using a standard called Word Error Rate (WER).
This formula is commonly used for measuring the accuracy of speech recognition. It compares 100% accurate transcription to AI-transcribed text and grade’s the AI’s performance. Here’s what it measures:
Substitutions - when a word needs a complete substitute
Deletions - when an incorrect word or character needs to be deleted
Insertions - when a missing word or character needs to be added
A transcription with a WER of 4.8, for example, is 4.8% inaccurate (or 95.2% correct).
Let’s get on with the fun part.
The most intelligible politician is...
Coming in first place as the most intelligible politician on the Trint Index is Nikki Haley, the U.S. ambassador to the United Nations, with a WER of just 0.52 which means her Trint transcription test is 99.48 percent accurate.
Who came second? Mark Rubio, Hillary Clinton, Ted Cruz, Chuck Schumer, Bernie Sanders?
With a WER of 0.62 and an accuracy of 99.38 percent, Hillary Clinton just barely beat out Chuck Schumer (99.35) for second most accurate in the eyes our bots.
Our fictional competitor, Vice President Selina Meyer from HBO’s Veep, scored amazingly well at 98.74 percent, earning fourth place. It’s worth noting that Julia Louis-Dreyfuss is a seasoned actress so it’s not surprising that her speech is clear and clean.
Senator Kamala Harris narrowly landed in fifth place at 98.69 percent. This one made us proud. The recording included laughter and interruptions that typically confuse bots, but the Trint A.I. was able to listen through them and deliver amazingly accurate transcripts.
We quickly realized that the lowest performing politicians have some things in common:
They swallow the last word of each sentence. House Minority Leader Nancy Pelosi came in 15th place (i.e. last) with 96.01 percent accuracy. She scored just under Senator John McCain, who landed 96.07 percent accuracy.
They have heavy accents. Ted Cruz’s famous Texas accent landed him 13th place with 96.69 percent accuracy. Senator Elizabeth Warren’s Massachusetts accent also gave the Trint A.I. a hint of trouble, pulling her back to 12th place with 97.59 percent accuracy.
President Donald Trump, 11th place with 97.89 percent accuracy, is also known for swallowing words. However, Trint’s CEO, Jeff Kofman notes that both President Trump and President Obama come out consistently well with automated speech recognition because both speak very slowly.
Take a look at the full results:
What it all means
In the end, the Trint Index demonstrates how advanced A.I.-speech recognition has become. It’s faster, easier, and more affordable than ever to transcribe audio recordings with incredible precision.
Want to experience the Trint AI for yourself or have an idea for our next Trint Challenge? Let us know by reaching out to email@example.com.