Presidential and Vice-presidential Candidates: Language beyond the Word
October 20, 2008
Art Graesser, Moongee Jeon, and Zhiqiang Cai, University of Memphis
It is popular these days to analyze the language of candidates. We use language as signals on their persuasive impact, entertainment value, and eventually the votes. This makes sense because language is the window to the thoughts and values of the candidates.
One popular recent approach is to analyze the words used by the candidates. Our colleague James Pennebaker at University of Texas has made the persuasive case that pronouns are important. Another colleague, Jeff Hancock at Cornell University, has made the case that cognitive words signal deception. These are all valid analyses. But the point that we wish to make is that it is also important to dig deeper into the language, into sentence composition and the coherence of the message. It is time to move beyond the word and into deep meaning.
We have recently analyzed the nomination acceptance speeches of candidates to perform deeper computer analyses of language. We used Coh-Metrix, the only computer tool free to the public that analyzes language on sentence composition and discourse coherence (that is, how ideas in sentences connect with other sentences in meaning). Coh-Metrix can be accessed via Google. It was developed at the University of Memphis on a large grant from Institute for Education Sciences to analyze the language and coherence of textbooks (with Danielle McNamara, Art Graesser, and Max Louwerse).
We analyzed the nomination acceptance speeches of the four nominees: Obama, McCain, Biden, and Palin. We selected these speeches because they were all on an even playing field on importance and potential impact on the voters. It is also perfectly obvious that the speeches are products of speech writers. So we don’t know whether the conclusions are products of the nominees or their speech writers. However, it is the candidate that is ultimately responsible for the messages. We used Coh-Metrix to see how they are different. So what did we learn?
There were differences in acceptance speeches on length of the speech and the sentences. The length of all of the candidates was approximately 3500 words, with the two presidential speeches about 50% longer than the vice-presidential nominees. The length of the sentences is an important consideration. Obama was the obvious leader on this dimension. The mean number of words per sentence was approximately 20 words whereas the rest of the pack was about 15 words. We know that the grade level of messages is determined by the length of sentences and the length of words: The greater length of words and sentences translates to a higher grade level (that is, greater difficulty). We found that Obama was the leader in the grade level of the message according to the Flesch-Kincaid scale of readability (the most popular and accepted measure of readability of messages). Obama’s speech had a10th grade level whereas the rest of the pack had a 7-th grade level.
Content Words Don’t Matter Much
Pennebaker made the case that pronouns and function words are important indicators that differed among candidates and that were important. True enough. But what about content words? These are nouns, main verbs, and adjectives. We found that content words did not differ much among candidates. Consider the 4 measures in the figure below – clearly no differences among candidates. There were no differences in the words’ concreteness, imagability, familiarity, and age of acquisition (defined as the age when most people learned the words) according to Coltheart’s MRC Psycholinguistics Database. We suspect that nominees are coached on the words they use so that might explain why there is an even playing field on selection of content words in their speeches.
Sentences Differ Somewhat Among Candidates
Let’s go beyond the words into sentences. We analyzed the syntactic complexity of sentences and the noun-phrase complexity. These are shown below. The syntactic complexity was approximately the same, except that McCain’s speech was a bit lower. Palin’s noun-phrase complexity showed a slight advantage, measured as the number of adjectives that modify the nouns. The “hockey moms” and the “six pack dads” are richer noun-phrases than those of the male candidates.
We found that the Democrats had a higher incidence of questions in their speeches than the Republicans. However, the presidential candidates had a higher incidence of negations. Questions and negations are flags of uncertainty, openness, skepticism, and other dimensions of complexity.
Coherence of the Messages Differ among Candidates
Coh-Metrix analyzes the coherence of messages on dozens of measures. Each measure analyzes the extent to which ideas are connected to each other logically and conceptually. One measure assesses the extent to adjacent sentences have common content words. The other measure assesses the extent to which adjacent sentences are semantically related. The latter measure is based on latent semantic analysis, a statistical computation that is based on hundreds of dimensions of meaning (developed by Landauer, Dumais, and Kintsch). The two presidential candidates had more coherent messages than the vice-presidential messages on these coherence measures.
So what might we conclude from all of this? One conclusion that there is much more going on than words. It is easy to think about words because they are simple, easy to train, and sometimes flashy. However, we live in a complex world of ideas and solutions to complex problems. It is important to also consider levels of language and discourse that move beyond the word and into deeper levels of meaning.
A second conclusion is that the nomination speeches of presidential candidates are a notch above the vice presidents. They are longer and more coherent, perhaps with the coaching of the speech writers. It will be interesting to see how the unprepared discourse segments differ among candidates. This will be our next question, with the assistance of Coh-Metrix.
A third conclusion is that the complexity of Obama’s language tends to rise to the top. The speech length, sentence length, grade level, sentence syntactic complexity, noun-phrase complexity, questions, negations, and coherence were all at the top or among the top-two nominees.
We will continue to analyze the speech of the four nominees. Stay tuned.