Review of, and notes from
Emotional Intelligence: Science & Myth
Gerald Matthews, Moshe Zeidner, Richard Roberts
| This is a first draft - I plan to keep updating the
review. It is a very difficult and time consuming book to
read! - S. Hein. May 15, 2003 Second draft posted on June 4 - If you already read the first draft, you can just check the new notes - They are |
This is a highly technical book and very difficult to read, but the authors seemed to be meticulous in their work. It is the most serious criticism of the field of emotional intelligence to date. It also raises the most interesting and thought provoking questions I have seen. It is well worth reading for someone who is serious about the field of EI and is willing to invest a considerable effort. It size is just one of the reasons it is intimidating, over 700 pages. Unfortunately, I am afraid only a few people will be able to get much out of the book due to its overly academic language. This is a real loss because the authors have much to say that is worth hearing. I will try to translate the most important findings from their book into simple English in the near future but for now I want to give this summary in case it takes me a while to finish this "translation"!
If you would like to order the book you may do so here and I will get a small kickback from Amazon.com
S. Hein
May 12, 2003
Quotes From the Foreword by Robert Sternberg
Their Criticism of Daniel Goleman
Criticism of the Bar-On EQi test
Comments on the Mayer, Salovey, Caruso model of EI and their MEIS/MSCEIT tests
The authors divide the EI field into 2 parts: The popular vs. the scientific. The popular view of EI defines it basically as a set of personality traits as opposed to a true form of intelligence. An example is a "resilient" person. Resiliency and intelligence are not related scientifically. Resiliency can not be developed extensively in the same was as a highly intelligent person's potential math ability can be developed to a virtually limitless degree. In other words, personality traits are relatively fixed within people. They are not like a person's IQ which has a huge potential for development in the right environment. Thus the authors follow the lead of Mayer, Salovey and Caruso by distinguishing between the popular view of EI, which the authors also call "personality based" or "personalitylike," vs. the scientific model of EI which (again following Mayer et al) they call "ability based" or "abilitylike" definitions of EI.
With respect to the popular version of EI they conclude that:
(On all these points they agree with Mayer et al and others who have already said the same thing)
With respect to the scientific version they conclude:
Overall, they seem to believe that there is such a thing as emotional intelligence, but there at present there are far too many claims being made about it and too little real research supporting its existence.
"There is a tension between the scientific and commercial enterprises. The burgeoning sector of commercial EI products ranges from serious tests, to on-line institutions, to soft, cuddly toys that purport to increase children's EI. Science, with its focus on the limitations and uncertainties facing EI, tends to provide a sobering message. Of course, such missives may not be welcome to the salesperson trying to sell the public an ever-extending range of commercially available products." (p 514)
"Contrary to what has been claimed by EI researchers, it is a misconception that intelligence increases developmentally (see Mayer, Caruso et al., 1999), and that this is a necessary condition for EI to qualify as an intelligence."
"The research on group differences in EI...is meager."
"The zeitgeist value of EI rests on its view of an egalitarian society where we all can have EI in abundance."
"...the closer the person is to the population norm, the more intelligent they are." Criticism of the scoring method used on the MEIS/MSCEIT tests. (p. 530)
"There are also doubts about the cultural fairness of expert judgments. Consensus scoring substitutes popular standards for the standards of a few individuals, on the assumption that the pooled response of large normative samples is accurate. However, there seems to be little direct evidence for this supposition, and consensus may be influenced by culture- or gender-based stereotypes and by beliefs that are popular but false. (p. 518) More criticism of the scoring method used on the MEIS/MSCEIT tests.
"Overall, our conclusions concerning the prospect for a coherent theory of EI supported by empirical evidence are pessimistic for both ability and mixed (i.e., personalitylike) approaches to the construct... So far there appears little evidence that would suggest that abilities defined in terms of objectively assessed processing efficiencies would prove to be linked either to each other (and hence to some general intelligence) or to real-world adaptive outcomes." (p. 539)
"Research on stress and personality deals with adaptive constructs but fails to support the idea that some people are geniuses of adaptation, handling all challenges with equal facility, whereas others struggle to cope regardless of circumstances. Instead, personality traits define multiple and independent patterns of context dependent strength and weakness..." p. 539 (This is an apparent criticism of Bar-On's EQi test, which Bar-On claims measures a person's ability to handle stress and implies that such an ability means they have high EI)
"In general, the significance of EI for applied psychology is very limited..." p. 539
"In educational and occupational psychology too, there is no evidence to date that some context-free, generic EI may be trained." p 540
"It is questionable whether the use of test scores for selecting individuals for jobs or training courses would be legally defensible. A job applicant who was unsuccessful because of low consensus-scored test result might bring a legal case on the basis that the low score primarily represented the person responding differently to other people, rather than the inability to answer the questions correctly.
"In occupational settings, EI research has increased awareness of the potential role that a wide array of emotional competencies may play at the worksite (e.g. emotional awareness, empathy conflict resolution, and emotion regulation.) EI research has helped increase awareness in top and middle management about the importance of empathy in managers (awareness of other's feelings, needs, and concerns) and the need to be receptive to workers' feelings and needs at the worksite. Conversely, awareness of EI has legitimized the practice of workers of acknowledging and feeling emotions experienced at the worksite, rather than denying or minimizing them. Moreover, it has increased awareness of the importance of listening to the information or feedback the emotion is giving one at the workplace. It is also inspiring research that may shed light on the reciprocal relations between work and emotions, with emotions potentially influencing work-related cognitive and motivational processes, which in turn affect task and social behavior, and performance outcomes." p 542
"The ratio of hyperbole to hard evidence is rather high, with overreliance on anecdote and unpublished surveys. EI has been commonly claimed to be useful in occupational assessment, prediction, selection, and on the job performance, with a half a dozen books of papers and workshops devoted to describing the usefulness of EI ...However, a review of the empirical evidence provides little justification for such unfettered enthusiasm surrounding the construct in career selection and assessment. In fact, there is not one single study, published in a peer-reviewed journal, that shows that EI predicts occupational success/performance above (and beyond) that predicted by IQ." p 542
"It has also not been demonstrated that the interventions focusing on the core constructs of EI, such as emotional awareness, are more successful than those based on other principles, such as behavior modification." p 542
"The benefits of EI appear to reside mainly in raising awareness of emotional issues and motivating educators and managers to take emotional issues seriously." p 543
"Currently, EI mostly serves a cheerleading function, helping to whip up support for potentially (though not always actually) useful interventions focused on a heterogeneous collection of emotional, cognitive, and behavioral skills." p. 544
"...existing research does not yet show that EI exists as a well-defined psychometric and theoretical construct, let alone that it is critical for adaptation for real-world challenges." p 547
Quotes From the foreword by Robert Sternberg
"Goleman's work does not represent a systematic scientific program of research...in that there appear to be no refereed published studies where hypotheses are predictively tested against data."
"I share their profound skepticism of much of the popular movement. The positive side of the movement is that it helps broaden our concept of intelligence and get us away from the common fixation on IQ-based or IQ-related measures. The negative side of the movement is that it is often crass, profit-driven, and socially and scientifically irresponsible."
"The same people who criticize the conventional psychometric testers for potentially making a mess out of the lives of people who have potential but do not score well on conventional tests do much worse in promoting what, for the most part, are largely unvalidated or poorly validated tests of emotional intelligence."
"The arguments of the authors leave one concerned that there is no equivalent to the Food and Drug Administration for educational and organizational tests and instructional programs. We would not want drugs to go to the market that are essentially untested and have only their promoters' claims to back them up. Yet we routinely rely on such claims to buy educational and organizational products and services. People's lives may be affected in much the same way that their lives can be affected by drugs, but in this case, they have not even the appearance of protection."
"I share the concerns of the authors of this book with respect to personality-based theories and measures of emotional intelligence...I am more sanguine than the authors with respect to the abilities-based models approach...I suspect that over time this approach will be vindicated."
Note: "Personality-based" theories are those of Goleman, Bar-On etc. The "ability-based model" is that of Mayer, Salovey and Caruso-- S. Hein
With respect to Goleman's test which he claims measures EI - the ECI -360:
"Considering it assesses so many disparate concepts, it is likely that the ECI will have some utility. Even here, however, reliability is a cause for concern, as is the fact that more sophisticated techniques exist for assessing constructs comprising it. In sum, it is difficult not to be cynical of this measure, given the lack of publicly accessible data supplied by its creators and the constellation of old concepts packaged under its new label." (p. 218)
Other quotes:
"It is claimed that EI may be the single most important factor predicting job success, especially within a given job category or profession (Goleman, 2001). There is no evidence in peer-reviewed journals to support this claim." p. 546
"The preface to Goleman's (1995) book highlights what he perceives as a social crisis, described in flowery phrases such as "the disintegration of civility and safety," "surging rage and despair," and "the rotting of the goodness of our communal lives." The remedy is the teaching of emotional intelligence in schools and, a theme developed in a later book (Goleman,2000), promoting EI in the workplace. It is questionable whether civilization is falling apart quite so catastrophically. In any case, while it is plausible that school-based programs for EI are beneficial, there is no convincing evidence showing dramatic changes in adaptation..." p. 546
"In the absence of definitive research findings, we cannot be sure that the myths are entirely false. However, at the least, these sweeping claims are inadequately supported by empirical evidence, and thre are solid indications from existing ability and personality research that the claims made are either false or highly overstated. Indeed, while Goleman's (1995) vision has been widely disseminated, much of the empirical research in the area is more sober in its conclusions. It is surprising that exaggerated and very possibly false statements can command such widespread public acceptance." p. 546-7
"Goleman (1995, 2000) seeks to give EI scientific credibility by linking the construct to brain structures such as amygdala and orbitofrontal cortex...Nevertheless, there is no evidence that individual differences, in the normal range, map in any direct way to variation in brain function." p. 545 "To equate EI with neurological properties of brain systems is conceptually naive and of little use in explaining empirical data on huma emotion function." p. 538
"Goleman (1995), and to some extent Bar-On (1997, 2002) appear to claim that all desirable aspects of emotional function reflect a general factor of EI. Such a factor would be on par with IQ in bringing together many apparently distinct personal qualities. We have seen that tests of EI fail, thus far, to meet psychometric criteria, or even to correlate highly with one another. In addition, the extensive literature on personality shows that qualities such as resilience under stress, self-control, sensitivity to others and social assertiveness are distinct constructs that relate to different fundamental personality dimensions, and to differing psychological processes." p. 545
Criticism of the Bar-On EQi test
The authors say that the test measures things which are not really new, not really EI and which have "actually been under investigation for decades." They go on to say, "The fact that the EQi may represent little more than personality should be a cause of concern for those organizations prepared to employ it in personnel selection." (p 213)
The authors say other researchers have "demonstrated that personality adds little, in terms of incremental validity (over general intelligence) in the selection conteext." They continue to say, "It is disturbing that the EQi does not show better convergence with the MEIS/MSCEIT. (p 213)
When they analyzed the Bar-On test they concluded that "...collectively, these data suggest the EQi is nothing but a proxy (and likely crude) measure of personality constructs of the Five Factor Model... (p 212)
"Bar-On's (2000) review of his scale, which incorporates a large-scale confirmatory factor analysis, suggests that the fifteen scales originally making up the EQi appear empirically indefensible." (p. 209)
"The authors conclude there is inadequate evidence to justify use of the EQi as a selection device." (p. 211)
Quotes from the introduction - Historical and Sociocultural Context of Emotional Intelligence
"Emotional intelligence is a relatively new and growing area of behavioral investigation having matured recently with the aid of lavish international media attention."
Criticisms
I wouldn't say the field has "matured."
Using words my teen friends probably wouldn't use (Sociocultural)
The authors have one of the better, (albeit mostly sarcastic) senses of humor of any academics in the field. Here is one example when they are speaking of a press release put out by the publishers of the Bar-On EQi:
"A press release of July 15, 1999 announces that a sample from the United States scored significantly more highly on the test than a Canadian sample, and claims that Americans are superior in expressing their thoughts and feelings, reality testing, and coping skills. Fortunately, the press release sees hope for the unfortunate Canadians, in that these skills can be trained. It is unknown whether tolerance of being patronized is an aspect of emotional intelligence."
"In general, the significance of EI for applied psychology is very limited..." p 543 -- I won't be surprised if this is proven wrong. I believe there EI has a huge amount of significance if it is taken seriously as a potential ability which can be developed in either healthy or unhealthy ways.
"To label the patient as 'emotionally illiterate' simply does not suggest any additional therapeutic direction." -- First, the authors are mixing terms. Being emotionally literate and being emotionally intelligent are not the same thing. Second, if a patient was emotionally literate, one clear direction for therapy would to help the patient become more emotionally literate (see http://eqi.org/elit.htm)
Needlessly Complicated Sentences - "Given that different jobs call for different amounts of social and emotional involvement and activity and for different types of interpersonal interaction, a more differentiated conceptualization discriminating different kinds of skills may be more practically valuable." p. 543
Lots of needlessly big words such as instantiations (p. 60)
Statistical jargon is used without explaining it for the non- PhD.
They criticize others for not sticking to science then they make this statement with regard to programs that are claimed to develop or raise EI: "Whether or not these programs are actually fostering EI competencies, various useful skills are most likely learned during participation in these programs." p. 543
Lots of questions are asked without answers being given.
Authors keep putting off giving you their conclusions. They keep saying things like we will address this later in the book.
On p 55 they ask some good questions and imply they will answer them, but from my reading of the book so far, they never give any clear answers. Here is what they say:
In the second part of this book, we will look at whether research identifies a coherent set of individual differences...
Then they ask these questions (I have supplied the numbers)
1. Are there some people whose brains equip them for handling emotional encounters?
2. Do some people possess information-processing routines that are especially efficient in performing computations on emotional data?
3. Are some people especially skilled in evaluating the emotional significance of encounters, and choosing the most appropriate coping strategy?
These are good questions, but they authors never answer them or even give us their educated guesses. I suspect that if the authors were asked directly to answer these questions either yes or no, they would be extremely reluctant to give an answer. I think they would say "Maybe, Perhaps. We really don't know. We need more research." I personally would answer yes to all three. But I would point out that people have been raised in different environments and this will affect how their brains have been trained, what skills and "information-processing routines" they have developed, and what skills they have or haven't learned.
Another complaint I have is about their words in question three. First, they use the word "appropriate." This is a very subjective word. What is "appropriate" in one culture or one family or one organization or one romantic relationship may not be "appropriate" in another. The authors make a big deal out of being scientific, and then they use a word like "appropriate." This sounds more like a word Goleman would use than a true scientific researcher. They also talk in terms of "coping." I believe there is too much emphasis on "coping" and not enough on problem solving. In my work with emotionally abused teenagers I often see the school counselors trying to give the teens "coping mechanisms" instead of taking action to stop the abuse by the parents. One school counselor wrote me and praised a teen for being involved in lots of school activities, but this doesn't stop the abuse at home. Nor does it raise awareness of abuse or hold the parents accountable.
Comments on the Mayer, Salovey, Caruso model of EI and their MEIS/MSCEIT tests
"Mayer, Salovey et al (2000) advance cogent arguments for treating EI as a type of mental ability, rather than an ability-personality mix. Their approach in developing the MEIS/MSCEIT has been partially vindicated in that this instrument avoids redundancy with existing personality measures... The MEIS also correlates appropriately with cognitive ability measures (i.e. positively but moderately). Conceptually, the four aspects of emotional intelligence that the MEIS claims to assess -- emotion identification, assimilation, understanding and management -- seem meaningful and distinctive. This work is probably the most original in the field, and it certainly merits further attention. However, we have also found some serious shortcomings to this approach, both at a conceptual level and in terms of research outcomes. " (p. 517)
The authors really don't like the idea of "consensus scoring," in other words, using 21 academics to come up with the best answers for the test questions. They say that according to this approach, "the closer the person is to the population norm, the more intelligent they are." (p. 530)
I see their point and it is a good one. Even this, though, assumes that the 21 academic "experts" represent normal people. And it also assumes that "normal" people would select answers that would be in the best interest of the whole human species. When I think of some of the leaders which have been elected by the general population in various countries, I tend to think otherwise. Knowing that the results of EI tests will affect people's lives, I would want to know more about these experts and how they are actually functioning in their own lives before I would feel comfortable with using them for such an important assignment.
The authors also say that a good scoring method must be able to show that "what is being measured is actually a cognitive ability, as opposed to some preference or cultural value." The authors don't believe the Mayer et al tests have shown this. Instead they believe the tests may be culturally biased. I suspect they are right about this. Not only that, but I suspect that there is somewhat of a class bias in the sense that the test questions and answers relate to the social class of the test designers since it would be difficult for them to completely avoid biasing the test with their own (and their peers', friends', family's) values/beliefs/standards, life experiences etc. For example, I seem to remember one of the questions from the MEIS talking about using a "financial planner." For many people in the world, in fact I would say for the majority, this term would have little or no meaning or relevancy. Different people will have entirely different kinds of emotional problems that need to be solved.
The authors write "There are also doubts about the cultural fairness of expert judgments. Consensus scoring substitutes popular standards for the standards of a few individuals, on the assumption that the pooled response of large normative samples is accurate. However, there seems to be little direct evidence for this supposition, and consensus may be influenced by culture- or gender-based stereotypes and by beliefs that are popular but false... Being in step with other people's beliefs may well be advantageous, but it is doubtful it can be labelled a "true intelligence..." (p. 518)
The authors comments make me think of someone like Copernicus or Galileo. They were geniuses in their fields but they disagreed with the "expert consensus" of the times. They would surely have failed their courses in astronomy if they stated what they really believed. Only history has proven them correct.
Additional Criticism of Dan Goleman's Test which he claims measures EI (the ECI-360:
"Considering it assesses so many disparate concepts, it is likely that the ECI will have some utility. Even here, however, reliability is a cause for concern, as is the fact that more sophisticated techniques exist for assessing constructs comprising it. In sum, it is difficult not to be cynical of this measure, given the lack of publicly accessible data supplied by its creators and the constellation of old concepts packaged under its new label." (p. 218)
Additional Criticism of Bar-On's test (the EQi):