How Creativity is Measured—And Why It’s So Difficult
By Isaac Kaplan
Feb 26, 2018 2:36 pm
Photo by Yagi Studio/Getty Images.
How would the world be different if people had tails?
It’s an open-ended question that doesn’t have a right answer. That’s why Marjorie Taylor, a psychologist and professor emerita at the University of Oregon, uses it to measure the creativity of children. The creativity of kids’ responses—which range from “pants would have holes in them” to “people would hold tails instead of hands”—are evaluated by adults, and those results in turn fuel research.
But, as Taylor will tell you, creativity isn’t a single set thing that you can measure by simply asking one open-ended question. Rather, creativity is challenging to evaluate uniformly, and psychologists have developed numerous different methods, all with their own advantages and drawbacks, to study the elusive, coveted quality.
The wide range of approaches to creativity—from psychoanalytic, to psychological, to neurobiological—generally reveals the diversity of the field, but has led some to describe it as “a degenerating research program,” as Mark Batey, a senior lecturer in organizational psychology at the University of Manchester, wrote in a 2012 article on measuring creativity.
Journalists (myself included) don’t always help matters. Boiling down a study on creativity to a headline potentially gives the impression that researchers have a universally agreed upon way of measuring creativity, and that this measure aligns with a reader’s own idea of what defines creativity. This isn’t solely an academic discussion. Today, as rote work becomes increasingly automated, creativity is being billed as a marketable professional skill across many industries. So, as individuals actively pursue methods to increase their own creativity, they should understand how researchers are defining it.
“When you’re talking about creativity, you have to understand you’re talking about a broad spectrum of abilities in different domains,” Taylor said. “It’s hard to measure.”
A recent literature review, published in December 2017, gives an overview of how psychologists have approached measuring creativity. Conducted by Sameh Said-Metwaly, a PhD student at KU Leuven in Belgium, along with professors Eva Kyndt and Wim van den Noortgate, the review looked at 152 papers published through 2016 and found that there was no common framework.
“Researchers generally agree that creativity involves the production of something that is both new and suitable for a particular purpose or use,” Said-Metwaly wrote to Artsy in an email. “In other words, creativity is the ability to be different in a useful way.”
But what does “useful” mean? If a researcher thinks it means utility to society as a whole, they may take a different approach to measuring creativity than another researcher who treats it as only existing within the context of a laboratory study, Batey noted.
Generally in psychology, there are four very broad aspects of creativity that researchers examine called the “four Ps,” and they deploy a host of different methods to do so.
The first “P” is the cognitive process element, which seeks to find out if you think in a creative way. The second is the personal element, which focuses on traits (like high energy or risk-taking) that are associated with creativity in individuals. Then there’s the product, which aims to find out if the ultimate outcome or solution by a person is creative. And finally, the press (or environment), which is meant to determine if the context in which something occurs is creative.
The cognitive process element was the most commonly studied element in the research reviewed by Said-Metwaly and his colleagues, appearing in 52.6 percent of the papers. Researchers looking into cognitive creativity typically use methods that test a user’s divergent thinking, which involves generating a range of possible solutions to a prompt, rather than a single correct answer.
Among the most famous examples of tests to measure divergent thinking is the “unusual uses” test. How many uses can you think of for a brick? Or a milk carton? Researchers will often put this question (or a variation of it) to participants, take their answers, and then analyze them, looking at the number of answers (or uses), originality, flexibility, and level of detail.
While process tests like this one are broadly adopted and used in a consistent way, it’s easy to see their limitations. For one thing, they’re predicated on a very general notion of creativity, rather than how creativity functions in the context of a specific field (for example, creativity likely means something different, and requires different skills, for mathematicians and artists). Similarly, such tests don’t involve real-world situations, nor do they occur in real-world contexts. And kids, who are not used to such repetitive questioning, find the test particularly strange, Taylor noted.
“For a lot of studies, creativity gets boiled down to the unusual uses task,” she said, adding that while she uses the test, she’s not a huge fan. “It’s very narrow in terms of focusing in on creativity as manipulation of a physical object.”
Taylor prefers a test in which the researcher will start a story and ask children to create the ending: For example, you’re walking down a path in the woods when you notice a key on the ground, and a door, then what? Their responses are then rated by adults for their creativity.
This story completion test is broadly in line with product approach to measuring creativity, which is about outcomes. Researchers using this approach take a person’s product (a painting, or a mathematical formula, for example) and give it to a group of experts, who then rate it for creativity. The underpinning theory is that people in a specific field are best equipped to evaluate the extent to which something is creative.
The drawbacks here are that certain people who are considered extremely creative are not seen as such in their own time. Additionally, it can be costly to bring in experts, and the personality of judges can impact the outcome. Plus, sometimes they seem to get it wrong. One 1991 study, cited by Said-Metwaly in the literature review, asked judges, of various ages with differing experience with art, to rate the creativity of mass-produced art and art found in museums. In the end, they rated both categories as being similarly creative.
The person approach relies generally on identifying certain qualities associated with creative achievement (high energy, behavioral flexibility, intuition, emotional variability, self esteem, risk-taking, perseverance, independence, introversion) and then asking people about them, using self-reported surveys. These questionnaires are easy to administer and standardized, allowing for the data to be easily interpreted, noted Said-Metwaly’s literature review.
But of course, the presence of these traits doesn’t guarantee that a person is creative or adept at divergent thinking tasks. And traits that may be linked to creativity in one field may not be found among creative individuals in another field. A pair of studies, conducted in 1998 and ‘99 by San Jose University psychology professor G. J. Feist, and cited in Said-Metwaly’s review, found that while certain traits are linked to creative individuals broadly (like openness to experience, hostility, and impulsiveness), specific professions see some variance. So while creative scientists are self-confident, autonomous, and arrogant, creative artists tend to be anxious, imaginative, and emotionally sensitive, according to Said-Metwaly’s overview.
The least studied element of creativity, appearing in only 4.1 percent of papers examined by Said-Metwaly, is the environmental factor. This includes research into how organizational structure, as well as things like school environment, relate to creativity.
Ultimately, the goal is not to pick one of the “four Ps” and trumpet it above all the others, nor to devise the single way to study the creative element that a researcher wants to explore. As Batey wrote in his paper, it is worth developing frameworks that mesh the different approaches to studying creativity together, in order to get a more holistic picture.
Creativity, noted Said-Metwaly, is an extremely multifaceted quality. It makes sense that there’s not only one method of measurement, but he argued that more needs to be done to reach a consensus on what creativity is and how to measure it, along with more research into the domain-specificity of creativity in various fields. But broadly, one thing researchers are united on is the need to continue to examine creativity.
“Creativity is so important and so interesting and so ubiquitous,” said Taylor. “It’s really important to try and understand it scientifically.”
It’s not gone unnoticed that some of the most innovative contemporary artists working today hail from Iceland.Björk,Ragnar Kjartansson,Katrín Sigurdardóttir, andOlafur Eliasson(who, to be fair, is half Danish) are all known for taking unique, interdisciplinary approaches to their art. The country, with a scant population of around 330,000, is also host to a thriving microcosm of designers, writers, and musicians.
Iceland has consistently ranked among the most innovative countries in the world. Indeed, one in four people there work in creative careers; a whopping one in 10 have published a book. The small nation boasts some 7,000 creative companies, and a growing number of creative jobs, despite the impact of 2008’s massive economic crisis.
While outsiders have long suspected that the island nation’s isolation and striking natural environment are key drivers of this creative boom, anew research study, published in theGifted and Talented Internationaljournal, has found that various other significant factors can more directly account for why Icelandic people are so creative and innovative.
Olafur Eliasson, Harpa. Photo by mariejirousek, via Flickr.
In 2011, University of Kansas professor and psychologist Barbara A. Kerr and her daughter, an art student, had a weekend stopover in Reykjavik on their way to a vacation in Prague. Charmed by the city’s creative community, Kerr’s daughter moved there after finishing school two years later, and found a supportive circle of creative peers that far surpassed what she’d found at an American university. Keen to understand what’s driving Iceland’s creative community, Kerr developed a university seminar for her doctoral and masters students, and took her team to Iceland.
Driven by the question “Why is Iceland so creative?” Kerr and eight students sought to examine “the people, places, processes, and products that may contribute to driving Icelandic innovation,” with a special focus on fine art and music. They collected data to address multiple variables, looking for distinctive elements in various areas: the abilities and personalities of individuals living in Iceland; the natural and built environment; social, familial, educational, and cultural dynamics; and government and economic policies.
They conducted a comprehensive literature review (CLR), which involved over 1,000 hours of analyzing existing scholarship, literature, and popular media, but also conducting new, on-the-ground research interviews in Reykjavik.
“It certainly takes us right out of the ivory tower,” Kerr said of the method. “We got in there to see to what degree our own observations support what the literature has to say, and it allowed the Icelanders themselves to comment upon the research.”
Working with their existing networks and new local contacts, like the editor of the popular newspaper theReykjavik Grapevine, Kerr and her team chose to interview 15 individuals whom they believed were representative of artists, musicians, and writers working in Iceland. These subjects were complemented by more focused, off-the-cuff conversations throughout Reykjavik and elsewhere in the country, at local events, bookstores, museums, or other venues. In some cases these were quite informal; Kerr gives the example of a researcher having a 5-to-10-minute conversation with an Icelandic man at a club to discuss nightlife or relationships.
What They Found
Photo by Jonatan Pie.
Despite the fact that Icelanders largely refuted the idea that they were any more creative than people of other nations, Kerr and her team’s CLR supported the theory that creativity among Icelanders was a result of their individual, social, educational, and cultural attributes.
While Icelanders did not possess remarkably distinct personalities or abilities, they were found to commonly display “attitudes of independence and tolerance” which, Kerr wrote, “supports openness to experience, the personality attribute that is most strongly correlated with creativity.”
In terms of the environment, Icelanders rejected the idea that the vast beauty of the landscape was a font of creative inspiration. Kerr said that interviewees were overwhelmingly skeptical of this, often saying one of two things: “The inspiration comes from community and culture, and not from the environment,” or “That’s just the national advertising, policy makers came up with that idea, just trying to attract tourists.” Researchers found that urban and social contexts were more important to creativity.
“At one point, we came to the conclusion that Icelanders are like fish in water—they live in an inspiring environment, but they’re used to it,” Kerr quipped. Research found that unsurprisingly, non-natives were more likely to be inspired by the natural landscape.
Kerr noted that one of the most important differences between Icelanders and people from the United States and European countries is “extreme differences in family life.” They found that relationships between couples are remarkably egalitarian, and women’s creative output is well-supported. And while the institution of marriage is not a priority (only 30 percent of Icelandic people are married), family life and caring for children are central to adulthood.
Both in schools and at home, it’s common practice to give children the space for play and exploration, which “encouraged both imagination and the creative process,” the study found. In schools, which are free and public, testing is de-emphasized, and students engage in hands-on learning and free-play. Over the past two decades, the dominant curriculum has beeninnovation education(IE), which originated in Iceland in 1991, and is also used in other parts of Scandinavia and the United Kingdom. Students are given opportunities to apply creativity to everyday life, and teachers act as “facilitators rather than lecturers,” the researchers wrote, serving to support students in generating high-quality, novel ideas through their work.
Kerr also pointed to an emphasis on craft and making things by hand. “Every Icelandic male knows how to knit, every female knows how to use tools,” she said. “It seems important to the development of creativity to simply know how to use tools for making and learning at home and school.”
The government can also be credited for fostering creativity, researchers found, as “policies in Iceland encourage the development of creative products,” like public art commissions for street artists. Interviewees noted, however, that the government’s support might be “impeding more radical expressions of creativity,” given their political motives.
The less hierarchical society also enables greater creativity, Kerr’s team found. For one, an online phone book enables creatives to reach out to peers or leading artists, as well as local officials and business professionals, to discuss projects and collaborations. “I think that’s especially important to artists—not only can they get in touch with each other but opinion-makers and promoters,” Kerr said. “If you have a rock band, you can get bookings, you can get reviewers to come and watch you, it’s not that hard to reach out.”
The greatest learnings from the study, researchers recognized, are in terms of educational opportunities for fostering creativity among children and young adults. Schools in Iceland are particularly key to the creative community, not only in providing children with designated space for creativity, but also in implementing the promising approach of the IE curriculum.
Research concluded, though, that a great deal of emphasis should be placed on the generous, social safety net that Icelanders enjoy. They summed this up with a quote from an Icelandic immigrant who collaborated on the study: “Never underestimate what knowing you will always have food, shelter, childcare, and education will do for your creativity.”