Tracy, S. (2010) Qualitative Quality: Eight “Big-Tent”. Criteria for Excellent Qualitative Research, vol. 16(10), pp.837-851

criteria, quite simply, are useful. Rules and guidelines help us learn, practice, and perfect. Indeed, research on learning (Dreyfus, Dreyfus, & Athanasiou, 1986) demonstrates that novices and advanced beginners in any craft (whether cooking, skiing, dancing, or playing music) rely heavily on rule-based structures to learn. Guidelines provide a path to expertise. (Tracy, 2010, p.838)

Criteria serve as shorthand about the core values of a certain craft. (Tracy, 2010, p.838)

A language of best practices provides the option to frame our work, if desired, as systematic and structured (LeGreco & Tracy, 2009) (Tracy, 2010, p.838)

Creswell (2007) specifically heeds this call by offering a unique set of evaluative criteria for each of five different qualitative areas—narrative, phenomenological, grounded theory, ethnographic research, and case study research. Among other edicts, Creswell advises narrative researchers to “focus on a single individual (or two or three individuals)” (p. 214). Meanwhile, he suggests that phenomenologists ask, “Is the transcription accurate” (p. 215), and subsequently lists a criterion for good grounded theory as showing “what major categories emerged” (p. 216). (Tracy, 2010, p.839)

Criteria for quality (end goal)	Various means, practices, and methods through which to achieve
Worthy topic	The topic of the research is Relevant, Timely, Significant, Interesting
Rich rigor	The study uses sufficient, abundant, appropriate, and complex Theoretical constructs, Data and time in the field, Sample(s), Context(s), Data collection and analysis processes
Sincerity	The study is characterized by Self-reflexivity about subjective values, biases, and inclinations of the researcher(s)Transparency about the methods and challenges
Credibility	The research is marked by Thick description, concrete detail, explication of tacit (nontextual) knowledge, and showing rather than tellingTriangulation or crystallizationMultivocalityMember reflections
Resonance	The research influences, affects, or moves particular readers or a variety of audiences through Aesthetic, evocative representationNaturalistic generalizationsTransferable findings
Significant contribution	The research provides a significant contribution Conceptually/theoreticallyPracticallyMorallyMethodologicallyHeuristically
Ethical	The research considers Procedural ethics (such as human subjects)Situational and culturally specific ethicsRelational ethicsExiting ethics (leaving the scene and sharing the research)
Meaningful coherence	The study Achieves what it purports to be aboutUses methods and procedures that fit its stated goalsMeaningfully interconnects literature, research questions/foci, findings, and interpretations with each other

worthy studies are interesting and point out surprises—issues that shake readers from their common-sense assumptions and practices. (Tracy, 2010, p.841)

When research merely confirms existing assumptions, people will deny its worth while acknowledging its truth. In short, audiences will think, “that’s obvious” rather than the more coveted “that’s interesting” (Murray, 1971)! (Tracy, 2010, p.841)

Richness is generated through a “requisite variety” (Weick, 2007, p. 16) of theoretical constructs, data sources, contexts, and samples. Requisite variety, a concept borrowed from cybernetics, refers to the need for a tool or instrument to be at least as complex, flexible, and multifaceted as the phenomena being studied. (Tracy, 2010, p.841)

a researcher with a head full of theories, and a case full of abundant data, is best prepared to see nuance and complexity. (Tracy, 2010, p.841)

Researchers should evidence their due diligence, exercising appropriate time, effort, care, and thoroughness. (Tracy, 2010, p.841)

Questions about rigor include the following:

• Are there enough data to support significant claims?
• Did the researcher spend enough time to gather interesting and significant data?
• Is the context or sample appropriate given the goals of the study?
• Did the researcher use appropriate procedures in terms of field note style, interviewing practices, and analysis procedures? (Tracy, 2010, p.841)

If data are new, unique, or rare, a valuable contribution could be achieved with very little data (e.g., Scarduzio & GeistMartin, 2008). Decisions about how much data to collect also intersect with the level of analysis. Close line-by-line data analyses can be rigorous even when using just several lines of transcription (e.g., Martin, 1990). (Tracy, 2010, p.841)

In terms of interviewing, demonstrations of rigor include the number and length of interviews, the appropriateness and breadth of the interview sample given the goals of the study, the types of questions asked, the level of transcription detail, the practices taken to ensure transcript accuracy, and the resultant number of pages of interview transcripts. (Tracy, 2010, p.841)

Rigorous data analysis may be achieved through providing the reader with an explanation about the process by which the raw data are transformed and organized into the research report. Despite the data-analysis approach, rigorous analysis is marked by transparency regarding the process of sorting, choosing, and organizing the data. (Tracy, 2010, p.841)

Qualitative methodology is as much art as it is effort, piles of data, and time in the field. (Tracy, 2010, p.841)

Sincerity as an end goal can be achieved through self-reflexivity, vulnerability, honesty, transparency, and data auditing. (Tracy, 2010, p.842)

Sincerity means that the research is marked by honesty and transparency about the researcher’s biases, goals, and foibles as well as about how these played a role in the methods, joys, and mistakes of the research. (Tracy, 2010, p.842)

Self-reflexivity encourages writers to be frank about their strengths and shortcomings. Ethnographers should report their own voice in relation to others and explicate how they claim to know what they know. (Tracy, 2010, p.842)

Researchers can practice self-reflexivity even before stepping into the field through being introspective, assessing their own biases and motivations, and asking whether they are well-suited to examine their chosen sites or topics at this time. (Tracy, 2010, p.842)

My own struggles to negotiate my position as the researcher

Questions to ask include “Why am I doing this study?” “Why now?” “Am I ready for this?” If you cannot answer these questions, then perhaps now is not the right time. (Tracy, 2010, p.842)

Self-reflexive practice moves from early stages of research design through negotiating access and trust, data collection, analysis, and presentation. Self-reflexive researchers examine their impact on the scene and note others’ reactions to them. In doing so, these researchers think about which types of knowledge are readily available, as well that which is likely to be shielded or hidden. They interrogate their own predilections or opinions and ask for feedback from participants. (Tracy, 2010, p.842)

good ethnography is not limited to knowledge or information about others “out there” but expands the definition to include stories about oneself. (Tracy, 2010, p.842)

Autoethnographies (Ellis & Bochner, 2000) and confessional tales (Van Maanen, 1988) focus on researchers’ subjective experiences, hopes, fears, and vulnerabilities. (Tracy, 2010, p.842)

One way to deal with the ambiguity of “how much self-reflexivity,” is to show rather than tell self-reflexivity by weaving one’s reactions or reflexive considerations of self-as-instrument throughout the research report. (Tracy, 2010, p.842)

Transparency refers to honesty about the research process. Seale (1999) terms this process auditing and notes that researchers should provide “a methodologically self-critical account of how the research was done” (p. 468). (Tracy, 2010, p.842)

how the researcher got into the context, the level of participation and immersion, fieldnote practices, and level of detail in transcription. Transparent research is marked by disclosure of the study’s challenges and unexpected twists and turns and revelation of the ways research foci transformed over time. (Tracy, 2010, p.842)

Transparency also means that credit is given where due in terms of author order and acknowledgements to participants, funding sources, research assistants, and supportive colleagues. (Tracy, 2010, p.842)

Sincere researchers are empathetic, kind, self-aware, and self-deprecating. (Tracy, 2010, p.843)

Credibility refers to the trustworthiness, verisimilitude, and plausibility of the research findings. (Tracy, 2010, p.843)

Richardson (2000a) argues that good ethnography expresses a reality that seems true, providing “a credible account of a cultural, social, individual, or communal sense of the ‘real’” (p. 254). (Tracy, 2010, p.843)

Thick description. One of the most important means for achieving credibility in qualitative research is thick description. By this, I mean in-depth illustration that explicates culturally situated meanings (Geertz, 1973) and abundant concrete detail (Bochner, 2000). (Tracy, 2010, p.843)

Ethnography’s level of detail should provide a complex and expansionistic depiction. In qualitative research, “things get bigger, not smaller and tighter, as we understand them” (Gonzalez, 2000, p. 629). To illustrate data’s complexity, researchers are advised to show, meaning that they provide enough detail that readers may come to their own conclusion about the scene. This is contrasted from the author telling the reader what to think. Showing is rhetorically more difficult and usually requires more words than telling. (Tracy, 2010, p.843)

Immersion and concrete detail are also necessary for researchers to ascertain tacit knowledge, considered to be the taken for granted, “largely unarticulated, contextual understanding that is often manifested in nods, silences, humor, and naughty nuances” (Altheide & Johnson, 1994, p. 492). (Tracy, 2010, p.843)

Hidden assumptions and meanings guide individuals’ actions whether or not participants explicitly say so. (Tracy, 2010, p.843)

The longer researchers are present and closely watching, the more likely they are to notice a culture’s values. Furthermore, researchers can access tacit knowledge not only by taking note of who is talking, and what they are talking about, but also who is not talking and what is not said. Indeed, good qualitative research delves beneath the surface to explore issues that are assumed, implicit, and have become part of participants’ common sense. Noticing, analyzing, and unpacking this knowledge is key to understanding interaction and behavior in the scene. (Tracy, 2010, p.843)

triangulation in qualitative research assumes that if two or more sources of data, theoretical frameworks, types of data collected, or researchers converge on the same conclusion, then the conclusion is more credible (Denzin, 1978). Put another way, “findings may be judged valid when different and contrasting methods of data collection yield identical findings on the same research subjects; a case of replication within the same setting” (Bloor, 2001, p. 384). (Tracy, 2010, p.843)

“all research findings are shaped by the circumstances of their production, so findings collected by different methods will differ in their form and specificity to a degree that will make their direct comparison problematic” (Bloor, 2001, p. 385). (Tracy, 2010, p.843)

Different methods, data, or researchers often do (and perhaps should) yield different results. For example, research participants may espouse very different values in interviews than the values they enact in contextual interactions—with both sets of data being equally “true.” (Tracy, 2010, p.843)

making use of multiple researchers, data sources, methods, and theoretical lenses is still considered valuable (Tracy, 2010, pp.843-844)

Multiple types of data, researcher viewpoints, theoretical frames, and methods of analysis allow different facets of problems to be explored, increases scope, deepens understanding, and encourages consistent (re) interpretation. (Tracy, 2010, p.844)

Crystallization encourages researchers to gather multiple types of data and employ various methods, multiple researchers, and numerous theoretical frameworks. However, it assumes that the goal of doing so is not to provide researchers with a more valid singular truth, but to open up a more complex, in-depth, but still thoroughly partial, understanding of the issue. (Tracy, 2010, p.844)

Multivocality. Closely aligned with the notion of crystallization and showing rather than telling, is multivocality. Multivocal research includes multiple and varied voices in the qualitative report and analysis. Multivocality emerges, in part, from the verstehen practice of analyzing social action from the participants’ point of view. Verstehen requires researchers to provide a thick description of actors’ performances and their local significance to interpret meaning (Lindlof & Taylor, 2002). (Tracy, 2010, p.844)

Qualitative researchers do not put words in members’ mouths, but rather attend to viewpoints that diverge with those of the majority or with the author. (Tracy, 2010, p.844)

Differences in race, class, gender, age, or sexuality can be the basis for very different meanings in the field, and credibility is enhanced when the research evidences attention to these possibilities. (Tracy, 2010, p.844)

Multivocality can also be achieved through intense collaboration with participants. (Tracy, 2010, p.844)

Researchers who act as a team with participants also surrender complete editorial control in turn for more nuanced analyses with deeper meaning to members at hand. (Tracy, 2010, p.844)

Member reflections. In addition to just paying heed to multiple voices during the data collection phase, another path toward credibility goes a step further—by seeking input during the processes of analyzing data and producing the research report. (Tracy, 2010, p.844)

“member reflections” allow for sharing and dialoguing with participants about the study’s findings, and providing opportunities for questions, critique, feedback, affirmation, and even collaboration. (Tracy, 2010, p.844)

Member reflections, on one hand, may take the form of member checks, member validation, and host verification— terms that refer to methods of “taking findings back to the field and determining whether the participants recognize them as true or accurate” (Lindlof & Taylor, 2002, p. 242). (Tracy, 2010, p.844)

However, because the labels of member checks, validation, and verification suggest a single true reality, I instead offer the umbrella term member reflections—which may be applicable to a wider range of paradigmatic approaches. (Tracy, 2010, p.844)

As such, member reflections are less a test of research findings as they are an opportunity for collaboration and reflexive elaboration. (Tracy, 2010, p.844)

Through the reflection process, participants can react, agree, or find problems with the research. Do participants take the time to read the results? Do they care? Do they find the study interesting? Enlightening? Objectionable? Answers to these questions speak volumes about the research process and its contributions. (Tracy, 2010, p.844)

The potential of research to transform the emotional dispositions of people and promote greater mutual regard has been termed “empathic validity” by Dadds (2008). (Tracy, 2010, p.845)

Resonance can be achieved through aesthetic merit, evocative writing, and formal generalizations as well as transferability. Not every qualitative study must achieve resonance in the same way, but all high-quality qualitative reports must have impact. (Tracy, 2010, p.845)

Aesthetic merit. A key path to resonance and impact is aesthetic merit, meaning that the text is presented in a beautiful, evocative, and artistic way. (Tracy, 2010, p.845)

Bochner (2000) looks for qualitative narratives that are vivid, engaging, and structurally complex, or, in short, a story that moves the “heart and the belly” as well as the “head” (p. 271). Likewise, Richardson (2000a) emphasizes the importance of aesthetics, saying that, writing should be creative, complex, and encourage the reader to feel, think, interpret, react, or change. (Tracy, 2010, p.845)

These include personal narrative, storytelling, evocation, emotion, and engaged embodiment (Holman Jones, 2005; Lindemann, 2010). (Tracy, 2010, p.845)

Resonance also emerges through a study’s potential to be valuable across a variety of contexts or situations—practices that have been called generalizability or transferability. (Tracy, 2010, p.845)

qualitative research engages in-depth studies that generally produce historically and culturally situated knowledge. (Tracy, 2010, p.845)

Transferability is achieved when readers feel as though the story of the research overlaps with their own situation and they intuitively transfer the research to their own action. (Tracy, 2010, p.845)

Transferability also relates to “evocative storytelling” (Ellis, 1995)—the production of vicarious emotional experience in the reader. Evocative stories have the power to create in readers the idea that they have experienced the same thing in another arena. (Tracy, 2010, p.845)

Through the process of naturalistic generalizations, readers make choices based on their own intuitive understanding of the scene, rather than feeling as though the research report is instructing them what to do. (Tracy, 2010, p.846)

the research will “contribute to our understanding of social life” (Richardson, 2000a, p. 254), “bring clarity to confusion, make visible what is hidden or inappropriately ignored, and generate a sense of insight and deepened understanding” (Tracy, 1995, p. 209). (Tracy, 2010, p.846)

Theoretically significant research is “intellectually implicative for the scholarly community” (Tracy, 1995, p. 210), extending, building, and critiquing disciplinary knowledge. At its most basic, research may provide a theoretical contribution by examining how existing theory or concepts make sense in a new and different context. (Tracy, 2010, p.846)

theoretical significance usually requires that we go beyond mere (re) application of existing theory. Rather, research that builds theory extends or problematizes current theoretical assumptions. Such contributions offer new and unique understandings that emerge from the data analysis—conceptualizations that help explain social life in unique ways and may be transferred to other contexts. (Tracy, 2010, p.846)

Heuristic significance moves people to further explore, research, or act on the research in the future. Research is heuristically significant when it develops curiosity in the reader and helps inspire new discoveries (Abbott, 2004). (Tracy, 2010, p.846)

Researchers can increase heuristic significance by providing readers with substantive and interesting suggestions for future research. Research is also heuristic when it influences a variety of audiences, such as policy makers, research participants, or the lay public, to engage in action or change—something that overlaps with practical significance. (Tracy, 2010, p.846)

Practically significant research asks whether the knowledge is useful. Does it help to shed light on or helpfully frame a contemporary problem? Does it empower participants to see the world in another way? Does it provide a story that may liberate individuals from injustice? (Tracy, 2010, p.846)

However, all research has an “agenda”—some agendas are just more explicit than others. Furthermore, aiming for practical change is no more subjective than research that aims to build theory. (Tracy, 2010, p.846)

another means toward significance is through engaging research methodology in a new, creative or insightful way—methodological significance. (Tracy, 2010, p.846)

one could play with new types of creative data analysis or representation practices—such as drawing analysis, the analysis of found poems, or evocation through improvisation. Or, a researcher could exhibit originality in the methods of narrative representation, data collection, coding, and organizing. In doing so, methodologically significant approaches not only may lead to theoretical insights and practical usefulness but also contribute to future researcher’s practice of methodological craft skills. (Tracy, 2010, p.847)

Procedural—also known as categorical—ethics refer to ethical actions dictated as universally necessary by larger organizations, institutions or governing bodies. For instance, procedural ethics are encompassed by the Institutional Review Board (IRB), including mandates such as do no harm, avoid deception, negotiate informed consent, and ensure privacy and confidentiality (Sales & Folkman, 2000). (Tracy, 2010, p.847)

Procedural ethics also suggest that research participants have a right to know the nature and potential consequences of the research—and understand that their participation is voluntary. (Tracy, 2010, p.847)

As a method of procedural ethics, researchers safeguard participants from undue exposure by securing all personal data—in a locked office or drawer, or a password-protected website. Furthermore, privacy can be achieved through conflating data in strategically creative ways. Such conflation is necessary for avoiding deductive disclosure which occurs when “persons who know certain facts about a participant (such as his or her zip code, profession, or ethnicity) may be able to use that information to deduce damaging or private information about that participant” from the body of data (Sales & Folkman, 2000, p. 18). (Tracy, 2010, p.847)

A situational ethic deals with “the unpredictable, often subtle, yet ethically important moments that come up in the field” (Ellis, 2007, p. 4). (Tracy, 2010, p.847)

A situational ethic assumes that each circumstance is different and that researchers must repeatedly reflect on, critique, and question their ethical decisions. Situational ethics often revolve around the utilitarian question “Do the means justify the ends?” (Tracy, 2010, p.847)

Relational ethics are related to an ethic of care that “recognizes and values mutual respect, dignity, and connectedness between researcher and researched, and between researchers and the communities in which they live and work” (Ellis, 2007, p. 4). (Tracy, 2010, p.847)

Christians (2005) introduced the concept of feminist communitarianism as a philosophy that stresses promise keeping, relationships, caring, collaboration, intimacy, emotionality, and connectedness. Such an approach stresses the primacy of relationships, compassion, nurturance, affection, promise keeping, and intimacy—interlocking “personal autonomy with communal well-being” (p. 151). (Tracy, 2010, p.848)

ethical considerations continue beyond the data collection phase to how researchers leave the scene and share the results. Certainly, researchers never have full control over how their work will read, be understood, and used. However, they can consider how best to present the research so as to avoid unjust or unintended consequences. (Tracy, 2010, p.848)

Stories about people who are poor, stigmatized, abused, or otherwise marginalized can serve to further negatively portray such people—even if that is not the intent of the author. (Tracy, 2010, p.848)

authors may choose to publish a “Legend of Cautions” that warns readers about the ways that the research analyses may be misread, misappropriated, or misused. Although it is rare to see such a formal legend, researchers can take care to present findings so as to ward off victim blaming and their unjust appropriation. (Tracy, 2010, p.848)

Meaningfully coherent studies (a) achieve their stated purpose; (b) accomplish what they espouse to be about; (c) use methods and representation practices that partner well with espoused theories and paradigms; and (d) attentively interconnect literature reviewed with research foci, methods, and findings. (Tracy, 2010, p.848)

studies that are meaningfully coherent eloquently interconnect their research design, data collection, and analysis with their theoretical framework and situational goals. (Tracy, 2010, p.848)

If the researcher champions a use-inspired approach, then the report should be written in simple language and clearly present practical implications—for instance in tables or bulleted points (Tracy & Rivera, 2010). (Tracy, 2010, p.849)

The reviewed literature situates the findings. The findings attend to the stated research questions or foci. Finally, the conclusions and implications meaningfully interconnect with the literature and data presented. (Tracy, 2010, p.849)

Good qualitative research is like a crystal, with various facets representing the aims, needs, and desires of various stakeholders including participants, the academy, society, lay public, policy makers, and last, but certainly not least, the researcher (Ellingson, 2008). (Tracy, 2010, p.850)