Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

  • We're Hiring!
  • Help Center

paper cover thumbnail

Research Methodology- An Introduction To Literary Studies

Profile image of Asst. Prof. Dr. Anila A . Pillai

2019, CENTRAL ASIAN JOURNAL OF LITERATURE, PHILOSOPHY AND CULTURE

becomes a kind of research and development. Research is done with the help of study, experiment, observation, analysis, comparison and reasoning. Research is in fact ubiquitous (everywhere). For example, we know that cigarette smoking is injurious to health; cow dung is a useful source of biogas; malaria is due to the virus protozoan plasmodium; AIDS (Acquired Immuno Deficiency Syndrome) is due to the virus HIV (Human Immuno Deficiency Virus). How did we know all these? We became aware of all these information only through research. More precisely, it seeks predictions of events, explanations, relationships and theories for them.

Related Papers

FEMINIZM IN TURKEY AND THE ISSUES OF TURKISH WOMEN WRITERS/ PINAR KÜR AND DUYGU ASENA CONCEPTUAL ISSUES OF ART/

mzisa buskivadze

literary studies research methods

Ahmed ALDUAIS

This study aims to present a general consideration for the postmodern paradigmatic shift and the reflection of these debates about literature studies which has had an important change in 20th century. It is a well-known fact that application of positive scientific methods has been inadequate about the expected results. The first reason behind this assumption is that it examines the phenomenon of literature and these natural sciences methods are not conductive to analyze these artistic methods. Postmodern paradigm presents a more comprehensive ground for the arts and the sciences dealing with arts. Due to the identicalness between arts reception and reviewer, it finds problematic applying the scientific methods to artistic facts art and objectifying the analysis carried out by the artistic sciences. In fact, postmodernism in the literature analysis have began effective in the late of 1980s, but it can be mentioned about the relativist and indeterminist traditions dating back to 1960s. After 1980s, Eurocentric theories such as Poststructuralism, Intertextuality and Reader Response; theories based on social criticism such as Critical Theory and Feminist Theory began to join to these same postmodern objections. These objections' common features can be listed as multidisciplinary versus scientific decomposition, eclecticism versus mono methodology; quantitative versus qualitative.

Irma Vilà Òdena

The Import of Literary Criticism

Abstract With the present global inclination to science, technology, and agriculture, criticism seems to have been tilted in that direction. The flame of reading literature seems to be going out gradually but steadily and with it, the reading and mastery of its prescriptive theories and their application to literary texts. If we consider that literature is concerned with life’s affairs which are more pressing than the reading black and white symbols of a text and their abstract interpretation, if we consider that literature is a vortex where distant and usually seemingly unrelated areas of human knowledge meet and interact, that war, the environment, the economy and even politics are involved in literary studies, then we would try to value the text more, and its criticism which is the decoding of the message of the text, which sometimes has a problem solving perspective, would be given its due import, since literature is in fact, a pattern of infinite potentialities, proffered by an absence presence, which have to be investigated through the right procedures. Because this domain is involved with every sphere of life, the import of getting its message cannot be overrated. However, the shade of meanings got without the use of literary theories would be highly informal, judgmental, and oftentimes superficial. Literary criticism is meant to throw light into the text through theories, and to assess the public comprehension of the text. In this vein, criticism is more important than the text, for it says in several ways what the text says. The main thrust of this paper is its attempt at encouraging literary criticism by showing how beneficial it is to the reader, writer and critic as individuals and to the society as a whole. Keyword: Criticism

Kelsay E Myers

This presentation of literary works that engage arts-based methods of research was created for Dr. Daniel Deslaurier's Integral Research: Arts and Imagination-Based Methods course in the Transformative Inquiry Department at California Institute of Integral Studies. Since Patricia Leavy (2015) focuses on qualitative studies that use ABR (that is her background, after all) in her book Method Meets Art: Arts-Based Research Practice, this presentation will focus on the creative, or artistic, aspects of literary writing to offer further examples of how literary aesthetics can enhance your own inquiries and writing (as that is my background).

Zulfiya Pardaeva

The article explores the features of the philological analysis of a literary text, in the implementation of which the integration of research methods is used. One of the manifestations of linguistic possibilities in a literary text is the verbal reproduction of a color detail. Color detail-a way of reflecting the romantic worldview, helps the poet to determine his attitude to the world. Using the methods of linguo-poetic analysis, the meaning of color in the lyrics of Marina Tsvetaeva is interpreted. The philological analysis of a literary text is designed to consolidate the skills and abilities acquired by students in the study of the main substantive provisions of linguistics and literary criticism.

Husain Necklace

This is a humble guide towards analyzing Literature for students of Literature as well as any audience or individuals who desire to understand and analyze Literature. It consist of no literary theories, only the basic ways of analyzing Literature which will help the readers comprehend literary texts much like any professional critique or analysts.

Bharat Gugane

Since its emergence, critical faculty has been following literature. The major concerns of critical enquiry have been focused upon the interpretation, evaluation and appreciation. Moreover it was engaged in affirming the canons of literary studies. Literature and literary studies are two different enterprises. The first one is concerned with creativity whereas the second one is related to learning. The faculty of critical study belongs to the second category. It is about the science of literature. Critical study analyses literature in terms of its nature and function. In all attempts defining literature, critics deal with contemporary instances of literature. While doing so critics focus on the content, and the biography of author. In other words he/she tries to decipher the text in terms of its social, economic and cultural underpinnings. This scholarly endeavour went through several transformations over a long period of time. It is interesting to see the evolution of critical stud...

Brazilian journalism research

Claudia Lago

RELATED PAPERS

Revista de la Facultad de Medicina, Universidad Nacional de Colombia

Jorge Chabur

ahmed R. G.

Tamás Gusztáv Filep

AJARCDE (Asian Journal of Applied Research for Community Development and Empowerment)

Endah Puspitojati

Surgical Neurology

Minoru Fujiki

Prof.Shosh Shahrabani

Jurnal Mahasiswa Jurusan Arsitektur

FIRMAN AKBAR R.

Tạp chí Nghiên cứu Y học

20070989 Đỗ Thị Kiều Trang

francis adrian rivera

Actas Urológicas Españolas

antonio perez

Bruce Elmegreen

Manual therapy

Marie-martine Lefèvre-colau

Pedro Duarte

Haroldo Teixeira

Nkululeko Mazibuko

Psychology in Russia

Olga Kliueva

Frontiers in Marine Science

Barbro Taraldset Haugland

Ciencia - Academia Mexicana de Ciencias

Adolfo Guzmán Arenas

Biological Invasions

Adriano Martinoli

Journal of Asian Scientific Research

Katrin Roosita Roosita

Sawwa: Jurnal Studi Gender

candra nuraini

Fabrice Tambe Endoh

Aaps Pharmscitech

Chadarat Ampasavate

Antonia Ćurić

RELATED TOPICS

  •   We're Hiring!
  •   Help Center
  • Find new research papers in:
  • Health Sciences
  • Earth Sciences
  • Cognitive Science
  • Mathematics
  • Computer Science
  • Academia ©2024

Literary Research Guide

An annotated listing of reference sources in english literary studies, james l. harner.

Copyright © 2008 by The Modern Language Association of America

This version of the Literary Research Guide is based on the sixth edition, published by the Modern Language Association in 2014. The Modern Language Association has since made the HTML, XML, and CSS of this edition available on GitHub under the terms of a CC-BY 4.0 license. The Modern Language Association did not create this version and is not responsible for the contents, display, accessibility, or any other aspect of it.
  • Art of Literary Research
  • Modern Researcher
  • Introduction to Scholarship in Modern Languages and Literatures
  • On Compiling an Annotated Bibliography
  • Attributing Authorship
  • ““ Rise of Periodical Studies ””
  • Guide to Reference
  • New Walford Guide to Reference Resources
  • Reader’s Adviser
  • General Subject-Indexes since 1548
  • Research Guide for Undergraduate Students
  • Guide to English and American Literature
  • Reference Guide for English Studies
  • Handbook to Literature
  • Dictionary of Literary Terms and Literary Theory
  • Oxford Classical Dictionary
  • Oxford Guide to Classical Mythology in the Arts
  • Dictionary of Literary Symbols
  • Benét’s Reader’s Encyclopedia
  • Cambridge Guide to Literature in English
  • Kindlers neues Literatur Lexikon
  • Serial Bibliographies
  • Other Bibliographies
  • Bibliographies
  • Background Reading
  • Guides to Libraries
  • Guides to Collections
  • Interlibrary Loan
  • Bibliographies and Guides
  • Union Catalogs
  • Catalogs of National Libraries
  • Use of Manuscripts in Literary Research
  • Dictionary of English Manuscript Terminology
  • Archive Finder
  • British Archives
  • “ National Archives ”
  • Guide to the National Archives of the United States
  • Catalogs of Major Collections
  • Schoenberg Database of Manuscripts
  • Guide to Serial Bibliographies for Modern Literatures
  • Early Periodical Indexes
  • Surveys of Research
  • Literature Bibliographies
  • General Bibliographies
  • Bibliographies of Bibliographies
  • Abstracts of English Studies
  • English and American Studies in German
  • Guide to Theses and Dissertations
  • Dissertation Abstracts
  • Canadian Theses / Thèses canadiennes
  • ProQuest Dissertations and Theses
  • Union List of Higher Degree Theses in Australian University Libraries
  • Catalogue of Austrian and Swiss Dissertations on English and American Literature
  • Dissertations in English and American Literature
  • Language and Literature of the Anglo-Saxon Nations
  • 18thConnect: Eighteenth-Century Scholarship Online
  • Literature Online
  • Literature Resource Center
  • Oxford Reference
  • Gale Virtual Reference Library
  • Guides to Biographies
  • Guides to Scholarship and Criticism
  • General Biographical Dictionaries
  • Dictionaries of Writers
  • ““ Digital Burney Newspapers Collection ””
  • MLA Directory of Periodicals
  • Author Newsletters and Journals
  • Related Works
  • New Serial Titles
  • Serials in the British Library
  • Project Muse
  • IngentaConnect
  • American Reference Books Annual
  • Times Literary Supplement
  • Guides to Reference Works
  • Histories and Surveys
  • Guides to Primary Works
  • Gothic and Horror Fiction
  • Historical Fiction
  • Mystery Fiction
  • Picaresque Fiction
  • Science and Fantasy Fiction
  • Utopian Fiction
  • Short Fiction (Short Story, Novella)
  • Literary Handbooks, Dictionaries, and Encyclopedias
  • Biographical Dictionaries
  • Biography and Autobiography
  • Travel Writing
  • Periodicals
  • Research Methods
  • Literary Research and Irish Literature
  • Cambridge History of Irish Literature
  • Dictionary of Irish Literature
  • Guide to Irish Bibliographical Material
  • Manuscripts
  • Printed Works
  • Dissertations and Theses
  • Guides to Scholarship
  • Dictionary of Irish Biography
  • Biographical Dictionary of Irish Writers
  • Drama and Theater
  • Edinburgh History of Scottish Literature
  • Companion to Scottish Literature
  • Scotland: A Literary Guide
  • Dictionaries
  • Guide to Welsh Literature
  • History of Welsh Literature
  • New Companion to the Literature of Wales
  • Bibliography of Anglo-Welsh Literature
  • Bywgraffiadur Ar-lein / Welsh Biography Online
  • Eastern Literature
  • Midwestern Literature
  • Southern Literature
  • Western Literature
  • African American Literature
  • American Indian Literatures
  • Asian American Literature
  • Hispanic American Literature
  • Jewish American Literature
  • Literary Research and Postcolonial Literatures in English
  • ““ Researching African Literatures ””
  • Asian Literature in English
  • Manuel bibliographique des études littéraires
  • Literaturrecherche für Germanisten
  • Manuale critico-bibliografico per lo studio della letteratura italiana
  • Companion to Neo-Latin Studies
  • Introduction to Medieval Latin
  • Portuguese Language and Luso-Brazilian Literature
  • Russian Reference Works
  • Guide to Reference Works for the Study of the Spanish Language and Literature
  • Comparative Literature and Literary Theory
  • Acronyms, Initialisms, and Abbreviations Dictionary
  • Individuals
  • Institutions
  • Organizations
  • General Introductions
  • Handbooks, Dictionaries, and Encyclopedias
  • Author Bibliographies
  • Descriptive Bibliography
  • Textual Criticism
  • Book Trade, History of the Book, and Publishing History
  • Directories of Book Dealers
  • Biographical Dictionaries of Collectors
  • Booksellers’ and Auction Catalogs
  • Computer Programs
  • Grants Register
  • Histories of the English Language
  • Handbooks, Linguistic Dictionaries, and Encyclopedias
  • General Linguistics
  • Special Topics
  • Types, Schools, and Movements
  • Guide to Microforms and Digital Resources
  • Microform Research Collections
  • Handbooks and Guides to Publishing
  • Directories of Publishing Opportunities
  • Guides to Writing
  • Style Manuals
  • Book Reviewing
  • History and Literature
  • Political Science and Literature
  • Psychology and Literature
  • Sociology and Literature
  • Index of Names
  • Index of Titles
  • Subject Index
  • Bibliography
  • Master's programmes in English
  • For exchange students
  • PhD opportunities
  • All programmes of study
  • Language requirements
  • Application process
  • Academic calendar
  • NTNU research
  • Research excellence
  • Strategic research areas
  • Innovation resources
  • Student in Trondheim
  • Student in Gjøvik
  • Student in Ålesund
  • For researchers
  • Life and housing
  • Faculties and departments
  • International researcher support

Språkvelger

Course - research methods in literature studies - litt3001, course-details-portlet, litt3001 - research methods in literature studies, examination arrangement.

Examination arrangement: Home examination Grade: Letter grades

Course content

This course will provide insight into central methodological issues within the field of literature studies. A primary goal is to give students an understanding of the diversity of methods that characterizes literary criticism. The course aims to train students in developing research questions and to assess the applicability and relevance of different research methods, thus providing them with the foundation for developing individual master's projects. The course will provide students with understanding of the connections between critical perspectives and methodological approaches.

Among others, the course will discuss the following topics:

- Methods for examining the relationship between literature and society, literature and history, author and text, and questions related to the literary canon

- Methods for examining the relationship between literature and form, genre and thematics

- Methods for examining the relationship between literature, reading and reception

- The use of physical and digital archives

- Methods for researching literature and education

Learning outcome

Candidates who have passed this course

  • have the knowledge to evaluate the applicability and relevance of different research methods in the research of others as well as in their own
  • can develop research questions in literature studies and at the same time describe what methods may be applicable to examine various problems
  • can discuss how different approaches to literature may result in different interpretations of the same text
  • are able to apply terms in, and knowledge of, literature studies in practical work with literature
  • are able to find and evaluate literary criticism

Learning methods and activities

Lectures/seminars. The course is taught in English or Norwegian. Students are expected to participate actively in the lectures/seminars for example by presentations, comments and other contributions. Students are required to use the course learning platform regularly.

Obligatory assignment: A written assignment in groups where students discuss different methodological choices in working with a literary text. An approved obligatory assignment is valid for 2 semesters (the semester in which the approval is given, plus the following semester).

Compulsory assignments

  • 1 written assignment (in groups)

Further on evaluation

The course is assessed by a written home examination (approx. 4000 words, 5 days). The exam must be answered in Norwegian or English.

Recommended previous knowledge

Relevant undergraduate courses in literature.

Required previous knowledge

At least 7.5 credits (studiepoeng) of courses in literature at ENG2000-, FRA2000-, NORD2000-, or TYSK2000-level.

Course materials

A pensum consisting of 700-800 pages of articles/chapter excerpts focused on research methods, plus a limited selection of short literary texts. Curriculum/reading list will be made available at the beginning of the semester.

Credit reductions

  • Blackboard - AUTUMN-2021

Version: 1 Credits:  7.5 SP Study level: Second degree level

Term no.: 1 Teaching semester:  AUTUMN 2023

Language of instruction: Norwegian

Location: Trondheim

  • English Literature
  • French Literature
  • Comparative Literature
  • Scandinavian Language and Literature
  • Scandinavian Literature
  • German Literature

Department with academic responsibility Department of Language and Literature

Examination

Examination arrangement: home examination.

Release 2023-11-30

Submission 2023-12-05

Release 2024-03-04

Submission 2024-03-08

  • * The location (room) for a written examination is published 3 days before examination date. If more than one room is listed, you will find your room at Studentweb.

For more information regarding registration for examination and examination procedures, see "Innsida - Exams"

More on examinations at NTNU

  • Harvard Library
  • Research Guides
  • Faculty of Arts & Sciences Libraries

Library Research Guide for Folklore and Mythology

  • Theory and Methods

What is Research?

Practicing folklore.

  • Research Design & Tools

Research Theories and Methods

  • Ethical Guidelines in Different Disciplines
  • Searching HOLLIS
  • Databases & Bibliographies
  • Encyclopedias
  • Dictionaries
  • Tale-Type and Motif Indices
  • Archives & Special Collections
  • Material Culture
  • Audiovisual Materials & Musical Scores
  • Requesting & Accessing Materials
  • Organizing and Managing Your Resarch
  • Learning With the Library

Research  is the systematic investigation of a subject, topic, or question. 

Data  is the information gathered during research.

Fieldwork  is the collection of data in its natural environment.

A white paper is a report or guide that synthesizes a complex topic or question and the state of information and ideas about it.

Scholarship  is, broadly, the activity of a scholar. More specifically though, the term refers to the writings of scholars which result from their research. The scholarship of a field or discipline are the books, articles, etc. which have been written on the field or discipline, or on a specific subject, topic, or question in the field or discipline.  

What is a theory?

A  theory  is the conceptual basis of a subject or area of study. It is the ideas which underlie how something is understood and the framework within which it is studied.  

What is a method?

A  method  is the process or tool used to collect data.

There are three method types: qualitative, quantitative, and historical. Likewise, some research uses mixed methods.

Qualitative research  is interested in the specific. It studies things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them, endeavoring to understand human behavior from the perspective of the individual.

Qualitative methods  collect data through observation. Qualitative methods include text analysis, interviews, focus groups, observation, record keeping, ethnographic research, case study research.

Qualitative data is descriptive. Qualitative data cannot be precisely measured and is, rather, analyzed for patterns and themes using coding. Qualitative data includes narratives, recordings, photographs, oral histories, etc.

Quantitative research  is interested in the general. It studies general laws of behavior and phenomena across different settings and contexts. This type of research endeavors to form conclusions about social phenomena, collecting data to test a theory and ultimately support or reject it.

Quantitative methods  collect data through measuring. Quantitative methods include experiments, surveys, questionnaires, statistical modeling, social networks, and demography.

Quantitative data  is numerical and statistical. It is data that can either be counted or compared on a numeric scale. Quantitative data includes statistical information. 

Historical research  is interested in the past. It reviews and interprets existing data to describe, explain, and understand past actions or events.

Historical methods  collect and analyze existing data and analyze it. Historical methods include text analysis, cultural analysis, visual analysis, archival research.

Historical data  is data which was created in the past. Historical data includes scholarship, records, artifacts.  

A methodology  is the rationale for the research approach and the methods used. It is based upon the theories underlying the field or discipline of the research.

Library of Congress YouTube Feed: Folklore

The American Folklife Center at the Library of Congress produces videos about the practice of folklore, featuring interviews with a variety of folklorists about their careers, methods, fieldwork experiences, and the implications and applications of their work.

book cover

Research Design: Qualitative, Quantitative, and Mixed Methods Approaches

John W. Creswell 2014, fourth edition

book cover

Research Design: Quantitative, Qualitative, Mixed Methods, Arts-Based, and Community-Based Participatory Research Approaches

Patricia Leavy 2017

  • Literatures
  • Linguistics
  • Anthropology
  • Human Geography

Cultural Studies

Folklore studies, literary studies.

Literary Studies, also called Literary Criticism, is the study of the written works of cultures, societies, groups, and individuals. Literary Studies examines the place of literature in society, and explores how we conceptualize and describe the world and ourselves.  

Literary Theories

There are a number of different theories about literature, why and how it is created. These theories influence how a work of literature is analyzed, interpreted, and understood. Literary Studies most often uses the method of textual analysis.

Cover Art

Linguistic Studies

Linguistics is the study of languages and their structures. Linguistic Studies examines how language is created and constructed, how it functions and is learned, and how we conceptualize and structure our world through our words.   

Language Theories

There are different theories about the creation and purpose of language. Some theories state that language is the result of the nature of society, while others emphasize the role of humans in constructing meaning. Linguistic Studies use methods such as textual analysis, ethnographic research, statistical modeling.

Cover Art

History Studies

History is the study of events, and their related ideas, individuals, and objects. History Studies examines how moments in time are connected, and how we make sense of things that happen.

Historiography  is the study of how historians have interpreted and written about historical events, in essence, how they perceive history itself. Traditionally, a historiography was a name for a history, literally a specific "writing of history".  

History Theories

There are many different theories about if and how events are related to one another, and these theories have influenced how history has been written about over the centuries. History Studies use methods such as textual analysis and archival research.

A related theory to history theories is Memory Theory , which considers how collective and individual memory is created and preserved. Memory Studies examines the ways in which events are recorded and remembered, or, alternatively, forgotten, and how we choose to create and remember (or forget) our past.

Cover Art

Anthropological Studies

Anthropology is the study of human societies, their behaviors and cultures. Anthropological Studies examine how societies are formed and function, and the many aspects which form our identities.

Social Anthropology  examines human behavior. Sometimes this sub-field is combined with Cultural Anthropology as Sociocultural Anthropology.

Cultural Anthropology  examines the cultures, or various beliefs and practices, of societies. Sometimes this sub-field is combined with Social Anthropology as Sociocultural Anthropology.

Physical Anthropology , also called Biological Anthropology, examines the biology of humans and how they interact with their environment.

Linguistic Anthropology  examines the place of language in shaping social life.

Archaeology  examines the material culture, or the objects, of humans. It is considered a sub-field of Anthropology in the United States, and a sub-field of History in other parts of the world.  

Ethnography is the study of a specific society using the methods of observation and immersion, or talking and living with individuals in order to understand them.   

Anthropological Theories

The is a long tradition of theories about how societies organize themselves and how they function. These theories determine how cultural beliefs and practices are understood, in essence, how we understand ourselves and others. Anthropology Studies use methods such as interviews, focus groups, observation, ethnographic research, and record keeping, as well as textual analysis and archival research.

Cover Art

Sociological Studies

Sociology is the study of societies, their behaviors, relationships, and interactions. It examines social order and social changes, trying to understand how and why we organize ourselves and relate to one another.

Historical Sociology   is the study of the behaviors and organization of societies of the past.   

Sociological Theories

There are different theories about how societies are structured and why they act the way they do. Sociological Studies often use the methods of surveys, experiments, ethnographic research, and textual analysis.

Sociological theories are theories about how the mechanics of societies function, whereas  Social Theory  encompasses more broadly theories which explain how societies think and act.

Cover Art

Geography Studies

Geography is the study of land, inhabitants, and natural phenomena. It examines the relationship between humans and their environment, and helps us to understand our relationship with the world. 

Human Geography  examines humans and their communities, and their relationships with place, space, and environment.

Physical Geography  examines the processes and patterns of environments, such as their atmosphere, hydrosphere, biosphere, and geosphere.

Cartography  is both the study of and the science and art of map-making. It reveals how we view and conceptualize the world and our relationship to it and to others.   

Geography Theories

There are a number of theories as to the relationship between humans and their environments, many of which are shared with the fields of Anthropology and Sociology. Geography Studies use a variety of research methods, including interviews, surveys, observation, and GIS or spatial analysis.

Cover Art

Cultural Studies is the study and analysis of culture. It is a cross-disciplinary field which examines the various aspects of a society, in order to understand how we form our identities. 

Culture  is the ideas, behaviors, customs, and objects of a region, society, group, or individual. 

Material culture   are the physical objects of a culture, such as tools, domestic objects, religious objects, works of art.  

Cultural Theories

Cultural theories draw upon theories in a variety of fields, including literary theories, semiotics, history theories, anthropological theories, social theories, museum studies, art history, and media studies. Cultural theories influence how we analyze and interpret the culture of societies. Cultural Studies tends to use methods such as interviews, observation, ethnographic research, record keeping, archival research, textual analysis, visual analysis.

Cover Art

Folklore Studies, also known as Folkloristics, is the study of the expressions of culture, particularly the practices and products of a society. Folklore Studies examines the things we make to understand how they make us.

Folklore  has been traditionally considered, narrowly, as the oral tales of a society. More broadly, the term refers to all aspects of a culture – beliefs, traditions, norms, behaviors, language, literature, jokes, music, art, foodways, tools, objects, etc.  

Folklore Theories

A number of theories have emerged over the years about how societies create themselves, and these theories influence how we view and understand the things which societies create. Folklore Studies use methods such as interviews, focus groups, observation, ethnographic research, and record keeping, as well as textual analysis, visual analysis and archival research.

Cover Art

Arts Studies

The arts are a range of disciplines which study, create, and engage with human expression. The arts include,

  • Architecture -- Design
  • Visual Arts -- Drawing, Painting, Illustration, Sculpting, Ceramics, Photography, Film
  • Literary Arts -- Fiction, Drama, Poetry, Creative Writing, Storytelling
  • Performance Arts -- Music, Dance, Theatre
  • Textile Arts -- Fashion
  • Craft -- Weaving, Woodwork, Paperwork, Glasswork, Jewelry-making
  • Culinary Arts -- Cooking, Baking, Chocolate-making, Brewing, Wine-making
  • Art History and Criticism

The arts are a collection of areas of studies which combine technical skills and creativity to produce objects which convey human experience.

Architecture  is the study and design of structures. It examines both the utilitarian and the sociological aspects of space, and the relationship between constructed space and humans. 

Art History  is the study and analysis of visual arts. 

Musicology  is the study and analysis of music.

Performance   is the study and the practice of art is time and space. 

Film & Media Studies  is the study of art which employs technologies.   

Art Theories

There are as many theories about the arts as there are areas of arts. These theories affect how we understand the identity and the agency of the artist, the meaning of the art, and the relationship between the art and society. Arts fields often employ textual and visual analysis research methods, as well as observation and experimentation. 

Cover Art

Folklorists study people's lives and thus they are responsible to preserve and protect culture. Folklorists are professionals and researchers and thus they have a responsibility to the field to uphold standards of behavior and work. Finally, folklorists interact with individuals and are responsible to uphold human rights. Though there is little direct legislation governing folklore studies, there are numerous laws concerning human rights and information, as well as professional standards in the field of cultural heritage preservation. 

Legislation

The codes of ethics and standards which govern folklore studies have been developed over time from a number of authorities.  

1948    United Nations, Universal Declaration of Human Rights

1948    American Anthropological Association, Resolution on Freedom of Publication

1971    American Anthropological Association, Principles of Professional Responsibility Statement of Ethics

1976    American Folklife Preservation Act (P.L. 94-201)

American Folklife Center established at the Library of Congress and given duty to preserve American folklife

1985    UNESCO, Protection of Expressions of Folklore Against Illicit Exploitation and Other Prejudicial Actions

1988    American Folklore Society, Statement of Ethics

1988    National Association for the Practice of Anthropology, Ethical Guidelines for Practitioners

1989    UNESCO, Recommendation on the Safeguarding of Traditional Culture and Folklore

1998    American Anthropological Association, Code of Ethics

2003    UNESCO, Convention for the Safeguarding of the Intangible Cultural Heritage

book cover

Handbook of Research Ethics and Scientific Integrity

Ron Iphofen, editor 2020

book cover

The Ethics of Research with Human Subjects

David B. Resnik 2018

book cover

The Ethics of Cultural Heritage

Tracy Ireland & John Schofeld 2014

book cover

Critical Ethnography

D. Soyini Madison 2005

book cover

Ethics in Ethnography

Margaret D. LeCompte & Jean J. Schensul 2015

book cover

The Ethics of Social Research

Joan E. Sieber, editor 1982

book cover

Research Ethics for Human Geography

Helen F. Wilson & Jonathan Darling, editors 2021

book cover

The Ethics of Cultural Studies

Joanna Zylinska 2005

book cover

Museum Collection Ethics

Steven Miller 2020

book cover

Theorizing Folklore from the Margins

Solimar Otero & Mintzi Auanda Martínez-Rivera, editors 2021

  • << Previous: Background
  • Next: Searching HOLLIS >>

Except where otherwise noted, this work is subject to a Creative Commons Attribution 4.0 International License , which allows anyone to share and adapt our material as long as proper attribution is given. For details and exceptions, see the Harvard Library Copyright Policy ©2021 Presidents and Fellows of Harvard College.

  • Privacy Policy
  • SignUp/Login

Research Method

Home » Textual Analysis – Types, Examples and Guide

Textual Analysis – Types, Examples and Guide

Table of Contents

Textual Analysis

Textual Analysis

Textual analysis is the process of examining a text in order to understand its meaning. It can be used to analyze any type of text, including literature , poetry, speeches, and scientific papers. Textual analysis involves analyzing the structure, content, and style of a text.

Textual analysis can be used to understand a text’s author, date, and audience. It can also reveal how a text was constructed and how it functions as a piece of communication.

Textual Analysis in Research

Textual analysis is a valuable tool in research because it allows researchers to examine and interpret text data in a systematic and rigorous way. Here are some ways that textual analysis can be used in research:

  • To explore research questions: Textual analysis can be used to explore research questions in various fields, such as literature, media studies, and social sciences. It can provide insight into the meaning, interpretation, and communication patterns of text.
  • To identify patterns and themes: Textual analysis can help identify patterns and themes within a set of text data, such as analyzing the representation of gender or race in media.
  • To evaluate interventions: Textual analysis can be used to evaluate the effectiveness of interventions, such as analyzing the language and messaging of public health campaigns.
  • To inform policy and practice: Textual analysis can provide insights that inform policy and practice, such as analyzing legal documents to inform policy decisions.
  • To analyze historical data: Textual analysis can be used to analyze historical data, such as letters, diaries, and newspapers, to provide insights into historical events and social contexts.

Textual Analysis in Cultural and Media Studies

Textual analysis is a key tool in cultural and media studies as it enables researchers to analyze the meanings, representations, and discourses present in cultural and media texts. Here are some ways that textual analysis is used in cultural and media studies:

  • To analyze representation: Textual analysis can be used to analyze the representation of different social groups, such as gender, race, and sexuality, in media and cultural texts. This analysis can provide insights into how these groups are constructed and represented in society.
  • To analyze cultural meanings: Textual analysis can be used to analyze the cultural meanings and symbols present in media and cultural texts. This analysis can provide insights into how culture and society are constructed and understood.
  • To analyze discourse: Textual analysis can be used to analyze the discourse present in cultural and media texts. This analysis can provide insights into how language is used to construct meaning and power relations.
  • To analyze media content: Textual analysis can be used to analyze media content, such as news articles, TV shows, and films, to understand how they shape our understanding of the world around us.
  • To analyze advertising : Textual analysis can be used to analyze advertising campaigns to understand how they construct meanings, identities, and desires.

Textual Analysis in the Social Sciences

Textual analysis is a valuable tool in the social sciences as it enables researchers to analyze and interpret text data in a systematic and rigorous way. Here are some ways that textual analysis is used in the social sciences:

  • To analyze interview data: Textual analysis can be used to analyze interview data, such as transcribed interviews, to identify patterns and themes in the data.
  • To analyze survey responses: Textual analysis can be used to analyze survey responses to identify patterns and themes in the data.
  • To analyze social media data: Textual analysis can be used to analyze social media data, such as tweets and Facebook posts, to identify patterns and themes in the data.
  • To analyze policy documents: Textual analysis can be used to analyze policy documents, such as government reports and legislation, to identify discourses and power relations present in the policy.
  • To analyze historical data: Textual analysis can be used to analyze historical data, such as letters and diaries, to provide insights into historical events and social contexts.

Textual Analysis in Literary Studies

Textual analysis is a key tool in literary studies as it enables researchers to analyze and interpret literary texts in a systematic and rigorous way. Here are some ways that textual analysis is used in literary studies:

  • To analyze narrative structure: Textual analysis can be used to analyze the narrative structure of a literary text, such as identifying the plot, character development, and point of view.
  • To analyze language and style: Textual analysis can be used to analyze the language and style used in a literary text, such as identifying figurative language, symbolism, and rhetorical devices.
  • To analyze themes and motifs: Textual analysis can be used to analyze the themes and motifs present in a literary text, such as identifying recurring symbols, themes, and motifs.
  • To analyze historical and cultural context: Textual analysis can be used to analyze the historical and cultural context of a literary text, such as identifying how the text reflects the social and political context of its time.
  • To analyze intertextuality: Textual analysis can be used to analyze the intertextuality of a literary text, such as identifying how the text references or is influenced by other literary works.

Textual Analysis Methods

Textual analysis methods are techniques used to analyze and interpret various types of text, including written documents, audio and video recordings, and online content. These methods are commonly used in fields such as linguistics, communication studies, sociology, psychology, and literature.

Some common textual analysis methods include:

Content Analysis

This involves identifying patterns and themes within a set of text data. This method is often used to analyze media content or other types of written materials, such as policy documents or legal briefs.

Discourse Analysis

This involves examining how language is used to construct meaning in social contexts. This method is often used to analyze political speeches or other types of public discourse.

Critical Discourse Analysis

This involves examining how power and social relations are constructed through language use, particularly in political and social contexts.

Narrative Analysis

This involves examining the structure and content of stories or narratives within a set of text data. This method is often used to analyze literary texts or oral histories.

This involves analyzing the meaning of signs and symbols within a set of text data. This method is often used to analyze advertising or other types of visual media.

Text mining

This involves using computational techniques to extract patterns and insights from large sets of text data. This method is often used in fields such as marketing and social media analysis.

Close Reading

This involves a detailed and in-depth analysis of a particular text, focusing on the language, style, and literary techniques used by the author.

How to Conduct Textual Analysis

Here are some general steps to conduct textual analysis:

  • Choose your research question: Define your research question and identify the text or set of texts that you want to analyze.
  • F amiliarize yourself with the text: Read and re-read the text, paying close attention to its language, structure, and content. Take notes on key themes, patterns, and ideas that emerge.
  • Choose your analytical approach: Select the appropriate analytical approach for your research question, such as close reading, thematic analysis, content analysis, or discourse analysis.
  • Create a coding scheme: If you are conducting content analysis, create a coding scheme to categorize and analyze the content of the text. This may involve identifying specific words, themes, or ideas to code.
  • Code the text: Apply your coding scheme to the text and systematically categorize the content based on the identified themes or patterns.
  • Analyze the data: Once you have coded the text, analyze the data to identify key patterns, themes, or trends. Use appropriate software or tools to help with this process if needed.
  • Draw conclusions: Draw conclusions based on your analysis and answer your research question. Present your findings and provide evidence to support your conclusions.
  • R eflect on limitations and implications: Reflect on the limitations of your analysis, such as any biases or limitations of the selected method. Also, discuss the implications of your findings and their relevance to the broader research field.

When to use Textual Analysis

Textual analysis can be used in various research fields and contexts. Here are some situations when textual analysis can be useful:

  • Understanding meaning and interpretation: Textual analysis can help understand the meaning and interpretation of text, such as literature, media, and social media.
  • Analyzing communication patterns: Textual analysis can be used to analyze communication patterns in different contexts, such as political speeches, social media conversations, and legal documents.
  • Exploring cultural and social contexts: Textual analysis can be used to explore cultural and social contexts, such as the representation of gender, race, and identity in media.
  • Examining historical documents: Textual analysis can be used to examine historical documents, such as letters, diaries, and newspapers.
  • Evaluating marketing and advertising campaigns: Textual analysis can be used to evaluate marketing and advertising campaigns, such as analyzing the language, symbols, and imagery used.

Examples of Textual Analysis

Here are a few examples:

  • Media Analysis: Textual analysis is frequently used in media studies to examine how news outlets and social media platforms frame and present news stories. Researchers can use textual analysis to examine the language and images used in news articles, tweets, and other forms of media to identify patterns and biases.
  • Customer Feedback Analysis: Textual analysis is often used by businesses to analyze customer feedback, such as online reviews or social media posts, to identify common themes and areas for improvement. This allows companies to make data-driven decisions and improve their products or services.
  • Political Discourse Analysis: Textual analysis is commonly used in political science to analyze political speeches, debates, and other forms of political communication. Researchers can use this method to identify the language and rhetoric used by politicians, as well as the strategies they employ to appeal to different audiences.
  • Literary Analysis: Textual analysis is a fundamental tool in literary criticism, allowing scholars to examine the language, structure, and themes of literary works. This can involve close reading of individual texts or analysis of larger literary movements.
  • Sentiment Analysis: Textual analysis is used to analyze social media posts, customer feedback, or other sources of text data to determine the sentiment of the text. This can be useful for businesses or organizations to understand how their brand or product is perceived in the market.

Purpose of Textual Analysis

There are several specific purposes for using textual analysis, including:

  • To identify and interpret patterns in language use: Textual analysis can help researchers identify patterns in language use, such as common themes, recurring phrases, and rhetorical devices. This can provide insights into the values and beliefs that underpin the text.
  • To explore the cultural context of the text: Textual analysis can help researchers understand the cultural context in which the text was produced, including the historical, social, and political factors that shaped the language and messages.
  • To examine the intended and unintended meanings of the text: Textual analysis can help researchers uncover both the intended and unintended meanings of the text, and to explore how the language is used to convey certain messages or values.
  • To understand how texts create and reinforce social and cultural identities: Textual analysis can help researchers understand how texts contribute to the creation and reinforcement of social and cultural identities, such as gender, race, ethnicity, and nationality.

Applications of Textual Analysis

Here are some common applications of textual analysis:

Media Studies

Textual analysis is frequently used in media studies to analyze news articles, advertisements, and social media posts to identify patterns and biases in media representation.

Literary Criticism

Textual analysis is a fundamental tool in literary criticism, allowing scholars to examine the language, structure, and themes of literary works.

Political Science

Textual analysis is commonly used in political science to analyze political speeches, debates, and other forms of political communication.

Marketing and Consumer Research

Textual analysis is used to analyze customer feedback, such as online reviews or social media posts, to identify common themes and areas for improvement.

Healthcare Research

Textual analysis is used to analyze patient feedback and medical records to identify patterns in patient experiences and improve healthcare services.

Social Sciences

Textual analysis is used in various fields within social sciences, such as sociology, anthropology, and psychology, to analyze various forms of data, including interviews, field notes, and documents.

Linguistics

Textual analysis is used in linguistics to study language use and its relationship to social and cultural contexts.

Advantages of Textual Analysis

There are several advantages of textual analysis in research. Here are some of the key advantages:

  • Systematic and objective: Textual analysis is a systematic and objective method of analyzing text data. It enables researchers to analyze text data in a consistent and rigorous way, minimizing the risk of bias or subjectivity.
  • Versatile : Textual analysis can be used to analyze a wide range of text data, including interview transcripts, survey responses, social media data, policy documents, and literary texts.
  • Efficient : Textual analysis can be a more efficient method of data analysis compared to manual coding or other methods of qualitative analysis. With the help of software tools, researchers can process large volumes of text data more quickly and accurately.
  • Allows for in-depth analysis: Textual analysis enables researchers to conduct in-depth analysis of text data, uncovering patterns and themes that may not be visible through other methods of data analysis.
  • Can provide rich insights: Textual analysis can provide rich and detailed insights into complex social phenomena. It can uncover subtle nuances in language use, reveal underlying meanings and discourses, and shed light on the ways in which social structures and power relations are constructed and maintained.

Limitations of Textual Analysis

While textual analysis can provide valuable insights into the ways in which language is used to convey meaning and create social and cultural identities, it also has several limitations. Some of these limitations include:

  • Limited Scope : Textual analysis is only able to analyze the content of written or spoken language, and does not provide insights into non-verbal communication such as facial expressions or body language.
  • Subjectivity: Textual analysis is subject to the biases and interpretations of the researcher, as well as the context in which the language was produced. Different researchers may interpret the same text in different ways, leading to inconsistencies in the findings.
  • Time-consuming: Textual analysis can be a time-consuming process, particularly if the researcher is analyzing a large amount of text. This can be a limitation in situations where quick analysis is necessary.
  • Lack of Generalizability: Textual analysis is often used in qualitative research, which means that its findings cannot be generalized to larger populations. This limits the ability to draw conclusions that are applicable to a wider range of contexts.
  • Limited Accessibility: Textual analysis requires specialized skills and training, which may limit its accessibility to researchers who are not trained in this method.

About the author

' src=

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Case Study Research

Case Study – Methods, Examples and Guide

Probability Histogram

Probability Histogram – Definition, Examples and...

Substantive Framework

Substantive Framework – Types, Methods and...

Factor Analysis

Factor Analysis – Steps, Methods and Examples

Qualitative Research

Qualitative Research – Methods, Analysis Types...

Descriptive Research Design

Descriptive Research Design – Types, Methods and...

  • Methodology
  • Open access
  • Published: 11 October 2016

Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research

  • Stephen J. Gentles 1 , 4 ,
  • Cathy Charles 1 ,
  • David B. Nicholas 2 ,
  • Jenny Ploeg 3 &
  • K. Ann McKibbon 1  

Systematic Reviews volume  5 , Article number:  172 ( 2016 ) Cite this article

49k Accesses

26 Citations

13 Altmetric

Metrics details

Overviews of methods are potentially useful means to increase clarity and enhance collective understanding of specific methods topics that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness. This type of review represents a distinct literature synthesis method, although to date, its methodology remains relatively undeveloped despite several aspects that demand unique review procedures. The purpose of this paper is to initiate discussion about what a rigorous systematic approach to reviews of methods, referred to here as systematic methods overviews , might look like by providing tentative suggestions for approaching specific challenges likely to be encountered. The guidance offered here was derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research.

The guidance is organized into several principles that highlight specific objectives for this type of review given the common challenges that must be overcome to achieve them. Optional strategies for achieving each principle are also proposed, along with discussion of how they were successfully implemented in the overview on sampling. We describe seven paired principles and strategies that address the following aspects: delimiting the initial set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology used to describe specific methods topics, and generating rigorous verifiable analytic interpretations. Since a broad aim in systematic methods overviews is to describe and interpret the relevant literature in qualitative terms, we suggest that iterative decision making at various stages of the review process, and a rigorous qualitative approach to analysis are necessary features of this review type.

Conclusions

We believe that the principles and strategies provided here will be useful to anyone choosing to undertake a systematic methods overview. This paper represents an initial effort to promote high quality critical evaluations of the literature regarding problematic methods topics, which have the potential to promote clearer, shared understandings, and accelerate advances in research methods. Further work is warranted to develop more definitive guidance.

Peer Review reports

While reviews of methods are not new, they represent a distinct review type whose methodology remains relatively under-addressed in the literature despite the clear implications for unique review procedures. One of few examples to describe it is a chapter containing reflections of two contributing authors in a book of 21 reviews on methodological topics compiled for the British National Health Service, Health Technology Assessment Program [ 1 ]. Notable is their observation of how the differences between the methods reviews and conventional quantitative systematic reviews, specifically attributable to their varying content and purpose, have implications for defining what qualifies as systematic. While the authors describe general aspects of “systematicity” (including rigorous application of a methodical search, abstraction, and analysis), they also describe a high degree of variation within the category of methods reviews itself and so offer little in the way of concrete guidance. In this paper, we present tentative concrete guidance, in the form of a preliminary set of proposed principles and optional strategies, for a rigorous systematic approach to reviewing and evaluating the literature on quantitative or qualitative methods topics. For purposes of this article, we have used the term systematic methods overview to emphasize the notion of a systematic approach to such reviews.

The conventional focus of rigorous literature reviews (i.e., review types for which systematic methods have been codified, including the various approaches to quantitative systematic reviews [ 2 – 4 ], and the numerous forms of qualitative and mixed methods literature synthesis [ 5 – 10 ]) is to synthesize empirical research findings from multiple studies. By contrast, the focus of overviews of methods, including the systematic approach we advocate, is to synthesize guidance on methods topics. The literature consulted for such reviews may include the methods literature, methods-relevant sections of empirical research reports, or both. Thus, this paper adds to previous work published in this journal—namely, recent preliminary guidance for conducting reviews of theory [ 11 ]—that has extended the application of systematic review methods to novel review types that are concerned with subject matter other than empirical research findings.

Published examples of methods overviews illustrate the varying objectives they can have. One objective is to establish methodological standards for appraisal purposes. For example, reviews of existing quality appraisal standards have been used to propose universal standards for appraising the quality of primary qualitative research [ 12 ] or evaluating qualitative research reports [ 13 ]. A second objective is to survey the methods-relevant sections of empirical research reports to establish current practices on methods use and reporting practices, which Moher and colleagues [ 14 ] recommend as a means for establishing the needs to be addressed in reporting guidelines (see, for example [ 15 , 16 ]). A third objective for a methods review is to offer clarity and enhance collective understanding regarding a specific methods topic that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness within the available methods literature. An example of this is a overview whose objective was to review the inconsistent definitions of intention-to-treat analysis (the methodologically preferred approach to analyze randomized controlled trial data) that have been offered in the methods literature and propose a solution for improving conceptual clarity [ 17 ]. Such reviews are warranted because students and researchers who must learn or apply research methods typically lack the time to systematically search, retrieve, review, and compare the available literature to develop a thorough and critical sense of the varied approaches regarding certain controversial or ambiguous methods topics.

While systematic methods overviews , as a review type, include both reviews of the methods literature and reviews of methods-relevant sections from empirical study reports, the guidance provided here is primarily applicable to reviews of the methods literature since it was derived from the experience of conducting such a review [ 18 ], described below. To our knowledge, there are no well-developed proposals on how to rigorously conduct such reviews. Such guidance would have the potential to improve the thoroughness and credibility of critical evaluations of the methods literature, which could increase their utility as a tool for generating understandings that advance research methods, both qualitative and quantitative. Our aim in this paper is thus to initiate discussion about what might constitute a rigorous approach to systematic methods overviews. While we hope to promote rigor in the conduct of systematic methods overviews wherever possible, we do not wish to suggest that all methods overviews need be conducted to the same standard. Rather, we believe that the level of rigor may need to be tailored pragmatically to the specific review objectives, which may not always justify the resource requirements of an intensive review process.

The example systematic methods overview on sampling in qualitative research

The principles and strategies we propose in this paper are derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research [ 18 ]. The main objective of that methods overview was to bring clarity and deeper understanding of the prominent concepts related to sampling in qualitative research (purposeful sampling strategies, saturation, etc.). Specifically, we interpreted the available guidance, commenting on areas lacking clarity, consistency, or comprehensiveness (without proposing any recommendations on how to do sampling). This was achieved by a comparative and critical analysis of publications representing the most influential (i.e., highly cited) guidance across several methodological traditions in qualitative research.

The specific methods and procedures for the overview on sampling [ 18 ] from which our proposals are derived were developed both after soliciting initial input from local experts in qualitative research and an expert health librarian (KAM) and through ongoing careful deliberation throughout the review process. To summarize, in that review, we employed a transparent and rigorous approach to search the methods literature, selected publications for inclusion according to a purposeful and iterative process, abstracted textual data using structured abstraction forms, and analyzed (synthesized) the data using a systematic multi-step approach featuring abstraction of text, summary of information in matrices, and analytic comparisons.

For this article, we reflected on both the problems and challenges encountered at different stages of the review and our means for selecting justifiable procedures to deal with them. Several principles were then derived by considering the generic nature of these problems, while the generalizable aspects of the procedures used to address them formed the basis of optional strategies. Further details of the specific methods and procedures used in the overview on qualitative sampling are provided below to illustrate both the types of objectives and challenges that reviewers will likely need to consider and our approach to implementing each of the principles and strategies.

Organization of the guidance into principles and strategies

For the purposes of this article, principles are general statements outlining what we propose are important aims or considerations within a particular review process, given the unique objectives or challenges to be overcome with this type of review. These statements follow the general format, “considering the objective or challenge of X, we propose Y to be an important aim or consideration.” Strategies are optional and flexible approaches for implementing the previous principle outlined. Thus, generic challenges give rise to principles, which in turn give rise to strategies.

We organize the principles and strategies below into three sections corresponding to processes characteristic of most systematic literature synthesis approaches: literature identification and selection ; data abstraction from the publications selected for inclusion; and analysis , including critical appraisal and synthesis of the abstracted data. Within each section, we also describe the specific methodological decisions and procedures used in the overview on sampling in qualitative research [ 18 ] to illustrate how the principles and strategies for each review process were applied and implemented in a specific case. We expect this guidance and accompanying illustrations will be useful for anyone considering engaging in a methods overview, particularly those who may be familiar with conventional systematic review methods but may not yet appreciate some of the challenges specific to reviewing the methods literature.

Results and discussion

Literature identification and selection.

The identification and selection process includes search and retrieval of publications and the development and application of inclusion and exclusion criteria to select the publications that will be abstracted and analyzed in the final review. Literature identification and selection for overviews of the methods literature is challenging and potentially more resource-intensive than for most reviews of empirical research. This is true for several reasons that we describe below, alongside discussion of the potential solutions. Additionally, we suggest in this section how the selection procedures can be chosen to match the specific analytic approach used in methods overviews.

Delimiting a manageable set of publications

One aspect of methods overviews that can make identification and selection challenging is the fact that the universe of literature containing potentially relevant information regarding most methods-related topics is expansive and often unmanageably so. Reviewers are faced with two large categories of literature: the methods literature , where the possible publication types include journal articles, books, and book chapters; and the methods-relevant sections of empirical study reports , where the possible publication types include journal articles, monographs, books, theses, and conference proceedings. In our systematic overview of sampling in qualitative research, exhaustively searching (including retrieval and first-pass screening) all publication types across both categories of literature for information on a single methods-related topic was too burdensome to be feasible. The following proposed principle follows from the need to delimit a manageable set of literature for the review.

Principle #1:

Considering the broad universe of potentially relevant literature, we propose that an important objective early in the identification and selection stage is to delimit a manageable set of methods-relevant publications in accordance with the objectives of the methods overview.

Strategy #1:

To limit the set of methods-relevant publications that must be managed in the selection process, reviewers have the option to initially review only the methods literature, and exclude the methods-relevant sections of empirical study reports, provided this aligns with the review’s particular objectives.

We propose that reviewers are justified in choosing to select only the methods literature when the objective is to map out the range of recognized concepts relevant to a methods topic, to summarize the most authoritative or influential definitions or meanings for methods-related concepts, or to demonstrate a problematic lack of clarity regarding a widely established methods-related concept and potentially make recommendations for a preferred approach to the methods topic in question. For example, in the case of the methods overview on sampling [ 18 ], the primary aim was to define areas lacking in clarity for multiple widely established sampling-related topics. In the review on intention-to-treat in the context of missing outcome data [ 17 ], the authors identified a lack of clarity based on multiple inconsistent definitions in the literature and went on to recommend separating the issue of how to handle missing outcome data from the issue of whether an intention-to-treat analysis can be claimed.

In contrast to strategy #1, it may be appropriate to select the methods-relevant sections of empirical study reports when the objective is to illustrate how a methods concept is operationalized in research practice or reported by authors. For example, one could review all the publications in 2 years’ worth of issues of five high-impact field-related journals to answer questions about how researchers describe implementing a particular method or approach, or to quantify how consistently they define or report using it. Such reviews are often used to highlight gaps in the reporting practices regarding specific methods, which may be used to justify items to address in reporting guidelines (for example, [ 14 – 16 ]).

It is worth recognizing that other authors have advocated broader positions regarding the scope of literature to be considered in a review, expanding on our perspective. Suri [ 10 ] (who, like us, emphasizes how different sampling strategies are suitable for different literature synthesis objectives) has, for example, described a two-stage literature sampling procedure (pp. 96–97). First, reviewers use an initial approach to conduct a broad overview of the field—for reviews of methods topics, this would entail an initial review of the research methods literature. This is followed by a second more focused stage in which practical examples are purposefully selected—for methods reviews, this would involve sampling the empirical literature to illustrate key themes and variations. While this approach is seductive in its capacity to generate more in depth and interpretive analytic findings, some reviewers may consider it too resource-intensive to include the second step no matter how selective the purposeful sampling. In the overview on sampling where we stopped after the first stage [ 18 ], we discussed our selective focus on the methods literature as a limitation that left opportunities for further analysis of the literature. We explicitly recommended, for example, that theoretical sampling was a topic for which a future review of the methods sections of empirical reports was justified to answer specific questions identified in the primary review.

Ultimately, reviewers must make pragmatic decisions that balance resource considerations, combined with informed predictions about the depth and complexity of literature available on their topic, with the stated objectives of their review. The remaining principles and strategies apply primarily to overviews that include the methods literature, although some aspects may be relevant to reviews that include empirical study reports.

Searching beyond standard bibliographic databases

An important reality affecting identification and selection in overviews of the methods literature is the increased likelihood for relevant publications to be located in sources other than journal articles (which is usually not the case for overviews of empirical research, where journal articles generally represent the primary publication type). In the overview on sampling [ 18 ], out of 41 full-text publications retrieved and reviewed, only 4 were journal articles, while 37 were books or book chapters. Since many books and book chapters did not exist electronically, their full text had to be physically retrieved in hardcopy, while 11 publications were retrievable only through interlibrary loan or purchase request. The tasks associated with such retrieval are substantially more time-consuming than electronic retrieval. Since a substantial proportion of methods-related guidance may be located in publication types that are less comprehensively indexed in standard bibliographic databases, identification and retrieval thus become complicated processes.

Principle #2:

Considering that important sources of methods guidance can be located in non-journal publication types (e.g., books, book chapters) that tend to be poorly indexed in standard bibliographic databases, it is important to consider alternative search methods for identifying relevant publications to be further screened for inclusion.

Strategy #2:

To identify books, book chapters, and other non-journal publication types not thoroughly indexed in standard bibliographic databases, reviewers may choose to consult one or more of the following less standard sources: Google Scholar, publisher web sites, or expert opinion.

In the case of the overview on sampling in qualitative research [ 18 ], Google Scholar had two advantages over other standard bibliographic databases: it indexes and returns records of books and book chapters likely to contain guidance on qualitative research methods topics; and it has been validated as providing higher citation counts than ISI Web of Science (a producer of numerous bibliographic databases accessible through institutional subscription) for several non-biomedical disciplines including the social sciences where qualitative research methods are prominently used [ 19 – 21 ]. While we identified numerous useful publications by consulting experts, the author publication lists generated through Google Scholar searches were uniquely useful to identify more recent editions of methods books identified by experts.

Searching without relevant metadata

Determining what publications to select for inclusion in the overview on sampling [ 18 ] could only rarely be accomplished by reviewing the publication’s metadata. This was because for the many books and other non-journal type publications we identified as possibly relevant, the potential content of interest would be located in only a subsection of the publication. In this common scenario for reviews of the methods literature (as opposed to methods overviews that include empirical study reports), reviewers will often be unable to employ standard title, abstract, and keyword database searching or screening as a means for selecting publications.

Principle #3:

Considering that the presence of information about the topic of interest may not be indicated in the metadata for books and similar publication types, it is important to consider other means of identifying potentially useful publications for further screening.

Strategy #3:

One approach to identifying potentially useful books and similar publication types is to consider what classes of such publications (e.g., all methods manuals for a certain research approach) are likely to contain relevant content, then identify, retrieve, and review the full text of corresponding publications to determine whether they contain information on the topic of interest.

In the example of the overview on sampling in qualitative research [ 18 ], the topic of interest (sampling) was one of numerous topics covered in the general qualitative research methods manuals. Consequently, examples from this class of publications first had to be identified for retrieval according to non-keyword-dependent criteria. Thus, all methods manuals within the three research traditions reviewed (grounded theory, phenomenology, and case study) that might contain discussion of sampling were sought through Google Scholar and expert opinion, their full text obtained, and hand-searched for relevant content to determine eligibility. We used tables of contents and index sections of books to aid this hand searching.

Purposefully selecting literature on conceptual grounds

A final consideration in methods overviews relates to the type of analysis used to generate the review findings. Unlike quantitative systematic reviews where reviewers aim for accurate or unbiased quantitative estimates—something that requires identifying and selecting the literature exhaustively to obtain all relevant data available (i.e., a complete sample)—in methods overviews, reviewers must describe and interpret the relevant literature in qualitative terms to achieve review objectives. In other words, the aim in methods overviews is to seek coverage of the qualitative concepts relevant to the methods topic at hand. For example, in the overview of sampling in qualitative research [ 18 ], achieving review objectives entailed providing conceptual coverage of eight sampling-related topics that emerged as key domains. The following principle recognizes that literature sampling should therefore support generating qualitative conceptual data as the input to analysis.

Principle #4:

Since the analytic findings of a systematic methods overview are generated through qualitative description and interpretation of the literature on a specified topic, selection of the literature should be guided by a purposeful strategy designed to achieve adequate conceptual coverage (i.e., representing an appropriate degree of variation in relevant ideas) of the topic according to objectives of the review.

Strategy #4:

One strategy for choosing the purposeful approach to use in selecting the literature according to the review objectives is to consider whether those objectives imply exploring concepts either at a broad overview level, in which case combining maximum variation selection with a strategy that limits yield (e.g., critical case, politically important, or sampling for influence—described below) may be appropriate; or in depth, in which case purposeful approaches aimed at revealing innovative cases will likely be necessary.

In the methods overview on sampling, the implied scope was broad since we set out to review publications on sampling across three divergent qualitative research traditions—grounded theory, phenomenology, and case study—to facilitate making informative conceptual comparisons. Such an approach would be analogous to maximum variation sampling.

At the same time, the purpose of that review was to critically interrogate the clarity, consistency, and comprehensiveness of literature from these traditions that was “most likely to have widely influenced students’ and researchers’ ideas about sampling” (p. 1774) [ 18 ]. In other words, we explicitly set out to review and critique the most established and influential (and therefore dominant) literature, since this represents a common basis of knowledge among students and researchers seeking understanding or practical guidance on sampling in qualitative research. To achieve this objective, we purposefully sampled publications according to the criterion of influence , which we operationalized as how often an author or publication has been referenced in print or informal discourse. This second sampling approach also limited the literature we needed to consider within our broad scope review to a manageable amount.

To operationalize this strategy of sampling for influence , we sought to identify both the most influential authors within a qualitative research tradition (all of whose citations were subsequently screened) and the most influential publications on the topic of interest by non-influential authors. This involved a flexible approach that combined multiple indicators of influence to avoid the dilemma that any single indicator might provide inadequate coverage. These indicators included bibliometric data (h-index for author influence [ 22 ]; number of cites for publication influence), expert opinion, and cross-references in the literature (i.e., snowball sampling). As a final selection criterion, a publication was included only if it made an original contribution in terms of novel guidance regarding sampling or a related concept; thus, purely secondary sources were excluded. Publish or Perish software (Anne-Wil Harzing; available at http://www.harzing.com/resources/publish-or-perish ) was used to generate bibliometric data via the Google Scholar database. Figure  1 illustrates how identification and selection in the methods overview on sampling was a multi-faceted and iterative process. The authors selected as influential, and the publications selected for inclusion or exclusion are listed in Additional file 1 (Matrices 1, 2a, 2b).

Literature identification and selection process used in the methods overview on sampling [ 18 ]

In summary, the strategies of seeking maximum variation and sampling for influence were employed in the sampling overview to meet the specific review objectives described. Reviewers will need to consider the full range of purposeful literature sampling approaches at their disposal in deciding what best matches the specific aims of their own reviews. Suri [ 10 ] has recently retooled Patton’s well-known typology of purposeful sampling strategies (originally intended for primary research) for application to literature synthesis, providing a useful resource in this respect.

Data abstraction

The purpose of data abstraction in rigorous literature reviews is to locate and record all data relevant to the topic of interest from the full text of included publications, making them available for subsequent analysis. Conventionally, a data abstraction form—consisting of numerous distinct conceptually defined fields to which corresponding information from the source publication is recorded—is developed and employed. There are several challenges, however, to the processes of developing the abstraction form and abstracting the data itself when conducting methods overviews, which we address here. Some of these problems and their solutions may be familiar to those who have conducted qualitative literature syntheses, which are similarly conceptual.

Iteratively defining conceptual information to abstract

In the overview on sampling [ 18 ], while we surveyed multiple sources beforehand to develop a list of concepts relevant for abstraction (e.g., purposeful sampling strategies, saturation, sample size), there was no way for us to anticipate some concepts prior to encountering them in the review process. Indeed, in many cases, reviewers are unable to determine the complete set of methods-related concepts that will be the focus of the final review a priori without having systematically reviewed the publications to be included. Thus, defining what information to abstract beforehand may not be feasible.

Principle #5:

Considering the potential impracticality of defining a complete set of relevant methods-related concepts from a body of literature one has not yet systematically read, selecting and defining fields for data abstraction must often be undertaken iteratively. Thus, concepts to be abstracted can be expected to grow and change as data abstraction proceeds.

Strategy #5:

Reviewers can develop an initial form or set of concepts for abstraction purposes according to standard methods (e.g., incorporating expert feedback, pilot testing) and remain attentive to the need to iteratively revise it as concepts are added or modified during the review. Reviewers should document revisions and return to re-abstract data from previously abstracted publications as the new data requirements are determined.

In the sampling overview [ 18 ], we developed and maintained the abstraction form in Microsoft Word. We derived the initial set of abstraction fields from our own knowledge of relevant sampling-related concepts, consultation with local experts, and reviewing a pilot sample of publications. Since the publications in this review included a large proportion of books, the abstraction process often began by flagging the broad sections within a publication containing topic-relevant information for detailed review to identify text to abstract. When reviewing flagged text, the reviewer occasionally encountered an unanticipated concept significant enough to warrant being added as a new field to the abstraction form. For example, a field was added to capture how authors described the timing of sampling decisions, whether before (a priori) or after (ongoing) starting data collection, or whether this was unclear. In these cases, we systematically documented the modification to the form and returned to previously abstracted publications to abstract any information that might be relevant to the new field.

The logic of this strategy is analogous to the logic used in a form of research synthesis called best fit framework synthesis (BFFS) [ 23 – 25 ]. In that method, reviewers initially code evidence using an a priori framework they have selected. When evidence cannot be accommodated by the selected framework, reviewers then develop new themes or concepts from which they construct a new expanded framework. Both the strategy proposed and the BFFS approach to research synthesis are notable for their rigorous and transparent means to adapt a final set of concepts to the content under review.

Accounting for inconsistent terminology

An important complication affecting the abstraction process in methods overviews is that the language used by authors to describe methods-related concepts can easily vary across publications. For example, authors from different qualitative research traditions often use different terms for similar methods-related concepts. Furthermore, as we found in the sampling overview [ 18 ], there may be cases where no identifiable term, phrase, or label for a methods-related concept is used at all, and a description of it is given instead. This can make searching the text for relevant concepts based on keywords unreliable.

Principle #6:

Since accepted terms may not be used consistently to refer to methods concepts, it is necessary to rely on the definitions for concepts, rather than keywords, to identify relevant information in the publication to abstract.

Strategy #6:

An effective means to systematically identify relevant information is to develop and iteratively adjust written definitions for key concepts (corresponding to abstraction fields) that are consistent with and as inclusive of as much of the literature reviewed as possible. Reviewers then seek information that matches these definitions (rather than keywords) when scanning a publication for relevant data to abstract.

In the abstraction process for the sampling overview [ 18 ], we noted the several concepts of interest to the review for which abstraction by keyword was particularly problematic due to inconsistent terminology across publications: sampling , purposeful sampling , sampling strategy , and saturation (for examples, see Additional file 1 , Matrices 3a, 3b, 4). We iteratively developed definitions for these concepts by abstracting text from publications that either provided an explicit definition or from which an implicit definition could be derived, which was recorded in fields dedicated to the concept’s definition. Using a method of constant comparison, we used text from definition fields to inform and modify a centrally maintained definition of the corresponding concept to optimize its fit and inclusiveness with the literature reviewed. Table  1 shows, as an example, the final definition constructed in this way for one of the central concepts of the review, qualitative sampling .

We applied iteratively developed definitions when making decisions about what specific text to abstract for an existing field, which allowed us to abstract concept-relevant data even if no recognized keyword was used. For example, this was the case for the sampling-related concept, saturation , where the relevant text available for abstraction in one publication [ 26 ]—“to continue to collect data until nothing new was being observed or recorded, no matter how long that takes”—was not accompanied by any term or label whatsoever.

This comparative analytic strategy (and our approach to analysis more broadly as described in strategy #7, below) is analogous to the process of reciprocal translation —a technique first introduced for meta-ethnography by Noblit and Hare [ 27 ] that has since been recognized as a common element in a variety of qualitative metasynthesis approaches [ 28 ]. Reciprocal translation, taken broadly, involves making sense of a study’s findings in terms of the findings of the other studies included in the review. In practice, it has been operationalized in different ways. Melendez-Torres and colleagues developed a typology from their review of the metasynthesis literature, describing four overlapping categories of specific operations undertaken in reciprocal translation: visual representation, key paper integration, data reduction and thematic extraction, and line-by-line coding [ 28 ]. The approaches suggested in both strategies #6 and #7, with their emphasis on constant comparison, appear to fall within the line-by-line coding category.

Generating credible and verifiable analytic interpretations

The analysis in a systematic methods overview must support its more general objective, which we suggested above is often to offer clarity and enhance collective understanding regarding a chosen methods topic. In our experience, this involves describing and interpreting the relevant literature in qualitative terms. Furthermore, any interpretative analysis required may entail reaching different levels of abstraction, depending on the more specific objectives of the review. For example, in the overview on sampling [ 18 ], we aimed to produce a comparative analysis of how multiple sampling-related topics were treated differently within and among different qualitative research traditions. To promote credibility of the review, however, not only should one seek a qualitative analytic approach that facilitates reaching varying levels of abstraction but that approach must also ensure that abstract interpretations are supported and justified by the source data and not solely the product of the analyst’s speculative thinking.

Principle #7:

Considering the qualitative nature of the analysis required in systematic methods overviews, it is important to select an analytic method whose interpretations can be verified as being consistent with the literature selected, regardless of the level of abstraction reached.

Strategy #7:

We suggest employing the constant comparative method of analysis [ 29 ] because it supports developing and verifying analytic links to the source data throughout progressively interpretive or abstract levels. In applying this approach, we advise a rigorous approach, documenting how supportive quotes or references to the original texts are carried forward in the successive steps of analysis to allow for easy verification.

The analytic approach used in the methods overview on sampling [ 18 ] comprised four explicit steps, progressing in level of abstraction—data abstraction, matrices, narrative summaries, and final analytic conclusions (Fig.  2 ). While we have positioned data abstraction as the second stage of the generic review process (prior to Analysis), above, we also considered it as an initial step of analysis in the sampling overview for several reasons. First, it involved a process of constant comparisons and iterative decision-making about the fields to add or define during development and modification of the abstraction form, through which we established the range of concepts to be addressed in the review. At the same time, abstraction involved continuous analytic decisions about what textual quotes (ranging in size from short phrases to numerous paragraphs) to record in the fields thus created. This constant comparative process was analogous to open coding in which textual data from publications was compared to conceptual fields (equivalent to codes) or to other instances of data previously abstracted when constructing definitions to optimize their fit with the overall literature as described in strategy #6. Finally, in the data abstraction step, we also recorded our first interpretive thoughts in dedicated fields, providing initial material for the more abstract analytic steps.

Summary of progressive steps of analysis used in the methods overview on sampling [ 18 ]

In the second step of the analysis, we constructed topic-specific matrices , or tables, by copying relevant quotes from abstraction forms into the appropriate cells of matrices (for the complete set of analytic matrices developed in the sampling review, see Additional file 1 (matrices 3 to 10)). Each matrix ranged from one to five pages; row headings, nested three-deep, identified the methodological tradition, author, and publication, respectively; and column headings identified the concepts, which corresponded to abstraction fields. Matrices thus allowed us to make further comparisons across methodological traditions, and between authors within a tradition. In the third step of analysis, we recorded our comparative observations as narrative summaries , in which we used illustrative quotes more sparingly. In the final step, we developed analytic conclusions based on the narrative summaries about the sampling-related concepts within each methodological tradition for which clarity, consistency, or comprehensiveness of the available guidance appeared to be lacking. Higher levels of analysis thus built logically from the lower levels, enabling us to easily verify analytic conclusions by tracing the support for claims by comparing the original text of publications reviewed.

Integrative versus interpretive methods overviews

The analytic product of systematic methods overviews is comparable to qualitative evidence syntheses, since both involve describing and interpreting the relevant literature in qualitative terms. Most qualitative synthesis approaches strive to produce new conceptual understandings that vary in level of interpretation. Dixon-Woods and colleagues [ 30 ] elaborate on a useful distinction, originating from Noblit and Hare [ 27 ], between integrative and interpretive reviews. Integrative reviews focus on summarizing available primary data and involve using largely secure and well defined concepts to do so; definitions are used from an early stage to specify categories for abstraction (or coding) of data, which in turn supports their aggregation; they do not seek as their primary focus to develop or specify new concepts, although they may achieve some theoretical or interpretive functions. For interpretive reviews, meanwhile, the main focus is to develop new concepts and theories that integrate them, with the implication that the concepts developed become fully defined towards the end of the analysis. These two forms are not completely distinct, and “every integrative synthesis will include elements of interpretation, and every interpretive synthesis will include elements of aggregation of data” [ 30 ].

The example methods overview on sampling [ 18 ] could be classified as predominantly integrative because its primary goal was to aggregate influential authors’ ideas on sampling-related concepts; there were also, however, elements of interpretive synthesis since it aimed to develop new ideas about where clarity in guidance on certain sampling-related topics is lacking, and definitions for some concepts were flexible and not fixed until late in the review. We suggest that most systematic methods overviews will be classifiable as predominantly integrative (aggregative). Nevertheless, more highly interpretive methods overviews are also quite possible—for example, when the review objective is to provide a highly critical analysis for the purpose of generating new methodological guidance. In such cases, reviewers may need to sample more deeply (see strategy #4), specifically by selecting empirical research reports (i.e., to go beyond dominant or influential ideas in the methods literature) that are likely to feature innovations or instructive lessons in employing a given method.

In this paper, we have outlined tentative guidance in the form of seven principles and strategies on how to conduct systematic methods overviews, a review type in which methods-relevant literature is systematically analyzed with the aim of offering clarity and enhancing collective understanding regarding a specific methods topic. Our proposals include strategies for delimiting the set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology, and generating credible and verifiable analytic interpretations. We hope the suggestions proposed will be useful to others undertaking reviews on methods topics in future.

As far as we are aware, this is the first published source of concrete guidance for conducting this type of review. It is important to note that our primary objective was to initiate methodological discussion by stimulating reflection on what rigorous methods for this type of review should look like, leaving the development of more complete guidance to future work. While derived from the experience of reviewing a single qualitative methods topic, we believe the principles and strategies provided are generalizable to overviews of both qualitative and quantitative methods topics alike. However, it is expected that additional challenges and insights for conducting such reviews have yet to be defined. Thus, we propose that next steps for developing more definitive guidance should involve an attempt to collect and integrate other reviewers’ perspectives and experiences in conducting systematic methods overviews on a broad range of qualitative and quantitative methods topics. Formalized guidance and standards would improve the quality of future methods overviews, something we believe has important implications for advancing qualitative and quantitative methodology. When undertaken to a high standard, rigorous critical evaluations of the available methods guidance have significant potential to make implicit controversies explicit, and improve the clarity and precision of our understandings of problematic qualitative or quantitative methods issues.

A review process central to most types of rigorous reviews of empirical studies, which we did not explicitly address in a separate review step above, is quality appraisal . The reason we have not treated this as a separate step stems from the different objectives of the primary publications included in overviews of the methods literature (i.e., providing methodological guidance) compared to the primary publications included in the other established review types (i.e., reporting findings from single empirical studies). This is not to say that appraising quality of the methods literature is not an important concern for systematic methods overviews. Rather, appraisal is much more integral to (and difficult to separate from) the analysis step, in which we advocate appraising clarity, consistency, and comprehensiveness—the quality appraisal criteria that we suggest are appropriate for the methods literature. As a second important difference regarding appraisal, we currently advocate appraising the aforementioned aspects at the level of the literature in aggregate rather than at the level of individual publications. One reason for this is that methods guidance from individual publications generally builds on previous literature, and thus we feel that ahistorical judgments about comprehensiveness of single publications lack relevance and utility. Additionally, while different methods authors may express themselves less clearly than others, their guidance can nonetheless be highly influential and useful, and should therefore not be downgraded or ignored based on considerations of clarity—which raises questions about the alternative uses that quality appraisals of individual publications might have. Finally, legitimate variability in the perspectives that methods authors wish to emphasize, and the levels of generality at which they write about methods, makes critiquing individual publications based on the criterion of clarity a complex and potentially problematic endeavor that is beyond the scope of this paper to address. By appraising the current state of the literature at a holistic level, reviewers stand to identify important gaps in understanding that represent valuable opportunities for further methodological development.

To summarize, the principles and strategies provided here may be useful to those seeking to undertake their own systematic methods overview. Additional work is needed, however, to establish guidance that is comprehensive by comparing the experiences from conducting a variety of methods overviews on a range of methods topics. Efforts that further advance standards for systematic methods overviews have the potential to promote high-quality critical evaluations that produce conceptually clear and unified understandings of problematic methods topics, thereby accelerating the advance of research methodology.

Hutton JL, Ashcroft R. What does “systematic” mean for reviews of methods? In: Black N, Brazier J, Fitzpatrick R, Reeves B, editors. Health services research methods: a guide to best practice. London: BMJ Publishing Group; 1998. p. 249–54.

Google Scholar  

Cochrane handbook for systematic reviews of interventions. In. Edited by Higgins JPT, Green S, Version 5.1.0 edn: The Cochrane Collaboration; 2011.

Centre for Reviews and Dissemination: Systematic reviews: CRD’s guidance for undertaking reviews in health care . York: Centre for Reviews and Dissemination; 2009.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700–0.

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9(1):59.

Article   PubMed   PubMed Central   Google Scholar  

Kastner M, Tricco AC, Soobiah C, Lillie E, Perrier L, Horsley T, Welch V, Cogo E, Antony J, Straus SE. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Med Res Methodol. 2012;12(1):1–1.

Article   Google Scholar  

Booth A, Noyes J, Flemming K, Gerhardus A. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. In: Integrate-HTA. 2016.

Booth A, Sutton A, Papaioannou D. Systematic approaches to successful literature review. 2nd ed. London: Sage; 2016.

Hannes K, Lockwood C. Synthesizing qualitative research: choosing the right approach. Chichester: Wiley-Blackwell; 2012.

Suri H. Towards methodologically inclusive research syntheses: expanding possibilities. New York: Routledge; 2014.

Campbell M, Egan M, Lorenc T, Bond L, Popham F, Fenton C, Benzeval M. Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health. Syst Rev. 2014;3(1):1–11.

Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med. 2008;6(4):331–9.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reportingqualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Article   PubMed   Google Scholar  

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217.

Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78.

Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005;365(9465):1159–62.

Alshurafa M, Briel M, Akl EA, Haines T, Moayyedi P, Gentles SJ, Rios L, Tran C, Bhatnagar N, Lamontagne F, et al. Inconsistent definitions for intention-to-treat in relation to missing outcome data: systematic review of the methods literature. PLoS One. 2012;7(11):e49163.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gentles SJ, Charles C, Ploeg J, McKibbon KA. Sampling in qualitative research: insights from an overview of the methods literature. Qual Rep. 2015;20(11):1772–89.

Harzing A-W, Alakangas S. Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics. 2016;106(2):787–804.

Harzing A-WK, van der Wal R. Google Scholar as a new source for citation analysis. Ethics Sci Environ Polit. 2008;8(1):61–73.

Kousha K, Thelwall M. Google Scholar citations and Google Web/URL citations: a multi‐discipline exploratory analysis. J Assoc Inf Sci Technol. 2007;58(7):1055–65.

Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569–72.

Booth A, Carroll C. How to build up the actionable knowledge base: the role of ‘best fit’ framework synthesis for studies of improvement in healthcare. BMJ Quality Safety. 2015;24(11):700–8.

Carroll C, Booth A, Leaviss J, Rick J. “Best fit” framework synthesis: refining the method. BMC Med Res Methodol. 2013;13(1):37.

Carroll C, Booth A, Cooper K. A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11(1):29.

Cohen MZ, Kahn DL, Steeves DL. Hermeneutic phenomenological research: a practical guide for nurse researchers. Thousand Oaks: Sage; 2000.

Noblit GW, Hare RD. Meta-ethnography: synthesizing qualitative studies. Newbury Park: Sage; 1988.

Book   Google Scholar  

Melendez-Torres GJ, Grant S, Bonell C. A systematic review and critical appraisal of qualitative metasynthetic practice in public health to develop a taxonomy of operations of reciprocal translation. Res Synthesis Methods. 2015;6(4):357–71.

Article   CAS   Google Scholar  

Glaser BG, Strauss A. The discovery of grounded theory. Chicago: Aldine; 1967.

Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton A. Integrative approaches to qualitative and quantitative evidence. In: UK National Health Service. 2004. p. 1–44.

Download references

Acknowledgements

Not applicable.

There was no funding for this work.

Availability of data and materials

The systematic methods overview used as a worked example in this article (Gentles SJ, Charles C, Ploeg J, McKibbon KA: Sampling in qualitative research: insights from an overview of the methods literature. The Qual Rep 2015, 20(11):1772-1789) is available from http://nsuworks.nova.edu/tqr/vol20/iss11/5 .

Authors’ contributions

SJG wrote the first draft of this article, with CC contributing to drafting. All authors contributed to revising the manuscript. All authors except CC (deceased) approved the final draft. SJG, CC, KAB, and JP were involved in developing methods for the systematic methods overview on sampling.

Authors’ information

Competing interests.

The authors declare that they have no competing interests.

Consent for publication

Ethics approval and consent to participate, author information, authors and affiliations.

Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada

Stephen J. Gentles, Cathy Charles & K. Ann McKibbon

Faculty of Social Work, University of Calgary, Alberta, Canada

David B. Nicholas

School of Nursing, McMaster University, Hamilton, Ontario, Canada

Jenny Ploeg

CanChild Centre for Childhood Disability Research, McMaster University, 1400 Main Street West, IAHS 408, Hamilton, ON, L8S 1C7, Canada

Stephen J. Gentles

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Stephen J. Gentles .

Additional information

Cathy Charles is deceased

Additional file

Additional file 1:.

Submitted: Analysis_matrices. (DOC 330 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Gentles, S.J., Charles, C., Nicholas, D.B. et al. Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research. Syst Rev 5 , 172 (2016). https://doi.org/10.1186/s13643-016-0343-0

Download citation

Received : 06 June 2016

Accepted : 14 September 2016

Published : 11 October 2016

DOI : https://doi.org/10.1186/s13643-016-0343-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Systematic review
  • Literature selection
  • Research methods
  • Research methodology
  • Overview of methods
  • Systematic methods overview
  • Review methods

Systematic Reviews

ISSN: 2046-4053

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

literary studies research methods

Research Methods

  • Getting Started
  • Literature Review Research
  • Research Design
  • Research Design By Discipline
  • SAGE Research Methods
  • Teaching with SAGE Research Methods

Literature Review

  • What is a Literature Review?
  • What is NOT a Literature Review?
  • Purposes of a Literature Review
  • Types of Literature Reviews
  • Literature Reviews vs. Systematic Reviews
  • Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

  • Summarizes and analyzes previous research relevant to a topic
  • Includes scholarly books and articles published in academic journals
  • Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

  • Help gather ideas or information
  • Keep up to date in current trends and findings
  • Help develop new questions

A literature review is important because it:

  • Explains the background of research on a topic.
  • Demonstrates why a topic is significant to a subject area.
  • Helps focus your own research questions or problems
  • Discovers relationships between research studies/ideas.
  • Suggests unexplored ideas or populations
  • Identifies major themes, concepts, and researchers on a topic.
  • Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
  • Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
  • Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

  • provides thorough knowledge of previous studies; introduces seminal works.
  • helps focus one’s own research topic.
  • identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
  • suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
  • identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
  • helps the researcher avoid repetition of earlier research.
  • suggests unexplored populations.
  • determines whether past studies agree or disagree; identifies controversy in the literature.
  • tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

literary studies research methods

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

literary studies research methods

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

  • clearly stated objectives with pre-defined eligibility criteria for studies
  • explicit, reproducible methodology
  • a systematic search that attempts to identify all studies
  • assessment of the validity of the findings of the included studies (e.g. risk of bias)
  • systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

  • << Previous: Getting Started
  • Next: Research Design >>
  • Last Updated: Aug 21, 2023 4:07 PM
  • URL: https://guides.lib.udel.edu/researchmethods

U.S. flag

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Cover of Handbook of eHealth Evaluation: An Evidence-based Approach

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

  • formulating the research question(s) and objective(s),
  • searching the extant literature,
  • screening for inclusion,
  • assessing the quality of primary studies,
  • extracting data, and
  • analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

  • Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
  • Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
  • Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
  • Analyzing data using quantitative or qualitative methods.
  • Presenting results in summary of findings tables.
  • Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Table 9.1. Typology of Literature Reviews (adapted from Paré et al., 2015).

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

  • Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
  • Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
  • Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
  • Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
  • A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
  • Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
  • Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
  • Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
  • Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
  • Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
  • Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
  • Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
  • Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
  • Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
  • Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
  • Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
  • Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
  • Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
  • Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
  • Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
  • Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
  • Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
  • Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
  • Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
  • Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
  • Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
  • King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
  • Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
  • Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
  • Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
  • Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
  • Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
  • Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
  • Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
  • Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
  • Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
  • Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
  • Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
  • Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
  • Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
  • Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
  • Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
  • Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
  • Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
  • Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
  • Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
  • Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
  • Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
  • Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
  • Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
  • Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
  • Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
  • Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
  • Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
  • Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
  • Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
  • Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
  • Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
  • Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
  • Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
  • Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

  • Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
  • PDF version of this title (4.5M)
  • Disable Glossary Links

In this Page

  • Introduction
  • Overview of the Literature Review Process and Steps
  • Types of Review Articles and Brief Illustrations
  • Concluding Remarks

Related information

  • PMC PubMed Central citations
  • PubMed Links to PubMed

Recent Activity

  • Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

statistics

Log in using your username and password

  • Search More Search for this keyword Advanced search
  • Latest content
  • For authors
  • Browse by collection
  • BMJ Journals More You are viewing from: Google Indexer

You are here

  • Volume 14, Issue 2
  • Nursing students’ Health Literacy skills: a scoping review protocol for driving research
  • Article Text
  • Article info
  • Citation Tools
  • Rapid Responses
  • Article metrics

Download PDF

  • http://orcid.org/0000-0002-5299-3302 Giampiera Bulfone 1 ,
  • Valentina Bressan 2 ,
  • Irene Zerilli 3 ,
  • Rocco Mazzotta 3 ,
  • Giuliana Favara 1 ,
  • Roberta Magnano San Lio 1 ,
  • http://orcid.org/0000-0002-0905-5003 Martina Barchitta 1 ,
  • Antonella Agodi 1
  • 1 Department of Medical, Surgical Science and Advanced Technology "G. F. Ingrassia" , University of Catania , Catania , Italy
  • 2 Department of Specialistic Medicine , Azienda Sanitaria Universitaria Friuli Centrale , Udine , Italy
  • 3 Department of Biomedicine and Prevention , University of Rome Tor Vergata , Roma , Italy
  • Correspondence to Dr Giampiera Bulfone; giampiera.bulfone{at}unict.it

Introduction The healthcare systems in Europe are changing rapidly due to the increased complexity of healthcare needs, specifically for the ageing population with chronic diseases. Nurses play a key role in providing care for patients with chronic diseases, encouraging patients to take care of their own health improving their Health Literacy (HL) too. Previous works have highlighted the paucity of HL content in nursing curricula, and the need to prioritise the development of HL skills in academic teaching and assessment methods. The aim of this study is to analyse HL skills nursing literature to further develop scientific knowledge in this area of research.

Methods and analysis This scoping review will be conducted following Arksey and O’Malley’s framework. This study is based on the Joanna Briggs Institute manual. A systematic search will be performed by four researchers using the electronic databases of MEDLINE (via PubMed), the Education Resources Information Centre, the Cumulative Index to Nursing and Allied Health Literature, Scopus, Web of Science and Google Scholar. We will include any paper that focuses on HL skills and undergraduate nursing students. We will select every primary study (quantitative, qualitative and mixed method design) published in peer-reviewed journals up until February 2023, in both Italian and English language, without any time limit.

Ethics and dissemination This scoping review is part of a large project of the University of Catania which aims at developing higher educational standards for nursing student. This project will not involve patients/public and does not require ethical committee approval. This scoping review will be submitted to international peer-reviewed journals.

Registration details The protocol was registered with the Open Science Framework on 20 April 2023 ( https://osf.io/cn8d7 ).

  • Nursing Care
  • Health Education
  • PUBLIC HEALTH

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2023-075682

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

To our knowledge, this will be the first scoping review on Health Literacy skills among nursing students.

The review will take a rigorous approach, adhering to the Joanna Briggs Institute manual.

Large searching strategies will be developed drawing on major databases and grey literature without any time limits.

A limit of this study is that articles in Italian and English only will be included and, being a scoping review, no quality of the chosen studies will be assessed.

Introduction

The Europe healthcare landscape is undergoing rapid transformations, driven by the growing complexities of healthcare needs, particularly among the ageing population affected by chronic diseases. It is essential to empower patients in order to improve their quality of life, well-being, health outcomes and self-management. 1 2 In order for patient to better manage self-care and exercise a more active involvement in decision-making regarding own care, they will have to be educated by healthcare professionals. 3 Nurses play a critical role in caring for patients with chronic diseases, managing symptoms and preventing disease progression. 4–7 Their work will include promoting patients’ self-care through improving Health Literacy (HL) skills. 6 7 HL means, on the part of the patient, to access, understand and use health-related information and services, enabling informed health decisions that help improve, protect and promote the quality of life for themselves, their families and communities. 8–10 Specifically, low HL is associated with adverse health outcomes, including increased emergency department visits, reduced chronic disease management, poor medication adherence, hospitalisation and elevated mortality risk. 11 12 Consequently, it is critical to prepare nurses to address the challenges posed by patients with limited HL skills. 13

Scholars must actively engage in HL research, education and promotion to address these critical needs. 14 Previous works by McCleary-Jones 15 and Toronto and Weatherford 16 have highlighted the paucity and inconsistency of HL content in nursing curricula, highlighting the need to prioritise the development of HL-related skills, specially teaching and assessment methods. 15 16 McCleary-Jones 15 and Toronto and Weatherford 16 stressed the urgency of rigorous research on methods and procedures to generate understanding the subject matter and develop the relevant skills. This knowledge might offer insight on existing research, at the same time identifying potential gaps and informing future investigations. Advances in this field of research could significantly benefit nursing education by adapting more effective models and strategies, ultimately ensuring that students’ proficiency in HL translates into professionals capable of addressing the current healthcare needs of chronic patients. 6 7 13

Despite the relevance of HL proficiency among nursing students, no synthesis has been conducted to summarise findings, identify gaps and guide scholars in further research. Therefore, this study aims to identify the primary sources and types of evidence available regarding HL skills in nursing students, contributing to the development of research in this critical field.

Methods and analysis

Study design.

A scoping review will be conducted following Arksey and O’Malley’s 17 five-stage framework, which includes (1) identifying the research question based on the patient (P), concept (C) and context (C), (2) identifying relevant studies, (3) selecting the studies, (4) charting the data and (5) collating, summarising and reporting the results. 17 In developing this study, researchers will consult the Joanna Briggs Institute manual. 18 19 The protocol was registered with the Open Science Framework (OSF) on 20 April 2023 (LINK).

Identifying research questions

Given the aim/s of this study, our research questions are the following:

(1) What are the areas of research regarding HL skills among nursing students that have already been extensively studied? (2) Which areas require further investigation? (3) What are the methods used in the existing literature, and which of these could be more effective in future research?

The research questions will be formulated using key elements aligned with the Population, Concept and Context framework approach. 18 19 The population will be active undergraduate nursing students of every year of the programme, of any age, sex and nationality. The concept, focusing on interventions, phenomena of interest and outcomes, revolves around the development of HL skills throughout the undergraduate programme. HL skills are linked to Information Literacy, that is the ability to recognise any moment when information is needed. 8–10 The context is defined as the academic or educational setting, of every region, and cultural background they come from.

Search strategy

Four authors (GF, RM, RMSL and IZ) will undertake the article search in three sequential steps. Initially, they will review some studies on HL in nursing students to identify subject headings and keywords to narrow down the search strategy. In the second step, the researchers, under the supervision of a senior researcher (MB), will engage in discussions in order to reach a consensus on databases selection and search terms. The chosen online databases include MEDLINE (via PubMed), Education Resources Information Centre, the Cumulative Index to Nursing and Allied Health Literature, Scopus, Web of Science and Google Scholar. The search terms include “ Nursing Students ” (Mesh), “ Health Literacy ” (Mesh), “ Information Literacy ” (Mesh) and “ Clinical Competence ” (Mesh), combined using Boolean operators ‘AND’ and ‘OR’ ( table 1 ). Each study will be imported into the RefWorks database manager to discard duplicates. In the third and final step, the reference lists of all identified reports will be analysed for additional studies.

  • View inline

Eligibility criteria

Papers on HL skills will be selected, especially those that focus on nursing students pursuing a nursing-degree level. The inclusion criteria consist of primary studies employing diverse research designs, including quantitative, qualitative and mixed methods. Selected studies are those already published in peer-reviewed journals up until February 2023 and will be available in either Italian or English. Letters, comments, conference abstracts, editorials, books, any form of review and doctoral theses will not be included. Where sample details lack clarity, researchers will meticulously examine the full text to ensure the inclusion of nursing students related material and information capable of providing data. No time frame limitation will be imposed ( table 2 ).

Selection of sources of evidence, charting the data process

Selection of sources of evidence

Based on the inclusion and exclusion criteria, three researchers (BG, VB and RM) will first independently screen titles, abstracts for inclusion criteria and then their full text. When a full text is not available, the researchers will contact the authors. Every researcher will structure a table in excel with title, authors, publication data, journal and notes to share information in a meeting with the senior researcher. In the case of disagreement concerning the inclusion of a study, an independent assessment will be conducted by a senior researcher (MB) ( table 2 ).

The number of the records removed and reasons for their removal will be reported in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 flow diagram for systematic reviews. 20

Charting the data

After selecting the final set of studies, each researcher (BG, VB and RM) will independently extract and chart the data for the first five studies. Then the team will discuss the findings together for problems related to this process and purposes of adding data to extract. The researchers then will continue the charting data independently with a final meeting in order to share their results. In the case of disagreement, an independent assessment will be conducted by a fourth author (MB) ( table 2 ).

The researchers (BG, VB and RM) will independently abstract their data with regard to authors, year of publication, country of origin, aim, study design and sample size. In the case of studies considering non-nursing undergraduate, only the nursing students’ sample size will be given. Researchers will also extract the instruments used for HL measurement as well as the author of the instrument, the concept of HL and findings related to the aim of the study. In the case of qualitative studies, the themes that will emerge will be analysed and synthesised ( table 2 ).

Critical appraisal of individual sources of evidence

Given the intent of the review, the quality of the included studies will not be assessed. 21

Synthesis and presentation of results

The results will be described through each of their research questions. The quantitative and qualitative results will be reported in the main text and tables. Qualitative results will be synthesised, adopting the content analysis method. 22 Content analysis as a research method is a systematic and objective way to describe and quantify phenomena; it is useful for making replicable and valid inferences from data. 22

Specifically, the results section will consist of two parts: the first will describe the results of the search strategy and selection process, including a PRISMA flow diagram, while the second part will provide key information regarding the characteristics of the selected studies (design of the study, country, year of publication, sample, instrument for assessing the HL skills and main finding).

Ethics and dissemination

This scoping review is part of a large project of the university of Catania aimed at developing higher quality educational approaches to nursing curricula. This project will not involve patients/students/public and will not require ethical committee approval. This scoping review will be submitted to international peer-reviewed journal. The protocol was registered with the OSF on 20 April 2023 (LINK).

Patient and public involvement

None. The patients and the public are not involved in this project neither in the dissemination plans.

Discussion and conclusion

The aim of this scoping review is to identify the main sources and types of available evidence regarding HL skills among nursing students to further develop knowledge and understanding in this area of research.

This could pave the way for innovative strategies in higher education to significantly enhance nursing students’ learning and clinical experience.

Furthermore, this scoping review will seek to both address some of the most important gaps in HL skills among nursing students and facilitate new approaches in healthcare practice, research and education.

Ethics statements

Patient consent for publication.

Not applicable.

  • World Health Organization Joint Governance Working Group
  • Metzing S ,
  • Jürgensen IN , et al
  • Wilandika A ,
  • Pandin MGR ,
  • Wildeboer AT ,
  • van Dooren AA , et al
  • Parnell TA ,
  • Stichler JF , et al
  • Pender NJ ,
  • Murdaugh CL ,
  • World Health Organization
  • Speroff T ,
  • Worley K , et al
  • Ishikawa H ,
  • Okuhara T , et al
  • Anderson JE ,
  • Oppenheimer S , et al
  • Whitehead D
  • McCleary-Jones V
  • Toronto CE ,
  • Weatherford B
  • Peters MDJ ,
  • Godfrey CM ,
  • Khalil H , et al
  • Tricco AC , et al
  • Tricco AC ,
  • Zarin W , et al
  • Colquhoun H ,

Contributors AA and MB are the senior researchers. GF, RMSL, RM and IZ developed the search strategy. GB, VB and RM are the reviewers. All authors drafted the manuscript, contributed to the development of the selection criteria and data extraction criteria. All authors read, provided feedback and approved the final manuscript.

Funding STARTING GRANT, University of Catania, Catania, Italy—No 85194 of 20230222 (2023-UNCTCLE-0085194). Incentives PLAN for University RESEARCH 2020/2022 (C.o.A. of 04.29.2020)—Intervention line 3 "Starting Grant".

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

  • Systematic review
  • Open access
  • Published: 19 February 2024

‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice

  • Annette Boaz   ORCID: orcid.org/0000-0003-0557-1294 1 ,
  • Juan Baeza 2 ,
  • Alec Fraser   ORCID: orcid.org/0000-0003-1121-1551 2 &
  • Erik Persson 3  

Implementation Science volume  19 , Article number:  15 ( 2024 ) Cite this article

1940 Accesses

70 Altmetric

Metrics details

The gap between research findings and clinical practice is well documented and a range of strategies have been developed to support the implementation of research into clinical practice. The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice.

We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched from January 2010 up to June 2022 and applied no language restrictions. Two independent reviewers appraised the quality of included studies using a quality assessment checklist. To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes.

We identified 32 reviews conducted between 2010 and 2022. The reviews are mainly of multi-faceted interventions ( n  = 20) although there are reviews focusing on single strategies (ICT, educational, reminders, local opinion leaders, audit and feedback, social media and toolkits). The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Furthermore, a lot of nuance lies behind these headline findings, and this is increasingly commented upon in the reviews themselves.

Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been identified. We need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of research perspectives (including social science) in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed.

Peer Review reports

Contribution to the literature

Considerable time and money is invested in implementing and evaluating strategies to increase the implementation of research into clinical practice.

The growing body of evidence is not providing the anticipated clear lessons to support improved implementation.

Instead what is needed is better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice.

This would involve a more central role in implementation science for a wider range of perspectives, especially from the social, economic, political and behavioural sciences and for greater use of different types of synthesis, such as realist synthesis.

Introduction

The gap between research findings and clinical practice is well documented and a range of interventions has been developed to increase the implementation of research into clinical practice [ 1 , 2 ]. In recent years researchers have worked to improve the consistency in the ways in which these interventions (often called strategies) are described to support their evaluation. One notable development has been the emergence of Implementation Science as a field focusing explicitly on “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice” ([ 3 ] p. 1). The work of implementation science focuses on closing, or at least narrowing, the gap between research and practice. One contribution has been to map existing interventions, identifying 73 discreet strategies to support research implementation [ 4 ] which have been grouped into 9 clusters [ 5 ]. The authors note that they have not considered the evidence of effectiveness of the individual strategies and that a next step is to understand better which strategies perform best in which combinations and for what purposes [ 4 ]. Other authors have noted that there is also scope to learn more from other related fields of study such as policy implementation [ 6 ] and to draw on methods designed to support the evaluation of complex interventions [ 7 ].

The increase in activity designed to support the implementation of research into practice and improvements in reporting provided the impetus for an update of a review of systematic reviews of the effectiveness of interventions designed to support the use of research in clinical practice [ 8 ] which was itself an update of the review conducted by Grimshaw and colleagues in 2001. The 2001 review [ 9 ] identified 41 reviews considering a range of strategies including educational interventions, audit and feedback, computerised decision support to financial incentives and combined interventions. The authors concluded that all the interventions had the potential to promote the uptake of evidence in practice, although no one intervention seemed to be more effective than the others in all settings. They concluded that combined interventions were more likely to be effective than single interventions. The 2011 review identified a further 13 systematic reviews containing 313 discrete primary studies. Consistent with the previous review, four main strategy types were identified: audit and feedback; computerised decision support; opinion leaders; and multi-faceted interventions (MFIs). Nine of the reviews reported on MFIs. The review highlighted the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. MFIs claimed an improvement in effectiveness over single interventions, although effect sizes remained small to moderate and this improvement in effectiveness relating to MFIs has been questioned in a subsequent review [ 10 ]. In updating the review, we anticipated a larger pool of reviews and an opportunity to consolidate learning from more recent systematic reviews of interventions.

This review updates and extends our previous review of systematic reviews of interventions designed to implement research evidence into clinical practice. To identify potentially relevant peer-reviewed research papers, we developed a comprehensive systematic literature search strategy based on the terms used in the Grimshaw et al. [ 9 ] and Boaz, Baeza and Fraser [ 8 ] overview articles. To ensure optimal retrieval, our search strategy was refined with support from an expert university librarian, considering the ongoing improvements in the development of search filters for systematic reviews since our first review [ 11 ]. We also wanted to include technology-related terms (e.g. apps, algorithms, machine learning, artificial intelligence) to find studies that explored interventions based on the use of technological innovations as mechanistic tools for increasing the use of evidence into practice (see Additional file 1 : Appendix A for full search strategy).

The search was performed in June 2022 in the following electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched for articles published since the 2011 review. We searched from January 2010 up to June 2022 and applied no language restrictions. Reference lists of relevant papers were also examined.

We uploaded the results using EPPI-Reviewer, a web-based tool that facilitated semi-automation of the screening process and removal of duplicate studies. We made particular use of a priority screening function to reduce screening workload and avoid ‘data deluge’ [ 12 ]. Through machine learning, one reviewer screened a smaller number of records ( n  = 1200) to train the software to predict whether a given record was more likely to be relevant or irrelevant, thus pulling the relevant studies towards the beginning of the screening process. This automation did not replace manual work but helped the reviewer to identify eligible studies more quickly. During the selection process, we included studies that looked explicitly at interventions designed to turn research evidence into practice. Studies were included if they met the following pre-determined inclusion criteria:

The study was a systematic review

Search terms were included

Focused on the implementation of research evidence into practice

The methodological quality of the included studies was assessed as part of the review

Study populations included healthcare providers and patients. The EPOC taxonomy [ 13 ] was used to categorise the strategies. The EPOC taxonomy has four domains: delivery arrangements, financial arrangements, governance arrangements and implementation strategies. The implementation strategies domain includes 20 strategies targeted at healthcare workers. Numerous EPOC strategies were assessed in the review including educational strategies, local opinion leaders, reminders, ICT-focused approaches and audit and feedback. Some strategies that did not fit easily within the EPOC categories were also included. These were social media strategies and toolkits, and multi-faceted interventions (MFIs) (see Table  2 ). Some systematic reviews included comparisons of different interventions while other reviews compared one type of intervention against a control group. Outcomes related to improvements in health care processes or patient well-being. Numerous individual study types (RCT, CCT, BA, ITS) were included within the systematic reviews.

We excluded papers that:

Focused on changing patient rather than provider behaviour

Had no demonstrable outcomes

Made unclear or no reference to research evidence

The last of these criteria was sometimes difficult to judge, and there was considerable discussion amongst the research team as to whether the link between research evidence and practice was sufficiently explicit in the interventions analysed. As we discussed in the previous review [ 8 ] in the field of healthcare, the principle of evidence-based practice is widely acknowledged and tools to change behaviour such as guidelines are often seen to be an implicit codification of evidence, despite the fact that this is not always the case.

Reviewers employed a two-stage process to select papers for inclusion. First, all titles and abstracts were screened by one reviewer to determine whether the study met the inclusion criteria. Two papers [ 14 , 15 ] were identified that fell just before the 2010 cut-off. As they were not identified in the searches for the first review [ 8 ] they were included and progressed to assessment. Each paper was rated as include, exclude or maybe. The full texts of 111 relevant papers were assessed independently by at least two authors. To reduce the risk of bias, papers were excluded following discussion between all members of the team. 32 papers met the inclusion criteria and proceeded to data extraction. The study selection procedure is documented in a PRISMA literature flow diagram (see Fig.  1 ). We were able to include French, Spanish and Portuguese papers in the selection reflecting the language skills in the study team, but none of the papers identified met the inclusion criteria. Other non- English language papers were excluded.

figure 1

PRISMA flow diagram. Source: authors

One reviewer extracted data on strategy type, number of included studies, local, target population, effectiveness and scope of impact from the included studies. Two reviewers then independently read each paper and noted key findings and broad themes of interest which were then discussed amongst the wider authorial team. Two independent reviewers appraised the quality of included studies using a Quality Assessment Checklist based on Oxman and Guyatt [ 16 ] and Francke et al. [ 17 ]. Each study was rated a quality score ranging from 1 (extensive flaws) to 7 (minimal flaws) (see Additional file 2 : Appendix B). All disagreements were resolved through discussion. Studies were not excluded in this updated overview based on methodological quality as we aimed to reflect the full extent of current research into this topic.

The extracted data were synthesised using descriptive and narrative techniques to identify themes and patterns in the data linked to intervention strategies, targeted behaviours, study settings and study outcomes.

Thirty-two studies were included in the systematic review. Table 1. provides a detailed overview of the included systematic reviews comprising reference, strategy type, quality score, number of included studies, local, target population, effectiveness and scope of impact (see Table  1. at the end of the manuscript). Overall, the quality of the studies was high. Twenty-three studies scored 7, six studies scored 6, one study scored 5, one study scored 4 and one study scored 3. The primary focus of the review was on reviews of effectiveness studies, but a small number of reviews did include data from a wider range of methods including qualitative studies which added to the analysis in the papers [ 18 , 19 , 20 , 21 ]. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. In this section, we discuss the different EPOC-defined implementation strategies in turn. Interestingly, we found only two ‘new’ approaches in this review that did not fit into the existing EPOC approaches. These are a review focused on the use of social media and a review considering toolkits. In addition to single interventions, we also discuss multi-faceted interventions. These were the most common intervention approach overall. A summary is provided in Table  2 .

Educational strategies

The overview identified three systematic reviews focusing on educational strategies. Grudniewicz et al. [ 22 ] explored the effectiveness of printed educational materials on primary care physician knowledge, behaviour and patient outcomes and concluded they were not effective in any of these aspects. Koota, Kääriäinen and Melender [ 23 ] focused on educational interventions promoting evidence-based practice among emergency room/accident and emergency nurses and found that interventions involving face-to-face contact led to significant or highly significant effects on patient benefits and emergency nurses’ knowledge, skills and behaviour. Interventions using written self-directed learning materials also led to significant improvements in nurses’ knowledge of evidence-based practice. Although the quality of the studies was high, the review primarily included small studies with low response rates, and many of them relied on self-assessed outcomes; consequently, the strength of the evidence for these outcomes is modest. Wu et al. [ 20 ] questioned if educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes. Although based on evaluation projects and qualitative data, their results also suggest that positive changes on patient outcomes can be made following the implementation of specific evidence-based approaches (or projects). The differing positive outcomes for educational strategies aimed at nurses might indicate that the target audience is important.

Local opinion leaders

Flodgren et al. [ 24 ] was the only systemic review focusing solely on opinion leaders. The review found that local opinion leaders alone, or in combination with other interventions, can be effective in promoting evidence‐based practice, but this varies both within and between studies and the effect on patient outcomes is uncertain. The review found that, overall, any intervention involving opinion leaders probably improves healthcare professionals’ compliance with evidence-based practice but varies within and across studies. However, how opinion leaders had an impact could not be determined because of insufficient details were provided, illustrating that reporting specific details in published studies is important if diffusion of effective methods of increasing evidence-based practice is to be spread across a system. The usefulness of this review is questionable because it cannot provide evidence of what is an effective opinion leader, whether teams of opinion leaders or a single opinion leader are most effective, or the most effective methods used by opinion leaders.

Pantoja et al. [ 26 ] was the only systemic review focusing solely on manually generated reminders delivered on paper included in the overview. The review explored how these affected professional practice and patient outcomes. The review concluded that manually generated reminders delivered on paper as a single intervention probably led to small to moderate increases in adherence to clinical recommendations, and they could be used as a single quality improvement intervention. However, the authors indicated that this intervention would make little or no difference to patient outcomes. The authors state that such a low-tech intervention may be useful in low- and middle-income countries where paper records are more likely to be the norm.

ICT-focused approaches

The three ICT-focused reviews [ 14 , 27 , 28 ] showed mixed results. Jamal, McKenzie and Clark [ 14 ] explored the impact of health information technology on the quality of medical and health care. They examined the impact of electronic health record, computerised provider order-entry, or decision support system. This showed a positive improvement in adherence to evidence-based guidelines but not to patient outcomes. The number of studies included in the review was low and so a conclusive recommendation could not be reached based on this review. Similarly, Brown et al. [ 28 ] found that technology-enabled knowledge translation interventions may improve knowledge of health professionals, but all eight studies raised concerns of bias. The De Angelis et al. [ 27 ] review was more promising, reporting that ICT can be a good way of disseminating clinical practice guidelines but conclude that it is unclear which type of ICT method is the most effective.

Audit and feedback

Sykes, McAnuff and Kolehmainen [ 29 ] examined whether audit and feedback were effective in dementia care and concluded that it remains unclear which ingredients of audit and feedback are successful as the reviewed papers illustrated large variations in the effectiveness of interventions using audit and feedback.

Non-EPOC listed strategies: social media, toolkits

There were two new (non-EPOC listed) intervention types identified in this review compared to the 2011 review — fewer than anticipated. We categorised a third — ‘care bundles’ [ 36 ] as a multi-faceted intervention due to its description in practice and a fourth — ‘Technology Enhanced Knowledge Transfer’ [ 28 ] was classified as an ICT-focused approach. The first new strategy was identified in Bhatt et al.’s [ 30 ] systematic review of the use of social media for the dissemination of clinical practice guidelines. They reported that the use of social media resulted in a significant improvement in knowledge and compliance with evidence-based guidelines compared with more traditional methods. They noted that a wide selection of different healthcare professionals and patients engaged with this type of social media and its global reach may be significant for low- and middle-income countries. This review was also noteworthy for developing a simple stepwise method for using social media for the dissemination of clinical practice guidelines. However, it is debatable whether social media can be classified as an intervention or just a different way of delivering an intervention. For example, the review discussed involving opinion leaders and patient advocates through social media. However, this was a small review that included only five studies, so further research in this new area is needed. Yamada et al. [ 31 ] draw on 39 studies to explore the application of toolkits, 18 of which had toolkits embedded within larger KT interventions, and 21 of which evaluated toolkits as standalone interventions. The individual component strategies of the toolkits were highly variable though the authors suggest that they align most closely with educational strategies. The authors conclude that toolkits as either standalone strategies or as part of MFIs hold some promise for facilitating evidence use in practice but caution that the quality of many of the primary studies included is considered weak limiting these findings.

Multi-faceted interventions

The majority of the systematic reviews ( n  = 20) reported on more than one intervention type. Some of these systematic reviews focus exclusively on multi-faceted interventions, whilst others compare different single or combined interventions aimed at achieving similar outcomes in particular settings. While these two approaches are often described in a similar way, they are actually quite distinct from each other as the former report how multiple strategies may be strategically combined in pursuance of an agreed goal, whilst the latter report how different strategies may be incidentally used in sometimes contrasting settings in the pursuance of similar goals. Ariyo et al. [ 35 ] helpfully summarise five key elements often found in effective MFI strategies in LMICs — but which may also be transferrable to HICs. First, effective MFIs encourage a multi-disciplinary approach acknowledging the roles played by different professional groups to collectively incorporate evidence-informed practice. Second, they utilise leadership drawing on a wide set of clinical and non-clinical actors including managers and even government officials. Third, multiple types of educational practices are utilised — including input from patients as stakeholders in some cases. Fourth, protocols, checklists and bundles are used — most effectively when local ownership is encouraged. Finally, most MFIs included an emphasis on monitoring and evaluation [ 35 ]. In contrast, other studies offer little information about the nature of the different MFI components of included studies which makes it difficult to extrapolate much learning from them in relation to why or how MFIs might affect practice (e.g. [ 28 , 38 ]). Ultimately, context matters, which some review authors argue makes it difficult to say with real certainty whether single or MFI strategies are superior (e.g. [ 21 , 27 ]). Taking all the systematic reviews together we may conclude that MFIs appear to be more likely to generate positive results than single interventions (e.g. [ 34 , 45 ]) though other reviews should make us cautious (e.g. [ 32 , 43 ]).

While multi-faceted interventions still seem to be more effective than single-strategy interventions, there were important distinctions between how the results of reviews of MFIs are interpreted in this review as compared to the previous reviews [ 8 , 9 ], reflecting greater nuance and debate in the literature. This was particularly noticeable where the effectiveness of MFIs was compared to single strategies, reflecting developments widely discussed in previous studies [ 10 ]. We found that most systematic reviews are bounded by their clinical, professional, spatial, system, or setting criteria and often seek to draw out implications for the implementation of evidence in their areas of specific interest (such as nursing or acute care). Frequently this means combining all relevant studies to explore the respective foci of each systematic review. Therefore, most reviews we categorised as MFIs actually include highly variable numbers and combinations of intervention strategies and highly heterogeneous original study designs. This makes statistical analyses of the type used by Squires et al. [ 10 ] on the three reviews in their paper not possible. Further, it also makes extrapolating findings and commenting on broad themes complex and difficult. This may suggest that future research should shift its focus from merely examining ‘what works’ to ‘what works where and what works for whom’ — perhaps pointing to the value of realist approaches to these complex review topics [ 48 , 49 ] and other more theory-informed approaches [ 50 ].

Some reviews have a relatively small number of studies (i.e. fewer than 10) and the authors are often understandably reluctant to engage with wider debates about the implications of their findings. Other larger studies do engage in deeper discussions about internal comparisons of findings across included studies and also contextualise these in wider debates. Some of the most informative studies (e.g. [ 35 , 40 ]) move beyond EPOC categories and contextualise MFIs within wider systems thinking and implementation theory. This distinction between MFIs and single interventions can actually be very useful as it offers lessons about the contexts in which individual interventions might have bounded effectiveness (i.e. educational interventions for individual change). Taken as a whole, this may also then help in terms of how and when to conjoin single interventions into effective MFIs.

In the two previous reviews, a consistent finding was that MFIs were more effective than single interventions [ 8 , 9 ]. However, like Squires et al. [ 10 ] this overview is more equivocal on this important issue. There are four points which may help account for the differences in findings in this regard. Firstly, the diversity of the systematic reviews in terms of clinical topic or setting is an important factor. Secondly, there is heterogeneity of the studies within the included systematic reviews themselves. Thirdly, there is a lack of consistency with regards to the definition and strategies included within of MFIs. Finally, there are epistemological differences across the papers and the reviews. This means that the results that are presented depend on the methods used to measure, report, and synthesise them. For instance, some reviews highlight that education strategies can be useful to improve provider understanding — but without wider organisational or system-level change, they may struggle to deliver sustained transformation [ 19 , 44 ].

It is also worth highlighting the importance of the theory of change underlying the different interventions. Where authors of the systematic reviews draw on theory, there is space to discuss/explain findings. We note a distinction between theoretical and atheoretical systematic review discussion sections. Atheoretical reviews tend to present acontextual findings (for instance, one study found very positive results for one intervention, and this gets highlighted in the abstract) whilst theoretically informed reviews attempt to contextualise and explain patterns within the included studies. Theory-informed systematic reviews seem more likely to offer more profound and useful insights (see [ 19 , 35 , 40 , 43 , 45 ]). We find that the most insightful systematic reviews of MFIs engage in theoretical generalisation — they attempt to go beyond the data of individual studies and discuss the wider implications of the findings of the studies within their reviews drawing on implementation theory. At the same time, they highlight the active role of context and the wider relational and system-wide issues linked to implementation. It is these types of investigations that can help providers further develop evidence-based practice.

This overview has identified a small, but insightful set of papers that interrogate and help theorise why, how, for whom, and in which circumstances it might be the case that MFIs are superior (see [ 19 , 35 , 40 ] once more). At the level of this overview — and in most of the systematic reviews included — it appears to be the case that MFIs struggle with the question of attribution. In addition, there are other important elements that are often unmeasured, or unreported (e.g. costs of the intervention — see [ 40 ]). Finally, the stronger systematic reviews [ 19 , 35 , 40 , 43 , 45 ] engage with systems issues, human agency and context [ 18 ] in a way that was not evident in the systematic reviews identified in the previous reviews [ 8 , 9 ]. The earlier reviews lacked any theory of change that might explain why MFIs might be more effective than single ones — whereas now some systematic reviews do this, which enables them to conclude that sometimes single interventions can still be more effective.

As Nilsen et al. ([ 6 ] p. 7) note ‘Study findings concerning the effectiveness of various approaches are continuously synthesized and assembled in systematic reviews’. We may have gone as far as we can in understanding the implementation of evidence through systematic reviews of single and multi-faceted interventions and the next step would be to conduct more research exploring the complex and situated nature of evidence used in clinical practice and by particular professional groups. This would further build on the nuanced discussion and conclusion sections in a subset of the papers we reviewed. This might also support the field to move away from isolating individual implementation strategies [ 6 ] to explore the complex processes involving a range of actors with differing capacities [ 51 ] working in diverse organisational cultures. Taxonomies of implementation strategies do not fully account for the complex process of implementation, which involves a range of different actors with different capacities and skills across multiple system levels. There is plenty of work to build on, particularly in the social sciences, which currently sits at the margins of debates about evidence implementation (see for example, Normalisation Process Theory [ 52 ]).

There are several changes that we have identified in this overview of systematic reviews in comparison to the review we published in 2011 [ 8 ]. A consistent and welcome finding is that the overall quality of the systematic reviews themselves appears to have improved between the two reviews, although this is not reflected upon in the papers. This is exhibited through better, clearer reporting mechanisms in relation to the mechanics of the reviews, alongside a greater attention to, and deeper description of, how potential biases in included papers are discussed. Additionally, there is an increased, but still limited, inclusion of original studies conducted in low- and middle-income countries as opposed to just high-income countries. Importantly, we found that many of these systematic reviews are attuned to, and comment upon the contextual distinctions of pursuing evidence-informed interventions in health care settings in different economic settings. Furthermore, systematic reviews included in this updated article cover a wider set of clinical specialities (both within and beyond hospital settings) and have a focus on a wider set of healthcare professions — discussing both similarities, differences and inter-professional challenges faced therein, compared to the earlier reviews. These wider ranges of studies highlight that a particular intervention or group of interventions may work well for one professional group but be ineffective for another. This diversity of study settings allows us to consider the important role context (in its many forms) plays on implementing evidence into practice. Examining the complex and varied context of health care will help us address what Nilsen et al. ([ 6 ] p. 1) described as, ‘society’s health problems [that] require research-based knowledge acted on by healthcare practitioners together with implementation of political measures from governmental agencies’. This will help us shift implementation science to move, ‘beyond a success or failure perspective towards improved analysis of variables that could explain the impact of the implementation process’ ([ 6 ] p. 2).

This review brings together 32 papers considering individual and multi-faceted interventions designed to support the use of evidence in clinical practice. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been conducted. As a whole, this substantial body of knowledge struggles to tell us more about the use of individual and MFIs than: ‘it depends’. To really move forwards in addressing the gap between research evidence and practice, we may need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of perspectives, especially from the social, economic, political and behavioural sciences in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed. Harvey et al. [ 53 ] suggest that when context is likely to be critical to implementation success there are a range of primary research approaches (participatory research, realist evaluation, developmental evaluation, ethnography, quality/ rapid cycle improvement) that are likely to be appropriate and insightful. While these approaches often form part of implementation studies in the form of process evaluations, they are usually relatively small scale in relation to implementation research as a whole. As a result, the findings often do not make it into the subsequent systematic reviews. This review provides further evidence that we need to bring qualitative approaches in from the periphery to play a central role in many implementation studies and subsequent evidence syntheses. It would be helpful for systematic reviews, at the very least, to include more detail about the interventions and their implementation in terms of how and why they worked.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Before and after study

Controlled clinical trial

Effective Practice and Organisation of Care

High-income countries

Information and Communications Technology

Interrupted time series

Knowledge translation

Low- and middle-income countries

Randomised controlled trial

Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–30. https://doi.org/10.1016/S0140-6736(03)14546-1 .

Article   PubMed   Google Scholar  

Green LA, Seifert CM. Translation of research into practice: why we can’t “just do it.” J Am Board Fam Pract. 2005;18:541–5. https://doi.org/10.3122/jabfm.18.6.541 .

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1–3. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:2–14. https://doi.org/10.1186/s13012-015-0209-1 .

Article   Google Scholar  

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:1–8. https://doi.org/10.1186/s13012-015-0295-0 .

Nilsen P, Ståhl C, Roback K, et al. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Sci. 2013;8:2–12. https://doi.org/10.1186/1748-5908-8-63 .

Rycroft-Malone J, Seers K, Eldh AC, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Sci. 2018;13:1–15. https://doi.org/10.1186/s13012-018-0811-0 .

Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS). Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011;4:212. https://doi.org/10.1186/1756-0500-4-212 .

Article   PubMed   PubMed Central   Google Scholar  

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior – an overview of systematic reviews of interventions. Med Care. 2001;39 8Suppl 2:II2–45.

Google Scholar  

Squires JE, Sullivan K, Eccles MP, et al. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:1–22. https://doi.org/10.1186/s13012-014-0152-6 .

Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc. 2021;109:561–74. https://doi.org/10.5195/jmla.2021.1223 .

Thomas JM. Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid Based Med. 2013;1:1–6.

Effective Practice and Organisation of Care (EPOC). The EPOC taxonomy of health systems interventions. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. epoc.cochrane.org/epoc-taxonomy . Accessed 9 Oct 2023.

Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag. 2009;38:26–37. https://doi.org/10.1177/183335830903800305 .

Menon A, Korner-Bitensky N, Kastner M, et al. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. https://doi.org/10.2340/16501977-0451 .

Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8. https://doi.org/10.1016/0895-4356(91)90160-b .

Article   CAS   PubMed   Google Scholar  

Francke AL, Smit MC, de Veer AJ, et al. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:1–11. https://doi.org/10.1186/1472-6947-8-38 .

Jones CA, Roop SC, Pohar SL, et al. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. https://doi.org/10.2522/ptj.20130512 .

Scott D, Albrecht L, O’Leary K, Ball GDC, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:1–17. https://doi.org/10.1186/1748-5908-7-70 .

Wu Y, Brettle A, Zhou C, Ou J, et al. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ Today. 2018;70:109–14. https://doi.org/10.1016/j.nedt.2018.08.026 .

Yost J, Ganann R, Thompson D, Aloweni F, et al. The effectiveness of knowledge translation interventions for promoting evidence-informed decision-making among nurses in tertiary care: a systematic review and meta-analysis. Implement Sci. 2015;10:1–15. https://doi.org/10.1186/s13012-015-0286-1 .

Grudniewicz A, Kealy R, Rodseth RN, Hamid J, et al. What is the effectiveness of printed educational materials on primary care physician knowledge, behaviour, and patient outcomes: a systematic review and meta-analyses. Implement Sci. 2015;10:2–12. https://doi.org/10.1186/s13012-015-0347-5 .

Koota E, Kääriäinen M, Melender HL. Educational interventions promoting evidence-based practice among emergency nurses: a systematic review. Int Emerg Nurs. 2018;41:51–8. https://doi.org/10.1016/j.ienj.2018.06.004 .

Flodgren G, O’Brien MA, Parmelli E, et al. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD000125.pub5 .

Arditi C, Rège-Walther M, Durieux P, et al. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD001175.pub4 .

Pantoja T, Grimshaw JM, Colomer N, et al. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD001174.pub4 .

De Angelis G, Davies B, King J, McEwan J, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ. 2016;2:e16. https://doi.org/10.2196/mededu.6288 .

Brown A, Barnes C, Byaruhanga J, McLaughlin M, et al. Effectiveness of technology-enabled knowledge translation strategies in improving the use of research in public health: systematic review. J Med Internet Res. 2020;22:e17274. https://doi.org/10.2196/17274 .

Sykes MJ, McAnuff J, Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. Int J Nurs Stud. 2018;79:27–35. https://doi.org/10.1016/j.ijnurstu.2017.10.013 .

Bhatt NR, Czarniecki SW, Borgmann H, et al. A systematic review of the use of social media for dissemination of clinical practice guidelines. Eur Urol Focus. 2021;7:1195–204. https://doi.org/10.1016/j.euf.2020.10.008 .

Yamada J, Shorkey A, Barwick M, Widger K, et al. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open. 2015;5:e006808. https://doi.org/10.1136/bmjopen-2014-006808 .

Afari-Asiedu S, Abdulai MA, Tostmann A, et al. Interventions to improve dispensing of antibiotics at the community level in low and middle income countries: a systematic review. J Glob Antimicrob Resist. 2022;29:259–74. https://doi.org/10.1016/j.jgar.2022.03.009 .

Boonacker CW, Hoes AW, Dikhoff MJ, Schilder AG, et al. Interventions in health care professionals to improve treatment in children with upper respiratory tract infections. Int J Pediatr Otorhinolaryngol. 2010;74:1113–21. https://doi.org/10.1016/j.ijporl.2010.07.008 .

Al Zoubi FM, Menon A, Mayo NE, et al. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18:2–11. https://doi.org/10.1186/s12913-018-3253-0 .

Ariyo P, Zayed B, Riese V, Anton B, et al. Implementation strategies to reduce surgical site infections: a systematic review. Infect Control Hosp Epidemiol. 2019;3:287–300. https://doi.org/10.1017/ice.2018.355 .

Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci. 2015;10:1–11. https://doi.org/10.1186/s13012-015-0306-1 .

Cahill LS, Carey LM, Lannin NA, et al. Implementation interventions to promote the uptake of evidence-based practices in stroke rehabilitation. Cochrane Database Syst Rev. 2020. https://doi.org/10.1002/14651858.CD012575.pub2 .

Pedersen ER, Rubenstein L, Kandrack R, Danz M, et al. Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression. Implement Sci. 2018;13:1–30. https://doi.org/10.1186/s13012-018-0788-8 .

Jenkins HJ, Hancock MJ, French SD, Maher CG, et al. Effectiveness of interventions designed to reduce the use of imaging for low-back pain: a systematic review. CMAJ. 2015;187:401–8. https://doi.org/10.1503/cmaj.141183 .

Bennett S, Laver K, MacAndrew M, Beattie E, et al. Implementation of evidence-based, non-pharmacological interventions addressing behavior and psychological symptoms of dementia: a systematic review focused on implementation strategies. Int Psychogeriatr. 2021;33:947–75. https://doi.org/10.1017/S1041610220001702 .

Noonan VK, Wolfe DL, Thorogood NP, et al. Knowledge translation and implementation in spinal cord injury: a systematic review. Spinal Cord. 2014;52:578–87. https://doi.org/10.1038/sc.2014.62 .

Albrecht L, Archibald M, Snelgrove-Clarke E, et al. Systematic review of knowledge translation strategies to promote research uptake in child health settings. J Pediatr Nurs. 2016;31:235–54. https://doi.org/10.1016/j.pedn.2015.12.002 .

Campbell A, Louie-Poon S, Slater L, et al. Knowledge translation strategies used by healthcare professionals in child health settings: an updated systematic review. J Pediatr Nurs. 2019;47:114–20. https://doi.org/10.1016/j.pedn.2019.04.026 .

Bird ML, Miller T, Connell LA, et al. Moving stroke rehabilitation evidence into practice: a systematic review of randomized controlled trials. Clin Rehabil. 2019;33:1586–95. https://doi.org/10.1177/0269215519847253 .

Goorts K, Dizon J, Milanese S. The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: a systematic review. BMC Health Serv Res. 2021;21:1–11. https://doi.org/10.1186/s12913-021-06190-0 .

Zadro JR, O’Keeffe M, Allison JL, Lembke KA, et al. Effectiveness of implementation strategies to improve adherence of physical therapist treatment choices to clinical practice guidelines for musculoskeletal conditions: systematic review. Phys Ther. 2020;100:1516–41. https://doi.org/10.1093/ptj/pzaa101 .

Van der Veer SN, Jager KJ, Nache AM, et al. Translating knowledge on best practice into improving quality of RRT care: a systematic review of implementation strategies. Kidney Int. 2011;80:1021–34. https://doi.org/10.1038/ki.2011.222 .

Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10Suppl 1:21–34. https://doi.org/10.1258/1355819054308530 .

Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research. Implementation Sci. 2012;7:1–10. https://doi.org/10.1186/1748-5908-7-33 .

Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5:e008592. https://doi.org/10.1136/bmjopen-2015-008592 .

Metz A, Jensen T, Farley A, Boaz A, et al. Is implementation research out of step with implementation practice? Pathways to effective implementation support over the last decade. Implement Res Pract. 2022;3:1–11. https://doi.org/10.1177/26334895221105585 .

May CR, Finch TL, Cornford J, Exley C, et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res. 2011;11:1–11. https://doi.org/10.1186/1472-6963-11-131 .

Harvey G, Rycroft-Malone J, Seers K, Wilson P, et al. Connecting the science and practice of implementation – applying the lens of context to inform study design in implementation research. Front Health Serv. 2023;3:1–15. https://doi.org/10.3389/frhs.2023.1162762 .

Download references

Acknowledgements

The authors would like to thank Professor Kathryn Oliver for her support in the planning the review, Professor Steve Hanney for reading and commenting on the final manuscript and the staff at LSHTM library for their support in planning and conducting the literature search.

This study was supported by LSHTM’s Research England QR strategic priorities funding allocation and the National Institute for Health and Care Research (NIHR) Applied Research Collaboration South London (NIHR ARC South London) at King’s College Hospital NHS Foundation Trust. Grant number NIHR200152. The views expressed are those of the author(s) and not necessarily those of the NIHR, the Department of Health and Social Care or Research England.

Author information

Authors and affiliations.

Health and Social Care Workforce Research Unit, The Policy Institute, King’s College London, Virginia Woolf Building, 22 Kingsway, London, WC2B 6LE, UK

Annette Boaz

King’s Business School, King’s College London, 30 Aldwych, London, WC2B 4BG, UK

Juan Baeza & Alec Fraser

Federal University of Santa Catarina (UFSC), Campus Universitário Reitor João Davi Ferreira Lima, Florianópolis, SC, 88.040-900, Brazil

Erik Persson

You can also search for this author in PubMed   Google Scholar

Contributions

AB led the conceptual development and structure of the manuscript. EP conducted the searches and data extraction. All authors contributed to screening and quality appraisal. EP and AF wrote the first draft of the methods section. AB, JB and AF performed result synthesis and contributed to the analyses. AB wrote the first draft of the manuscript and incorporated feedback and revisions from all other authors. All authors revised and approved the final manuscript.

Corresponding author

Correspondence to Annette Boaz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix a., additional file 2: appendix b., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Boaz, A., Baeza, J., Fraser, A. et al. ‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice. Implementation Sci 19 , 15 (2024). https://doi.org/10.1186/s13012-024-01337-z

Download citation

Received : 01 November 2023

Accepted : 05 January 2024

Published : 19 February 2024

DOI : https://doi.org/10.1186/s13012-024-01337-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Implementation
  • Interventions
  • Clinical practice
  • Research evidence
  • Multi-faceted

Implementation Science

ISSN: 1748-5908

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

literary studies research methods

At the Human Longevity Lab, studying methods to slow or reverse aging

Longevity Lab

  • Feinberg School of Medicine

The Potocsnak Longevity Institute at Northwestern University Feinberg School of Medicine has launched the Human Longevity Laboratory, a longitudinal,  cross-sectional study that will investigate the relationship between chronological age and biological age across different organ systems and validate interventions that may reverse or slow down the processes of aging.

“The relationship between chronological age (how many years old you are) and biological age (how old your body appears in terms of your overall health), and how they may differ, is key to understanding human longevity,” said Dr. Douglas Vaughan, director of the Potocsnak Longevity Institute. “Knowledge gained from this research may allow scientists to develop methods to slow the process of aging and push back the onset of aging-related disease, hopefully extending the ‘healthspan.’”

Anyone is eligible to participate in the Northwestern research study, but the scientists are focused on studying people who are disadvantaged with respect to biological aging, including those with HIV.

Our primary aim is to find ways to slow down the rate of aging in people that are aging too quickly and provide them with an opportunity to extend their healthspan.”

“We are particularly interested in bringing in people who are at risk for accelerated aging — people with chronic HIV infections, patients with chronic kidney disease, people exposed to toxic substances regularly (smoke and chemicals) and others,” Vaughan said. “Our primary aim is to find ways to slow down the rate of aging in people that are aging too quickly and provide them with an opportunity to extend their healthspan.”

The comprehensive research protocol includes assessments across various systems (cardiovascular, respiratory, neurocognitive, metabolic, and musculoskeletal), and novel molecular profiling of the epigenome. The studies will be performed at no cost to participants at Northwestern Medicine.

Over the next year, the team plans to enroll a diverse cohort representing individuals of all ages, ethnicities and socioeconomic backgrounds to form a picture of how aging affects all members of the population.

A participant’s results will be reviewed with them after their testing is complete. “That is information that might motivate some participants to improve their lifestyle, exercise more, lose weight or change their diet,” said Dr. John Wilkins, associate director of the Human Longevity Laboratory. Wilkins is also an associate professor of medicine in cardiology and of preventive medicine at the Feinberg School of Medicine, as well as a Northwestern Medicine physician.

Ultimately, the Human Longevity Laboratory will launch clinical trials designed to test therapeutics or interventions that might slow the velocity of aging.

View this site for more information on the study.

literary studies research methods

Dr. Vaughan plans to develop a network of sites duplicating the Human Longevity Laboratory with partners in the U.S. and globally. 

“We hope to clone our laboratory in terms of basic equipment and the protocol,” Vaughan said. “We intend to build a large database that is the most diverse and comprehensive in the world that will contribute significantly to our research.” Potential collaborative partners and sites have already been identified in Asia, Brazil, the Netherlands and in West Africa.

The Human Longevity Laboratory is part of the multi-center Potocsnak Longevity Institute , whose goal is to foster new discoveries and build on Northwestern’s ongoing research in the rapidly advancing science of aging. The Institute is funded by a gift from Chicago industrialist John Potocsnak and family.

“Aging is a primary risk factor for every disease affecting adults — including diabetes, arthritis, dementia, heart disease, diabetes, aging-related cancer, hypertension and frailty,” Vaughan said. “The biological processes that drive aging may be malleable. We think we can slow that process down, delay it, even theoretically reverse it. The curtain is being pulled back on what drives aging. We want to contribute to that larger discovery process.”

Editor’s Picks

mark hersam

Mark Hersam elected to National Academy of Engineering

Two receive nsf career award, looking to the past 100 years since the 1924 indian citizenship act, related stories.

Aging and chronic health conditions

With age, accumulating health problems increase risk of depression and anxiety

New cause of neuron death in alzheimer's discovered, conscientious personalities less at risk of dementia diagnosis.

Logo for University of Central Florida Pressbooks

Introduction to Strategies for Conducting Literary Research

Welcome to Strategies for Conducting Literary Research! This course walks you through the process of conducting literary research while helping to refine your library skills. Along the way, we will draw from the Association of College and Research Libraries (ACRL) Information Literacy Framework . According to the ACRL, “Research is iterative and depends upon asking increasingly complex or new questions whose answers lead to additional questions or lines of inquiry in any field.” We will discuss this concept more in-depth throughout the course. The course will focus on a research project created by Jada, an English major who conducted a literary study of James Baldwin’s  classic short story,  “Sonny’s Blues.” 1

 Meet Jada!

literary studies research methods

The Complexity of Literary Studies Research

In a research-based course, your instructor might require you to write a research paper of 5-6 or 10-12 pages, but the knowledge you need to perform this task can fill up dozens of pages. Writing about literature is a complicated, often messy process; it needs to meet high standards while incorporating knowledge from other fields such as psychology, history, science, and other arts. It entails knowledge about language, genre, structures, styles, and more. To produce good research about literature, we need to know a lot of things about a lot of things!

Although we discuss the research process in a linear fashion throughout these chapters, you’ll find that, in practice, literary research is a highly recursive process. We’re constantly circling back through the process as we write. Because writing instructors (even those who made this course) are locked into presenting the writing process in a linear way, we tend to discuss it in terms of stages such as preliminary research, drafting, revising, and so on. But writing a research paper requires us to rethink and redo our work at any stage. It’s not uncommon for writers to be in the middle of proofreading (one of the final stages) and realize they need to go back and gather more research. Though this course focuses on research about literature, the skills and knowledge in these modules apply to many other areas and topics, especially in the humanities.

Course Learning Objectives

  • Understand the assignment
  • Identify a research problem
  • Develop audience awareness
  • Enter a scholarly conversation
  • Understand theory’s integral role within humanities research
  • Understand how theory relates to particular research methodologies and methods for gathering evidence
  • Learn to use online library catalogs, database search strategies, library services, citation management, and search alerts
  • Evaluate source credibility
  • Posit your research question
  • Posit a thesis statement
  • Compose a title
  • Define your key term
  • Write persuasively
  • Write academic prose
  • Steer clear of plagiarism
  • Finish your research project

Strategies for Conducting Literary Research Copyright © 2021 by Barry Mauer & John Venecek is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

  • Search Menu
  • Supplements
  • Advance articles
  • Editor's Choice
  • Special Issues
  • Author Guidelines
  • Submission Site
  • Why Publish With Us?
  • Open Access
  • About Nicotine & Tobacco Research
  • About Society for Nicotine & Tobacco Research
  • Editorial Board
  • Advertising and Corporate Services
  • Journals Career Network
  • Self-Archiving Policy
  • Dispatch Dates
  • Journals on Oxford Academic
  • Books on Oxford Academic

Article Contents

  • Meta-Analysis
  • Declaration of Interests
  • Author Contributions

The Impact of Menthol Cigarette Bans: A Systematic Review and Meta-Analysis

ORCID logo

  • Article contents
  • Figures & tables
  • Supplementary Data

Sarah D Mills, Snigdha Peddireddy, Rachel Kurtzman, Frantasia Hill, Victor Catalan, Jennifer S Bissram, Kurt M Ribisl, The Impact of Menthol Cigarette Bans: A Systematic Review and Meta-Analysis, Nicotine & Tobacco Research , 2024;, ntae011, https://doi.org/10.1093/ntr/ntae011

  • Permissions Icon Permissions

This review investigates the impacts of banning the sale of menthol cigarettes at stores.

A systematic search of studies published in English up to November 2022 was conducted. The following databases were searched: PubMed/Medline, CINAHL, PsycINFO, Web of Science, and Embase, as well as a non-indexed journal. Studies evaluating either the impact of real-world or hypothesized menthol cigarette bans were included. Primary outcomes include tobacco use behaviors. Secondary outcomes include cigarette sales, retailer compliance, and the tobacco industry’s response to a menthol ban. Data on tobacco use behavior after a menthol ban were pooled using random-effects models. Two pairs of reviewers independently extracted data and assessed study quality.

Of the 964 articles that were identified during the initial search, 78 were included in the review and 16 were included in the meta-analysis. Cessation rates among menthol cigarette smokers were high after a menthol ban. Pooled results show that 24% (95% confidence interval [95% CI]: 20%, 28%) of menthol cigarette smokers quit smoking after a menthol ban, 50% (95% CI: 31%, 68%) switched to non-menthol cigarettes, 12% (95% CI: 3%, 20%) switched to other flavored tobacco products, and 24% (95% CI: 17%, 31%) continued smoking menthol cigarettes. Hypothesized quitting and switching rates were fairly close to real-world rates. Studies found the tobacco industry attempts to undermine menthol bans. National menthol bans appear more effective than local or state menthol bans.

Menthol cigarette bans promote smoking cessation suggesting their potential to improve public health.

Findings from this review suggest that menthol cigarette bans promote smoking cessation among menthol cigarette smokers and have the potential to improve public health.

  • tobacco use
  • menthol cigarettes
  • flavored tobacco products

Email alerts

Citing articles via.

  • About Nicotine & Tobacco Research
  • Recommend to your Library

Affiliations

  • Online ISSN 1469-994X
  • Copyright © 2024 Society for Research on Nicotine and Tobacco
  • About Oxford Academic
  • Publish journals with us
  • University press partners
  • What we publish
  • New features  
  • Open access
  • Institutional account management
  • Rights and permissions
  • Get help with access
  • Accessibility
  • Advertising
  • Media enquiries
  • Oxford University Press
  • Oxford Languages
  • University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

  • Copyright © 2024 Oxford University Press
  • Cookie settings
  • Cookie policy
  • Privacy policy
  • Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Development and application of emotion recognition technology — a systematic literature review

  • Systematic Review
  • Open access
  • Published: 24 February 2024
  • Volume 12 , article number  95 , ( 2024 )

Cite this article

You have full access to this open access article

  • Runfang Guo 1 , 2 ,
  • Hongfei Guo 4 ,
  • Liwen Wang 2 ,
  • Mengmeng Chen 3 ,
  • Dong Yang 2 &
  • Bin Li 1 , 2  

There is a mutual influence between emotions and diseases. Thus, the subject of emotions has gained increasing attention.

The primary objective of this study was to conduct a comprehensive review of the developments in emotion recognition technology over the past decade. This review aimed to gain insights into the trends and real-world effects of emotion recognition technology by examining its practical applications in different settings, including hospitals and home environments.

This study followed the Preferred Reporting Items for Systematic Reviews (PRISMA) guidelines and included a search of 4 electronic databases, namely, PubMed, Web of Science, Google Scholar and IEEE Xplore, to identify eligible studies published between 2013 and 2023. The quality of the studies was assessed using the Critical Appraisal Skills Programme (CASP) criteria. The key information from the studies, including the study populations, application scenarios, and technological methods employed, was summarized and analyzed.

In a systematic literature review of the 44 studies that we analyzed the development and impact of emotion recognition technology in the field of medicine from three distinct perspectives: “application scenarios,” “techniques of multiple modalities,” and “clinical applications.” The following three impacts were identified: (i) The advancement of emotion recognition technology has facilitated remote emotion recognition and treatment in hospital and home environments by healthcare professionals. (ii) There has been a shift from traditional subjective emotion assessment methods to multimodal emotion recognition methods that are grounded in objective physiological signals. This technological progress is expected to enhance the accuracy of medical diagnosis. (iii) The evolving relationship between emotions and disease throughout diagnosis, intervention, and treatment processes holds clinical significance for real-time emotion monitoring.

These findings indicate that the integration of emotion recognition technology with intelligent devices has led to the development of application systems and models, which provide technological support for the recognition of and interventions for emotions. However, the continuous recognition of emotional changes in dynamic or complex environments will be a focal point of future research.

Introduction

Emotional expression plays a crucial role in human life and work. The earliest definition of “emotion” appeared in the writings of William James (1884), the founder of American psychology. He believed that emotions are sensations of physical change and that any emotion is inevitably accompanied by physiological changes, such as facial expressions, muscle tension, and visceral activity [ 1 ]. Similarly, Danish physiologist Lange (1885) presented a similar viewpoint: emotions are not only physiological states that integrate sensations, thoughts, and behaviors but also psychological responses generated by various external stimuli [ 2 ]. As a result, researchers in numerous fields have recognized the importance of accurately identifying emotions. In recent years, research on emotion recognition has been applied predominantly in fields such as psychology, affective computing, and clinical therapy.

According to the World Health Organization (WHO), approximately 280 million people worldwide experience depression, with more than 700,000 people dying from suicide [ 3 ]. There are many mood-related disorders, such as bipolar disorder (BD), which is characterized by recurrent episodes of alternating mania and depressive symptoms [ 4 , 5 ]. The manic and pathological states of BD can also be understood as extreme expressions of basic emotions such as sadness, happiness, and disgust. Emotions may be intentionally or unintentionally suppressed, and many individuals might struggle to differentiate between fear and anxiety and between guilt and shame, making it challenging for them to accurately describe complex emotions. Patients with mood-related disorders experience more severe emotional fluctuations than healthy individuals [ 6 ], which can, to some extent, reflect the progression of the disease, the risk of relapse, and impaired functioning [ 7 , 8 ]. Therefore, the continuous monitoring of emotional instability and other variables that may reflect disease activity (such as symptom duration, severity, and frequency) has clinical significance.

Self-monitoring is ubiquitous in the field of psychiatry research. Humans can describe emotions through text, language, or facial expressions and even reflect internal emotions through physiological signals. Emotional charting tools, such as the National Institute of Mental Health’s Life Chart Method (NIMH-LCM) [ 9 ], the Symptom Checklist-90-Revised (SCL-90-R) [ 10 ], and the Profile of Mood States (POMS), are often used to manage and monitor emotional changes [ 11 ]. Due to the sudden spread of COVID-19 and drastic societal changes, emotions are highly susceptible to external influences and are closely related to behavior during the pandemic [ 12 ]. To reduce the transmission rate of the novel coronavirus, various personal protective measures and policies aimed at reducing gatherings may pose challenges in measuring emotions [ 13 ]. Therefore, simple methods such as voice information or facial expressions may no longer be suitable for emotion monitoring in psychiatry, and perhaps social media could serve as an important source of data [ 14 , 15 ]. Combining emotional data with mobile phone movement data and linking policies with human behavior can reveal the immense potential of multimodal data in emotion detection [ 16 ].

Currently, several intelligent monitoring tools can provide standardized responses to language or behavior and help individuals understand the emotions underlying specific actions [ 17 ]. Advancements in wearable devices, mobile terminals, and the Internet of Things (IoT) have provided more efficient multidimensional applications for intelligent emotional monitoring. These methods, which are based on ecological momentary assessment techniques, play an important role in reminding patients to perform self-monitoring [ 18 ]. The integration of momentary assessment and sensor data holds significant potential for clinical research and treatment. Sandstrom (2016), in the context of momentary depression and anxiety assessments, combined behavioral data from GPS, accelerometers, and anonymous call records to reveal clinically relevant psychological and behavioral patterns [ 19 ]. Effectively constructing an emotion classification model using neurophysiological, facial feature, and behavioral data recorded from portable devices, along with machine learning methods, showcases a novel research area.

This study reviews relevant literature from the past decade to delineate current trends and hotspots in emotion recognition technology, elaborating on its practical applications for patients with mental/physical disorders in both hospital and home environments. Emotional monitoring during patients’ diagnosis, intervention, and treatment has been demonstrated to have a certain effect on reducing morbidity and mortality and improving quality of life. Our goal was to assess the importance and practical application of emotion recognition technology in the treatment of patients with psychological/physical illnesses in the development of psychosomatic medicine. The remaining structure is as follows: Sect.  2 provides a detailed account of the process of collecting and selecting articles for this review. Section  3 provides an overview of emotion recognition methods applied in hospitals and home environments, along with an analysis of their development. Section  4 discusses the contributions of emotion recognition technology to patient treatment and healthcare, highlights the positive and negative impacts, and suggests potential future directions for this research. Finally, Sect.  5 offers a summary of the paper.

Material collection and research methods

Retrieval strategy.

A literature review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 20 ]. MeSH terms in Medline were searched. Three categories of keywords were preliminarily identified based on the research question, namely, emotion, recognition, and patients. Emotional-related MeSH terms such as psychology and mental; recognition-related MeSH terms such as express and survey; and patient-related MeSH terms such as clinical et al. were identified. Searches were conducted in the PubMed, Web of Science, Google Scholar, and IEEE Xplore databases using the Boolean operators “AND” and “OR” to combine keywords. Search data were recorded throughout the process. A review of the initially retrieved articles involved summarizing the index titles and keywords, conducting a secondary collection of free terms in each database, and organizing free terms. The search scope was expanded to obtain more precise or comprehensive results. Three researchers conducted a one-week discussion in July 2023 to finalize the research topic and retrieval strategy. Two trained researchers screened the relevance of article titles and abstracts to the research topic, with cross-checking by another reviewer. The first and corresponding authors performed final full-text reviews of included articles and submitted the results for collective team discussion. The literature search results are shown in Table  1 .

Eligibility criteria

Inclusion criteria.

Papers published in English only.

Research published in 2013–2023.

Studies in which the participants were patients with mental or physical disorders or eligible patient populations were extracted from publicly available databases.

Articles that proposed or developed at least one method, model, procedure or system for emotion monitoring.

Exclusion criteria

Duplicate articles were retrieved from different databases.

Abstracts, conference minutes and reports that could not be obtained by searching or contacting the authors.

Abstracts and original articles that were not related to the topic of the study.

Studies that focused on emotions exhibited by patients in response to external stimuli rather than emotions identified using a certain method or technology.

A manual search was conducted across four databases (see Fig.  1 ). A total of 3736 articles were identified, and their titles and abstracts were transferred to the reference management software EndNote 20. After duplicates were removed ( n  = 502), 3234 unique studies were identified and screened using the inclusion/exclusion criteria. The majority ( n  = 2334) of studies were excluded at the title and abstract screening stage, with an additional 622 excluded during full-text screening. The documents excluded for other reasons included abstracts for which the full text could not be obtained through a search or by contacting authors, conference proceedings, or reports ( n  = 6); studies focusing on patients’ emotional responses to external stimuli ( n  = 221); and low-quality outcome literature based on the CASP assessment ( n  = 7). Finally, 44 articles were selected for review.

figure 1

Flow chart of research screening

Article quality evaluation tool

The Critical Appraisal Skills Programme (CASP) criteria were adapted from the 1994 version of the “Users’ Guides to the Medical Literature” published by the American Medical Association. In this study, the CASP criteria were employed to assess the quality of the studies [ 21 ]. The CASP criteria comprise 10 items, each with three response options: “yes,” “no,” and “unclear.” The greater the number of “Yes” responses is, the greater the quality of the literature. Based on the assessment outcomes, the included literature was categorized into three levels according to quality: high, moderate, and low. To ensure the quality of the systematic review, studies with low appraisal results (i.e., with more than 3 “no” and “unclear” responses) were excluded. The detailed CASP evaluation results can be found in Appendix 1 .

Data extraction

Data were extracted independently by two researchers who were trained in data extraction, and the data were cross-checked by another researcher. Relevant data were manually extracted, including the first author’s name, year of publication, country of publication, title, DOI number, type of research, research method, purpose of the emotion recognition method, emotion recognition technology, data collection device, sample set, application scenarios, modeling foundation, psychological/emotional categories, statistical analysis method, and results.

The bias risks and types assessed in the individual studies included those proposed by the Cochrane Collaboration, such as selection bias, performance bias, detection bias, attrition bias, reporting bias, and other biases [ 22 ]. Any discrepancies or uncertainties related to bias assessment were resolved through discussions between the authors and relevant experts.

Among the 44 selected articles, 24 were experimental studies, 18 were observational studies, and 2 were mixed-methods studies. The data from 10 articles were sourced from public datasets, while the data from 33 articles were obtained through institutional recruitment. The patient populations discussed in these articles included individuals with mental disorders (BD, autism spectrum disorder, depression), neurological conditions (stroke, epilepsy, facial paralysis, facial numbness), cancer, and genetic alopecia.

The primary application scenarios addressed in the selected articles were hospital treatment and home healthcare. Emotional recognition methods predominantly involve the utilization of scales, speech analysis, facial features, physiological signals, or multimodal techniques to construct models and systems. Research has indicated that through clinical validation (diagnosis, intervention, and treatment), certain emotion monitoring devices demonstrated good performance in reducing morbidity and mortality rates and enhancing quality of life [ 23 , 24 ]. For a detailed overview, please refer to Fig.  2 .

figure 2

Overview of the application of emotion recognition methods

Application of emotion recognition methods based on different scenes

Twenty-seven studies focused on hospital applications, 11 studies were conducted in outpatient or home monitoring settings, and the remaining 6 studies indicated applicability across all scenarios.

Application of emotion recognition technology in hospitals

In previous medical practices, most doctors or experts diagnosed patients’ emotional issues by using invasive devices and medical assessments. Automatic emotion recognition methods assist doctors not only in evaluating the overall condition of patients but also in accurately identifying diseases associated with emotional features in real time. In some studies, clinical disease features and emotional characteristics were combined as unique biomarkers that are involved in the clinical diagnostic process. They have also been used to assess patients’ performance during treatment and to aid in implementing psychological intervention therapies [ 25 , 26 ].

In a study involving psychiatric patients, Masulli (2022) introduced a data-driven eye-tracking model [ 25 ]. The focus of this study was on the use of a cross-diagnostic approach to link clinical dimensional scores with eye gaze behavior. A study by Quirien (2022) suggested that the regular use of the European Organization for Research and Treatment of Cancer Core Quality of Life questionnaire (EORTC QLQ-C30) emotional function (EF) scale for screening anxiety and depression symptoms in glioma patients contributes to the early identification of emotional disorders. This practice serves as a foundation for referrals and treatment decisions [ 26 ]. Overall, these studies indicate that psychological care and interventions can enhance patients’ mental well-being within a clinical practice setting. Hence, achieving accurate and efficient emotion recognition and continuous monitoring is the initial step toward improving patients’ conditions.

Application of emotion recognition technology in home environments

In recent years, the development of the IoT has driven rapid advancements in the field of healthcare, leading clinical practitioners to focus on home-centered care models. Medical devices connected through the IoT offer users the opportunity to receive in-home treatment and rehabilitation, thereby alleviating pressure on healthcare systems. The application of emotional recognition methods in the “home health” domain has sparked significant interest among researchers. Faccio (2018) developed an electronic health tool based on the Family Resilience (FaRe) questionnaire aimed at monitoring the emotional state of cancer patients while at home [ 27 ]. This tool not only provides diagnostic criteria for physicians but also allows for the formulation of corresponding intervention measures. Veerbeek (2013) created a web-based psychological monitoring data collection system called “Monitoring the Mental Health of the Elderly” [ 28 ].

These systems exhibit the novelty of coordinated hardware and software. Participants complete questionnaires or scales on smart devices at scheduled times each day while their sleep and behavioral activity are continuously monitored through devices such as pulse oximeters, cameras (smart mobile devices, home surveillance devices, computers, etc.), and wearable devices equipped with sensors (e.g., tracking phone and text message usage, social interactions, and generated movement data). All these data are stored on cloud servers and fed to the backend in real time, facilitating easy access to medical information and monitoring services and ultimately reducing diagnosis time. Overall, the research indicates that the intelligent emotion recognition systems used in home environments must possess smart terminals and home treatment platforms.

Emotion recognition technology based on different patterns

Emotion recognition based on psychometric scales.

Fifteen articles assessed the effectiveness of the scales for emotion recognition, as shown in Table  2 .

Emotional chart tools can assist patients in understanding their medical condition, identifying warning signs of adverse emotional episodes and relapses, and describing the instability of individual emotions. Embedding these scales in monitoring systems and applications based on momentary assessment tools can compensate for the limitations of traditional paper-based emotional chart tools due to environmental constraints. Moreover, retrospective reporting eliminates the impact of inaccurate assessment results caused by factors such as measurement outcomes, cognitive levels, and understanding errors, for example, low compliance and potential recall bias [ 29 , 30 , 31 ].

Tsanas (2016) [ 32 ] indicated that the Automated Monitoring of Symptom Severity (AMoSS) application system, which was embedded in smartphones and based on the mood zoom scale, enabled efficient, long-term, and effective daily emotional monitoring for patients with mood disorders. Throughout the entire process, participants exhibited good compliance, and the data were quantitatively processed and more easily preserved.

Emotion recognition based on speech

Three articles assessed the effectiveness of speech-based emotion recognition, as shown in Table  3 .

Mel-frequency cepstral coefficients (MFCCs) have been widely used in speech-based emotion recognition [ 33 , 34 , 35 , 36 , 37 ]. Several studies have shown that support vector machine (SVM) classifiers can group multidimensional datasets by identifying hyperplanes. Chin KC (2021) used the “MFCC + SVM” approach in their research, and the results showed that the prediction accuracy, positive predictive value, negative predictive value, sensitivity, and specificity were 92.87%, 84.62%, 93.57%, 52.38%, and 98.64%, respectively [ 38 ].

Furthermore, deep convolutional neural networks (DCNNs) group multidimensional datasets by recognizing data features through recursion and iteration. Rejaibi (2022) tested the Distress Analysis Interview Corpus/Wizard-of-Oz (DAIC-WOZ) database, Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) dataset, and Anonymized Videos from Diverse countries (AVi-D) dataset using the “MFCC + DCNN” framework, achieving an overall accuracy of 76.27% [ 39 ]. These consistent results demonstrate that speech-based emotion recognition technology has also become an independent and viable application.

Emotion recognition based on facial expression

Twelve articles evaluated the effectiveness of facial expression-based emotion recognition, and the results are shown in Table  4 .

Facial expression-based emotion recognition technology utilizes computer vision and artificial intelligence to identify a person’s psychological emotions [ 40 ]. Rapid and subtle microexpressions are among the most useful external indicators for detecting hidden emotional changes. Ekman annotated static and dynamic expression in microexpression videos within the Facial Action Coding System (FACS) [ 41 ] (related datasets include the Facial Expression Recognition 2013 (FER 2013) dataset [ 42 ] and the Real-world Affective Faces Database (RAF-DB) [ 43 ]).

Convolutional neural networks (CNNs) have the ability to rapidly capture changes in facial position and image scale, and they have made significant advancements in pattern recognition, particularly in tasks such as facial detection [ 44 ] and text recognition [ 45 ]. The visual transformer (ViT) is a powerful artificial intelligence technology capable of recognizing or classifying objects within images [ 46 , 47 ]. As the algorithmic performance of ViT has continued to improve and advance, it has gradually outperformed CNNs on small- and medium-sized image classification datasets [ 41 , 48 ]. Jiayu Ye (2022) proposed a depression vision transformer (Dep-ViT) model to address the facial expression recognition problem in patients with depression. Compared to four other excellent models (the deep-emotion, ResNet, SCN, and ViT models), the Dep-ViT model achieved the highest accuracy [ 49 ].

Emotion recognition based on physiological signals

Three articles assessed the effectiveness of emotion recognition based on physiological signals, and the results are shown in Table  5 .

Physiological signals can provide a relatively objective reflection of an individual’s emotional state, increasing the accuracy of emotion recognition systems based on physiological signals. These physiological signals include galvanic skin response (GSR) signals, electromyographic (EMG) signals, electroencephalogram (EEG) signals, heart rate, and respiration, among others. In their research, Verma Aakash (2018) developed an emotion recognition wearable system based on Arduino for individuals with behavioral disorders [ 50 ]. This system measures skin conductivity using a GSR sensor and skin transparency using a pulse sensor and provides real-time heart rate data.

Gentili’s (2016) research indicated that combining physiological parameters with behavioral data allows for more accurate identification of subtle emotional changes [ 51 ]. Compared to changes in speech and facial expressions, the rhythmic variations in behavioral data are more representative. At present, the available physiological signal data are limited, and it is necessary to establish a complete and high-quality physiological signal database and to explore emotion models based on cognitive mechanisms combined with physiological signals.

Emotion recognition based on multimodality

Ten articles assessed the effectiveness of multimodal emotion recognition, and the results are shown in Table  6 .

In previous research, most emotion recognition technologies relied primarily on single modalities and lacked multiple-dimensional parameters. An increasing number of studies are developing more comprehensive and optimized emotion recognition systems by incorporating various forms of data, such as psychometric questionnaire, audio signal, facial expression, EEG, and electrocardiogram (ECG) data. Hossain (2016) achieved a high recognition rate of 99.4% in a patient emotion recognition system based on a Gaussian mixture model (GMM) by combining facial expressions and audio signals [ 52 ]. In Yuying Tong’s (2020) work, a method that combines EEG and facial expression features to identify the emotions of patients with depression was proposed. This research validated the effectiveness of facial expression classification for different emotions in patients with depression and showed significant accuracy through repeated measurements [ 53 ].

Different clinical applications of emotion recognition

Emotional recognition technology has various applications in the clinical field, positively impacting clinical research and leading to precise diagnoses, interventions, and treatments, with the potential to enhance patients’ mental health and treatment outcomes. The results are summarized in Table  7 .

Modern healthcare and nursing prioritize not only fundamental medical treatment but also psychological therapy. Clinical practitioners and healthcare professionals utilize extensive emotional monitoring data to facilitate their understanding of clinical outcomes. Research indicates that rapid psychological diagnostic results can be obtained through smart applications and instantaneous assessment techniques [ 54 , 55 ]. This not only addresses the challenges faced by patients who must travel long distances for medical consultations but also streamlines the medical consultation process.

Clinical doctors can use emotional monitoring data to formulate coping strategies and relapse prevention plans. Research has shown that through self-monitoring and labeling emotional behaviors, patients can gain a better understanding of their emotions and take measures to prevent more severe emotional issues, thereby improving their mental health outcomes [ 56 ].

In the past decade, the field of intelligent emotion recognition has attracted the interest of numerous researchers, leading to the development of various methods based on single or multimodal approaches to effectively identify patients’ emotional states. The recognition of patients’ emotions plays a crucial role in healthcare, including in psychological counseling [ 57 ], anxiety and stress assessments [ 28 ], and pain assessments [ 52 ].

Evaluation of the application of emotion recognition methods

A comprehensive intelligent healthcare system enables patients to receive real-time condition monitoring, timely diagnosis and effective treatment. Through intelligent devices based on cloud computing and the IoT, patients’ emotions can be rapidly and accurately identified, with notifications sent to healthcare professionals to ensure patient safety. In addition, cloud data centers can provide data storage services, data analysis, and audiovisual data processing. These patients offer secure access to healthcare professionals when they need to evaluate patients’ emotional states [ 58 ]. Emotion recognition has evolved from initially targeting patients with mental disorders (such as depression and BD) to encompassing patients with neurological conditions (such as cerebrovascular diseases, peripheral neuropathies, and spinal cord lesions). The most extensively studied applications of emotion recognition in these patients are among patients with conditions such as epilepsy, stroke, facial paralysis, facial numbness, and coma. The common feature of such patients is that they cannot express real emotions through objective external features (such as language and facial expressions) and autonomous behaviors. Therefore, it is necessary to design an automated system to effectively detect the emotions of such patients.

Methods based on neural networks and facial features have shown good performance in recognizing the emotions of patients with facial paralysis and are highly valuable in the medical field [ 59 ]. Furthermore, it is important for healthcare professionals to consider disease severity, as the extent of organ damage can affect the ability to recognize emotions and feelings. Researchers strive to ensure that any recognition system can identify these behaviors effectively. In addition to identifying basic emotions (anger, disgust, fear, happiness, sadness, and surprise), it is important to consider the intensity of these emotions. This understanding can help healthcare professionals anticipate patients’ concerns and stress levels, facilitating appropriate treatment. EEG research has indicated that patients with depression exhibit hemispheric asymmetry in brain signals, and their EEGs show regular variations [ 53 ]. Continuous emotional monitoring can provide insights into the patterns of emotional fluctuations in patients, and comprehensive psychological interventions may be beneficial for the recovery of patients with depression. Emotional recognition systems based on cloud computing and the IoT can, to some extent, address the following four major healthcare issues for patients with emotional disturbances: the shortage of healthcare professionals, long outpatient waiting times, the inability to detect changes in patient emotions early, and the increase in additional treatment costs. Consequently, these systems can support higher-quality healthcare services, thereby enhancing patient care and treatment experiences.

Evaluation of emotion recognition technology

Positive and negative effects of emotion recognition technology.

The use of emotion recognition technology in healthcare offers numerous advantages. First, this technology provides a quick and convenient method for conducting emotional tests through smart devices, eliminating the delays associated with traditional paper questionnaires and increasing user compliance. Second, it enables continuous monitoring of emotional states, aiding in disease understanding and the identification of factors affecting emotions and early warnings of disease progression or relapse, thereby enhancing patient treatment and quality of life. Additionally, patients can provide timely feedback without treatment interruption, helping healthcare professionals gain a timelier understanding of their conditions and offer necessary support. Furthermore, this technology automates data storage and processing, making it easier for healthcare professionals to access and analyze patient emotion information, thereby enhancing treatment personalization. Finally, incorporating multimedia elements into emotion tests improves user engagement, ultimately enhancing the user experience and increasing participation and compliance rates.

However, there are notable concerns associated with the use of emotion recognition technology in healthcare. First, long-term emotional monitoring can put pressure on patients, especially when they are required to complete daily emotional questionnaires at specific times, potentially affecting their participation and willingness to cooperate. Second, privacy concerns loom large as patients worry that the technology could compromise their personal privacy, particularly when it relates to emotional and mental health issues, leading some patients to adopt a cautious approach and withhold information regarding their true emotional states. Additionally, an excessive range of features and options in emotion recognition applications may overwhelm patients, diverting their attention and hindering their ability to focus on the primary goal of emotional monitoring, thereby diminishing the effectiveness of these applications. These concerns necessitate careful consideration of patient well-being and privacy in the implementation of this technology.

Limitations of emotion recognition technology

The implementation of emotion recognition technology in healthcare involves several challenges. First, data access is a critical issue, as the early stages of technology development demand a substantial amount of data for training predictive and decision models. While public databases are widely used by researchers, common problems with research datasets, such as data imbalance and limited dataset size, can lead to disparities between the data used for training and experimentation. Second, cost is a significant consideration. While medical technology aims to reduce costs, the mining, storage, and analysis of data, along with human resource and hardware utilization, can be financially burdensome. Third, cultural differences pose challenges. Older individuals may lack access to or be unwilling to use smart devices, and participants’ engagement with online monitoring systems may vary in terms of time and extent. Additionally, differences in education levels may impact the quality of the data generated and necessitate validation efforts. Finally, there is a notable lack of consensus on security, ethics, and privacy concerns in this context, further complicating the implementation of emotion recognition technology in healthcare. Addressing these challenges is essential for harnessing the full potential of this technology while ensuring patient privacy, data quality, and cost-effectiveness.

Future development direction

In healthcare systems and health services, automatic emotion recognition technology is already being used to monitor the conditions of patients with mental health disorders. However, the future development of this technology will not only focus on psychological conditions such as depression and anxiety but also expand to monitor the severity of diseases and conditions such as cognitive impairment.

To advance emotion recognition technology, we need to overcome the limitations of currently available methods, which primarily involve the combination of questionnaires, speech analysis, facial expressions, and physiological signals. Instead, we should consider integrating a broader range of modalities to achieve more precise emotion recognition. This innovation might include incorporating data from other sensory inputs, such as touch and taste, as well as textual and image data from social media. Furthermore, as artificial intelligence and machine learning continue to advance, emotion recognition technology should move toward automation and real-time capabilities. This shift will aid in providing more personalized and immediate healthcare services, assisting patients in better managing their emotional well-being. Finally, issues related to security, ethics, and privacy remain areas that require further research and attention. It is essential to ensure that the development of emotion recognition technology complies with ethical and legal requirements while safeguarding patient privacy and data security.

Limitations

The limitations of this study mainly lie in the review process and the assessment criteria. During the review process, our study scope may have been constrained by the limitations of the search strategy used during the literature retrieval. Although we made efforts to cover as wide a range of literature as possible, there may still be cases where some relevant studies were overlooked. We were limited to four databases and manually searched English-language literature published in the past decade to observe and evaluate the latest international research results on this topic. However, we cannot determine whether research conducted before this time frame or in other languages or databases might contain more recent research findings. During the evaluation process, it was noted that some of the selected studies lacked sufficient detail or robustness in terms of system performance. We acknowledge that such studies may lack of representative significance. From the perspective of reviewers, the attractiveness of research methods and the novelty of performance sometimes take precedence. Although the extensive heterogeneity of the results prevented us from conducting a meta-analysis, we were able to synthesize data from many studies using a comprehensive approach with robust analytical processes, encompassing a range of different study designs. Furthermore, the included studies were assessed by the reviewers as moderate to high quality, which strengthens the conclusions that can be drawn from the synthesized results. In summary, we took measures to ensure that our search strategy was as robust as possible.

This study elaborated on the potential role of emotions in disease diagnosis and treatment. Emotional recognition technology based on intelligent devices and models can support the design and implementation of emotion recognition and intervention measures. By collecting patients’ physiological signals through intelligent devices and conducting real-time analysis with emotion recognition models, healthcare professionals can better understand patients’ psychological states, guiding the formulation of diagnosis and treatment plans. Real-time monitoring of patient emotions can also serve as an indicator for assessing treatment efficacy, providing a reference for optimizing and adjusting treatment plans and thereby improving patient satisfaction and recovery rates. Most studies were conducted when patients were in a static state and had sufficient time for testing. In dynamic or complex environments, continuous emotion recognition technology for addressing emotional changes still requires further research and improvement. This includes but is not limited to the following aspects. First, it is necessary to improve the robustness of emotion recognition models so that they can effectively recognize emotions in complex environments, such as noise interference and motion interference. Second, it is necessary to further explore and develop emotion recognition methods based on multimodal data that combine multiple information sources, such as physiological signals, speech, and body movements, to improve the accuracy and reliability of emotion recognition. This is an important area for future development.

Data availability

No datasets were generated or analysed during the current study.

James W. What is an emotion. Mind, 1884,9(34):188–205.

Lange CG. The Emotions: a psycho physiological study. Emotions, 1885,33–90.

Maithri M, Raghavendra U, Gudigar A, et al. Automated emotion recognition: current trends and future perspectives. Comput Methods Programs Biomed. 2022;215:106646.

Article   CAS   PubMed   Google Scholar  

Goodwin F, Jamison KR. Manic-depressive illness: bipolar disorders and recurrent depression. Ecol. 2012.

Kupka RW, Knoppert-van der Klein E, Nolen WA. Handboek bipolaire stoornissen. Uitgeverij De Tijdstroom; 2008.

Bonsall MB, Wallace-Hadrill SM, Geddes JR, et al. Nonlinear time-series approaches in characterizing mood stability and mood instability in bipolar disorder. Proc Biol Sci. 2012;279(1730):916–24.

CAS   PubMed   Google Scholar  

Judd LL, Schettler PJ, Akiskal HS, et al. Long-term symptomatic status of bipolar I vs. bipolar II disorders. Neuropsychopharmacol. 2003;6(2):127–37.

Google Scholar  

Strejilevich SA, Martino DJ, Murru A, et al. Mood instability and functional recovery in bipolar disorders. Acta Psychiatr Scand. 2013;128(3):194–202.

Leverich GS, Nolen WA, Rush AJ, et al. The stanley foundation bipolar treatment outcome network. I. Longitudinal methodology. Affect Disord. 2001;67(1–3):33–44.

Article   CAS   Google Scholar  

Schmitz N, Hartkamp N, Franke GH. Assessing clinically significant change: application to the SCL-90-R. Psychol Rep. 2000;86(1):263–74.

McNair DM, Lorr M, Droppleman LF. Profile of mood states manual. San Diego: Educational and industrial testing service; 1971.

Metzler H, Rimé B, Pellert M, Niederkrotenthaler T, Di Natale A, Garcia D. Collective emotions during the COVID-19 outbreak. Emot Aprile. 2023;23(3):844–58.

Boccaccio FM, Gubitosa S, Pirrone C. Exploring the interplay between Facial Expression Recognition and Emotional Intelligence in adults: implications for Novel Learning methods. Educ Sci. 2023;13(11):1112.

Article   Google Scholar  

Hale T, Angrist N, Goldszmidt R, et al. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat Hum Behav. 2021;5:529–38.

Article   PubMed   Google Scholar  

Pellert M, Schweighofer S, Garcia D. The individual dynamics of affective expression on social media. EPJ Data Sci. 2020;9:1.

Oliver N, et al. Mobile phone data for informing public health actions across the COVID-19 pandemic life cycle. Sci Adv. 2020;6:eabc0764.

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Amico F, Healy G, Arvaneh M, et al. Multimodal validation of facial expression detection software for real-time monitoring of affect in patients with suicidal intent. Eur Psychiatry. 2016;33(S1):596–S596.

Sandstrom GM, Lathia N, Mascolo C, et al. Opportunities for smartphones in clinical care: the future of mobile mood monitoring. Clin Psychiatry. 2016;77(2):135–7.

Page MJ, Moher D, Bossuyt PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160.

Article   PubMed   PubMed Central   Google Scholar  

BMJ (OPEN ACCESS), Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.

Critical appraisal skills programme. (2022). CASP (Systematic Review) Checklist. https://casp-uk.net/referencing/ .

Higgins J, Green S. Cochrane handbook for systematic reviews of interventions version. The cochrane collaboration; 2011.

Roger S, McIntyre L, Bloudek JY, Timmons. Patrick Gillard & Amanda Harrington. Total healthcare cost savings through improved bipolar I disorder identification using the Rapid Mood Screener in patients diagnosed with major depressive disorder. Curr Med Res Opin. 2023;39(4):605–11.

Rocamora R, Chavarría B, Pérez E, Pérez-Enríquez C, Barguilla A, Panadés-de Oliveira L et al. Mood disturbances, anxiety, and impact on quality of life in patients admitted to Epilepsy Monitoring units. Frontiers in Neurology, 2021,12.

Masulli P, Galazka M, Eberhard D, et al. Data-driven analysis of gaze patterns in face perception: methodological and clinical contributions. Cortex. 2022;147:9–23.

Oort Q, Zwinkels H, Koekkoek JAF, et al. Is the EORTC QLQ-C30 emotional functioning scale appropriate as an initial screening measure to identify brain tumour patients who may possibly have a mood disorder? Psychooncology. 2022;31(6):995–1002.

Faccio F et al. Development of an eHealth tool for cancer patients: monitoring psycho-emotional aspects with the Family Resilience (FaRe) Questionnaire. ecancermedicalscience 12 (2018).

Veerbeek M, Oude Voshaar R, Depla M, et al. Mental health care monitor older adults (MEMO): monitoring patient characteristics and outcome in Dutch mental health services for older adults. Methods Psychiatr Res. 2013;22(2):100–9.

Kobak KA, Greist JH, Jefferson JW, et al. New technologies to improve clinical trials. Clin Psychopharmacol. 2001;21(3):255–6.

Stone AA, Shiffman S, Schwartz JE, et al. Patient compliance with paper and electronic diaries. Control Clin Trials. 2003;24(2):182–99.

Conner TS, Barrett LF. Trends in ambulatory self-report: the role of momentary experience in psychosomatic medicine. Psychosom Med. 2012;74(4):327–37.

Tsanas A, Saunders KE, Bilderbeck AC, et al. Daily longitudinal self-monitoring of mood variability in bipolar disorder and borderline personality disorder. Affect Disord. 2016;205:225–33.

Koolagudi SG, Rao KS. Emotion recognition from speech: a review. Speech Technol. 2012;15(2):99–117.

Pao TL, Chen YT, Yeh JH. Emotion recognition from Mandarin speech signals. 2004 International symposium on Chinese spoken language processing. IEEE, 2004, p.301–4. (IEEE Cat. No.04EX877).

Zhu L, Chen L, Zhao D, et al. Emotion recognition from Chinese speech for smart affective services using a combination of SVM and DBN. Sens (Basel). 2017;17(7):1694.

Article   ADS   Google Scholar  

Likitha MS, Gupta SRR, Hasitha K, Raju AU. Speech based human emotion recognition using MFCC. Institute of electrical and electronics engineers. 2018, p. 2257–60.

Wang Y, Hu W. Speech emotion recognition based on improved MFCC. In Proceedings of the 2nd international conference on computer science and application engineering. 2018 Oct 22 (pp. 1–7).

Chin KC, Hsieh TC, Chiang WC, et al. Early recognition of a caller’s emotion in out-of-hospital cardiac arrest dispatching: an artificial intelligence approach. Resuscitation. 2021;167:144–50.

Emna Rejaibi A, Komaty F, Meriaudeau, et al. MFCC-based recurrent neural network for automatic clinical depression recognition and assessment from speech. Biomed Signal Process Control. 2022;71:103–7.

Yang B, Cao J, Ni, Zhang R. Facial expression recognition using weighted mixture deep neural network based on double-channel facial images. IEEE, 2017.

Ekman Paul and Wallace V. Friesen. Facial action coding system. Environmental psychology & Nonverbal behavior; 1978.

Goodfellow IJ, Erhan D, Luc Carrier P, et al. Challenges in representation learning: a report on three machine learning contests. Neural Netw. 2015;64:59–63.

Li S, Deng W, JunPing, Du. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017,pp:21–26.

Li H et al. A convolutional neural network cascade for face detection. Proceedings of the IEEE conference on computer vision and pattern recognition. 2015,pp:5325– 5334.

Zhang J, et al. Watch attend and parse: an end-to-end neural network based approach to handwritten mathematical expression recognition. Pattern Recognit. 2017;71:196–206.

Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Adv Neural Inform Process Syst 30. 2017. arXiv:1706.03762.

Aouayeb M, Hamidouche W, Soladie C et al. Learning Vision Transformer with Squeeze and Excitation for Facial Expression Recognition. arXiv. 2021, arXiv:2107.03107.

Dosovitskiy A, Beyer L, Kolesnikov A et al. An image is worth 16x16 words: transformers for image recognition at scale. arXiv. 2020, arXiv:2010.11929.

Ye J, Fu G, Liu Y et al. Dep-ViT: uncertainty suppression model based on facial expression recognition in depression ptients. Artif Neural Networks Mach Learn. 2022, pp:113–24.

Verma A, Dogra A, Malik K, Talwar M. Emotion recognition system for patients with behavioral disorders. Adv Intell Syst Comput. 2018;624:139–45.

Gentili C, Valenza G, Nardelli M, et al. Longitudinal monitoring of heartbeat dynamics predicts mood changes in bipolar patients: a pilot study. Affect Disord. 2017;209:30–8.

Hossain MS. Patient state recognition system for healthcare using speech and facial expressions. Med Syst. 2016;40(12):272.

Tong Y, Zhao G, Zhao J, et al. Biases of happy faces in face classification processing of depression in Chinese patients. Neural Plast. 2020;2020:7235734.

Bai R, Xiao L, Guo Y, et al. Tracking and monitoring mood stability of patients with major depressive disorder by machine learning models using passive digital data: prospective naturalistic multicenter study. JMIR mHealth uHealth. 2021;9(6):e30540.

Dubad M, Elahi F, Marwaha S. The clinical impacts of mobile mood-monitoring in young people with mental health problems: the MeMO study. Front Psychiatry. 2021;12:687270.

Gangeri L, Alfieri S, Greco M, et al. Construction process and development stages of pandemic emotions questionnaire in cancer patients (PEQ-CP). BMC Psychol. 2022;10(1):226.

Li Y, Wu B, Li X, et al. Research on mental stress recognition of depressive disorders in patients with androgenic alopecia based on machine learning and fuzzy K-Means clustering. Front Genet. 2021;12:751791.

Li Y, Dai W, Ming Z, Qiu M. Privacy protection for preventing data over collection in smart city. Comput. 2016;65(5):1339–50.

MathSciNet   Google Scholar  

Parra-Dominguez GS, Sanchez-Yanez RE, Garcia-Capulin CH. Towards facial gesture recognition in photographs of patients with facial palsy. Healthc (Basel). 2022;10(4):659.

Download references

Acknowledgements

Not applicable.

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Anhui Provincial University Natural Science Key Project (No. 2022AH051458). The University Synergy Innovation Program of Anhui Province (No. GXXT-2022-058) provided us with language polishing.

Author information

Authors and affiliations.

The First Affiliated Hospital of Bengbu Medical University, Bengbu Medical University, 287 Changhuai Road, Bengbu, China

Runfang Guo & Bin Li

School of Public Health, Bengbu Medical University, Bengbu, China

Runfang Guo, Liwen Wang, Dong Yang & Bin Li

School of Health Management, Bengbu Medical University, Bengbu, China

Mengmeng Chen

School of Humanities, Southeast University, Nanjing, China

Hongfei Guo

You can also search for this author in PubMed   Google Scholar

Contributions

Runfang Guo and Bin Li planned the paper, structured the article and contributed to the core research ideas. Bin Li provided valuable feedback and helped in article preparation. Hongfei Guo, Liwen Wang, Mengmeng Chen and Dong Yang discussed the research problems and put forward suggestions. All authors have reviewed the manuscript and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Bin Li .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors state that they have no conflicts of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Guo, R., Guo, H., Wang, L. et al. Development and application of emotion recognition technology — a systematic literature review. BMC Psychol 12 , 95 (2024). https://doi.org/10.1186/s40359-024-01581-4

Download citation

Received : 05 December 2023

Accepted : 11 February 2024

Published : 24 February 2024

DOI : https://doi.org/10.1186/s40359-024-01581-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Recognition
  • Medical treatment
  • Find a journal
  • Publish with us
  • Track your research
  • Open access
  • Published: 19 February 2024

Developing a psychological care competences framework for nurses in China: a mixed methods study

  • Qinghong Fang 1   na1 ,
  • Xingwen Li 1   na1 ,
  • Yuanyuan Luo 1 ,
  • Zhihui Yang 1 ,
  • Lin Xiao 1 ,
  • Wenxuan Tan 1 ,
  • Suting Liu 1 ,
  • Jiahui Luo 2 &
  • Lili Zhang 1  

BMC Nursing volume  23 , Article number:  129 ( 2024 ) Cite this article

165 Accesses

Metrics details

With social transformation, rapid economic development and deepening awareness of psychological health in China, people’s demand for psychological health services is becoming increasingly urgent. A key challenge for Chinese medical organizations is to train enough qualified psychological care nurses. A greater understanding of psychological care competences (PCC) can help in clinical nurse selection, training, and assessment.

To develop a PCC framework for Chinese nurses and obtain a consensus on the framework among experts.

A descriptive mixed methods study was designed consisting of a literature review and semi-structured interviews followed by three Delphi rounds. The experts ( n  = 16) involved were nurses, nursing managers and educators from nine Chinese provinces with a specific interest in psychological care. Descriptive statistics assisted in data analysis.

Using the Iceberg Model as a theoretical foundation, five main dimensions and associated subdomains were integrated from 39 chosen articles. The semi-structured interviews with 24 nursing managers and nurses confirmed all of the themes from the literature review while generating new themes, both of which were incorporated into the initial PCC framework. After three Delphi rounds, the experts reached consensus on the PCC framework, including five domains (knowledge, skills, professional ethics, personal traits, internal motivations) and 22 subdomains with connotations. The response rate (RR) values for the three rounds of consultation were 80.00%, 87.50% and 92.86%, the composite reliability (Cr) values were 0.89–0.90, and the Kendall coordination coefficients were 0.155-0.200 ( P  < 0.05).

Conclusions

On the basis of the Iceberg Model, literature review and qualitative research methods along with Delphi technique were used to develop a scientific and systematic PCC framework. The research methods were feasible and the results were reliable, thereby providing a basis for adopting this framework into nursing education. A formal assessment tool should be developed to test the PCC of nurses in clinical practice.

Peer Review reports

Psychological problems have become a significant healthcare challenge. Approximately 300 million people worldwide experience psychological problems [ 1 ]. In China, 173 million people have mental health problems, accounting for 17.5% of domestic adults [ 2 ]. Compared to nonpatients, general hospital inpatients were reported to have significantly higher levels of anxiety and depression [ 3 ]. Approximately half of all patients admitted to general hospitals have a primary diagnosis of psychosomatic illness, while half of the remaining 50% have secondary or mixed psychological problems [ 4 ]. More worryingly, patients with physical health multimorbidity may have an elevated risk of suicidal behavior [ 5 , 6 ]. Statistics show that there were 3.26 suicides per 100,000 nonpsychiatric inpatients in 48 Chinese general hospitals between 2015 and 2017 [ 7 ]. To handle psychological problems, domestic and foreign calls have been made to take mental health promotion actions as one of the main tasks and intervene in health influencing factors comprehensively [ 8 , 9 ].

Psychological care, as a key component of holistic care, favorably enhances patients’ health outcomes [ 10 , 11 ]. As the largest cohort of professionals that cares for patients and spends the most time with patients and families, nurses play a significant role in psychological care [ 12 ]. When patients exhibited psychiatric and psychological symptoms, their specific psychological condition was documented in nursing records, providing clinicians with prompt reminders [ 13 ]. What’s more, nurse-lead, interdisciplinary medical staff could give patients with timely and effective psychological care [ 13 ], as well as help refer patients to psychiatrists if necessary to meet their psychological requirements. Zhang et al [ 14 ] also emphasized that some patients may suffer from a higher risk of self-injury or suicidal behavior if not timely nursing intervened. Beyond that, nurse-facilitated psychological care may strengthen the lines of defense and resistance against the stressors faced by the patients throughout sickness and promote their psychological recovery [ 15 ]. However, Chinese nonpsychiatric nurses now have inadequate psychological care competence compared to psychiatric nurses, resulting in high rates of underdiagnosis of patients’ psychological problems [ 16 ]. In addition, some Chinese nurses lack psychological care knowledge and skills, which makes it difficult for them to independently solve psychological problems [ 17 ]. For the above reasons, the Chinese psychological care competence of nonpsychiatric nurses needs to be improved.

The nursing competence framework is used as a reference for promoting clinical nursing development and orienting professional practice standards [ 18 ]. Psychological care competences (PCC) are multidimensional, including knowledge, skills, self-concept, motivation, and characteristics [ 19 ]. Many countries have established professional PCC frameworks. King et al. [ 19 ] identified the existing competence sets related to disaster mental health through a literature review. Carroll et al. [ 20 ] explored midwives’ competency in perinatal mental health by exploratory descriptive study. Waite et al. [ 21 ] reviewed the definition of structural competence in mental health care. Because competence frameworks are intended for practice in specific contexts, direct adaptation of PCC frameworks from other countries in China may be inappropriate due to cultural and healthcare system differences. Hence, it is necessary to develop a PCC framework to provide comprehensive guidance for Chinese nonpsychiatric nurses.

For the development of a competence set, researchers commonly use methods and theories. It is typical to seek expert perspectives to identify a consensus stance because expert opinions are thought to be more reliable than others [ 22 ]. The Delphi methodology is one way to build a competence framework, utilizing anonymous sequential questionnaires to achieve a consensus among ‘experts’ [ 23 ]. The exchange of opinions across several rounds promotes Delphi consensus [ 24 ]. This collective review by a representative group is appropriate for competence framework development [ 23 ]. Meanwhile, the Iceberg Model, proposed by the American psychologist Spencer [ 25 ], can be used to direct the construction of competency frameworks. Therefore, based on the Iceberg Model, a mixed-method approach was used in this study.

The aim of this study was to (1) develop a PCC framework for Chinese nurses and (2) obtain a consensus on the framework among experts.

This study was designed in two phases using a mixed-method approach that included literature review, qualitative study and Delphi technique, as shown in Fig.  1 . The first phase developed the initial psychological care competences (PCC) framework, in which PCC characteristics were extracted through literature review and qualitative interviews. Five competence domains and 22 sub-domains were identified. In the second phase, the framework was revised through Delphi rounds until a consensus was reached. This report was checked against the Conducting and REporting of DElphi Studies (CREDES) [ 26 ].

figure 1

Summary of research design in developing competence framework for psychological care

Research team

The team comprised eight members: one professor supervised the overall planning of the study, three associate professors were responsible for supervising the research process, and four nursing graduate students were in charge of researching the specific implementation procedure.

Phase 1: development of the initial PCC framework

Literature review.

The search strategy was developed collectively by two writers (QHF, XWL) who had received systematic review training, and the search terms were determined after discussion by the research team. The electronic databases PubMed, Embase, Cochrane Library, CINAHL, PsycTESTS, PsycArticles, PsycInfo, Psychology and Behavioral Sciences Collection, CNKI, Chinese Biomedical Literature Database, and Wanfang were searched from their earliest dates to October 2022. A mix of medical subject headings and free text terms of the following key concepts was used for the search strategy: nurs*, midwi*, APN, NP, psychological car*, psychological nurs*, depression management, stress management, mental health car*, mental health service*, competenc*, self-competence, abilit*, skill*, knowledge*, attitude*, behavio* (Appendix 1 ).

The team decided on the inclusion criteria as follow: (1) nonpsychiatric nurses were the primary focus of the articles, (2) the research content provided a clear definition of nurses’ PCC, (3) the outcome indicators were nurses’ competences to perform psychological care or the development of nurses’ PCC assessment tools, (4) the research types included quantitative, qualitative, mixed-method and descriptive papers and various types of reviews, (5) articles were written in Chinese and English. Literature exclusion criteria were (1) repeated papers, (2) articles for which full text could not be obtained.

A total of 27,266 records were retrieved and imported into EndNote X8. After removing duplicates, 19,245 articles remained. The first author (QHF) screened the titles and abstracts and excluded irrelevant papers according to the inclusion and exclusion criteria, and 327 articles were included for full-text screening. Next, two authors (QHF, XWL) reviewed the full texts of the potentially relevant articles. Any disagreements about inclusion were resolved through discussion, and if needed, the third author (YYL) was consulted to reach a consensus. Finally, 39 articles were included in this study.

The directed content analysis approach [ 27 ] was used to extract elements related to PCC from the literature. Based on the Iceberg Model, competency elements were classified into different themes, such as knowledge, skills and personal traits [ 28 – 29 ]. The team discussed and combined each individual element into the framework.

Qualitative interviews

From November 2022 to February 2023, semistructured interviews were conducted to supplement PCC. Purposive sampling was used for participant recruitment. The inclusion criteria were willingness to participate in the study, holding a bachelor’s degree or above in nursing, and having at least five years of clinically psychological care experience in nonpsychiatric wards. In this research, 24 participants from four comprehensive hospitals in southern China were recruited.

A structured guide was initially developed based on the STAR (S = situation, T = task, A = action, R = result) principle of the Critical Incident Technique (CIT) to effectively extract PCC and prevent interviewees from going off-topic during the conversation. Then, one nursing manager and one nurse who met the participant inclusion criteria were invited to pretest the guide. The interview guide was revised and finally determined through a discussion among the team, mainly covering the opinions and experiences of psychological care (Appendix 2 ).

After receiving informed consent from the participants, interviews were held and recorded. The interviews were then transcribed verbatim within 48 h. Two experienced researchers (QHF, XWL) repeatedly read the data and used the directed content analysis approach [ 27 ] to analyze the data through NVivo 11.0. First, the meaning of sentences significant to the topic were outlined. The initial codes were then extracted and grouped into categories and subcategories based on similarities and differences. Finally, the description of each category cluster was repeatedly compared to reflect the Iceberg Model and the framework subsequently was refined. Any discrepancies were resolved by discussion and consensus with the research team.

Phase 2: delphi rounds to determine the final PCC framework

Panel of experts.

Purposive sampling was used to recruit experts. The inclusion criteria for experts included: (1) work experience: engaged in psychology care, nursing practice, nursing management, or nursing education with psychology care research experience; (2) educational background: held at least a bachelor’s degree; (3) professional title: intermediate and above; (4) work years: had worked for 5 years or more; and (5) provided informed consent. The minimum sample size for the Delphi technique is 10–15 participants [ 30 ]. With the help of corresponding author (LLZ), the team obtained the e-mail addresses of 20 experts from ten Chinese provinces. Ten experts were from hospitals, and the others from schools. We sent invitation e-mails to them at the same time.

Data collection

The three-round surveys were conducted between March and May 2023. There were three parts in the survey questionnaire: (a) Introduction: This section briefly explained the study’s purpose, voluntary nature, and confidentiality procedure; (b) The initial PCC framework: This section included 5 domains and 22 subdomains proposed in phase 1. Participants were asked to rank the importance of each item on a 5-point Likert scale ranging from 1 (not at all important) to 5 (very important). They were also allowed to comment on each competence and propose new competences; (c) Demographic characteristics: This section included basic information about the experts, such as age, gender, educational background, work direction, professional title, seniority, institution, familiarity coefficient (Cs), and judgment coefficient (Ca). The Cs data were separated into five levels (0.20 = unfamiliar, 0.4 = less familiar, 0.6 = generally familiar, 0.8 = very familiar, 1.0 = extremely familiar), and the Ca data were divided into four categories: theoretical analysis, literature reference, practical experience, and subjective judgment.

The experts received questionnaires through e-mail, and they were expected to complete the questionnaires within one week. The team collected and analyzed the results of the expert questionnaire responses. Competence items reached consensus when the mean was > 3.50 (the importance of items), the coefficient of variation (CV) was < 0.25 and ≥ 75% of respondents rated an item as 4 or 5 points [ 31 – 32 ]. Competence items that did not reach consensus in the first round were modified, deleted or added to the statement and subsequently were fed back in the second round, allowing the experts to potentially revise their responses based on the opinions of other experts. After reviewing the results from the second round, the team later used the feedback to modify the questionnaire for the third round. The updated questionnaire was given to experts in the same manner as before. Experts were asked to complete the revised questionnaire in the same way as before. Following the Delphi guidelines, Delphi rounds need to be repeated until an agreement is reached [ 26 ]. A consensus was reached after the third Delphi round. To preserve the privacy of participants, the team ensured that during the surveys, participants did not meet or know the identities of other participants, and the questionnaires were given to each participant individually [ 33 ].

Data analysis

Excel 2019 and IBM SPSS version 25.0 were used to analyze the quantitative data. Descriptive analysis was conducted using frequency, proportion, mean and standard deviation. The recovery rate (RR) and the authority coefficient of experts were calculated to test the reliability and accuracy of expert consultation results. The calculation formula of the authority coefficient (Cr) is Cr=(Cs + Ca)/2 [ 34 ]. The consistency among experts was presented by the coefficient of variation (CV) and Kendall coefficient (W). The difference was statistically significant with P  < 0.05 when the chi-square test was applied for significance analysis. In addition, Yaahp version 10.3 was used to build a model with two levels corresponding to the competence framework [ 35 ]. Based on the third-round value judgment of the experts, the importance weights of each item were calculated by using the analytic hierarchy process (APH) method. For consistency testing, the consistency index (C.I.) and average random consistency index (R.I.) are usually adopted, and the ratio of C.I. and R.I. is called C.R. When the consistency ratio of the judgment matrix is CR < 0.10, the judgment matrix is considered to have satisfactory consistency [ 36 ].

Quality control

The validity of the questionnaire responses in each round was double checked. After receiving the questionnaires, team members separated the expert personal information forms and the consultation forms into two different folders. The consultation forms did not include expert names, ensuring that expert opinions were anonymous during data analysis. Furthermore, to ensure external validation of the results, the final PCC framework was reviewed and approved by four external experts (two professors from medical universities and two specialist nurses from tertiary hospitals).

Ethical considerations

This study adhered to the tenets of the Declaration of Helsinki. The university affiliated hospital’s ethics committee gave its approval for the study (ID: NFEC-2023-037). Informed consent was provided and obtained from all participants before the study commenced.

Literature review and qualitative interviews

After literature screening, the 39 included articles were finally analyzed. At this stage, we formed the first draft of the PCC framework, including 5 domains and 18 sub-domains.

In the stage of qualitative interviews, 24 participants took part in the interviews with a mean age of 39.67 ± 6.80 years old and a mean working time of 18.20 ± 8.32 years from fields of psychological care, nursing education and nursing management. The demographic characteristics of these participants are shown in Table  1 . The prior report offered more details about the interview results [ 37 ].

Based on the results of the literature review and semi-structured interviews, the team formed the initial PCC framework, including 5 domains and 22 subdomains: knowledge (knowledge of nursing psychology, knowledge of clinical nursing, knowledge of humanities and social sciences), skills (psychological care practical skills, clinical nursing practical skills, psychological crisis management skills, interpersonal skills, health education skills, thinking skills), professional ethics (self-development, responsibility, empathy, equal and respect, professional attitudes, legal awareness), personal traits (self-mental quality, self-reflection, character quality, self-image) and motivations (psychological care awareness, sense of professional identity, sense of collective honor).

Positivity, demographic characteristics, authority and coordination of experts

The response rates of experts for the three Delphi rounds were 80.00% (16/20), 87.50% (14/16), and 92.86% (13/14). These experts came from seven hospitals and six universities in nine Chinese provinces (Guangdong, Sichuan, Beijing, Tianjin, Henan, Shanxi, Hunan, Shanghai, and Gansu). The demographic characteristics of these participants are shown in Table  2 . The Cr scores for the three Delphi rounds were 0.89 (Ca = 0.94; Cs = 0.84), 0.89 (Ca = 0.94; Cs = 0.84) and 0.90 (Ca = 0.94; Cs = 0.85), respectively. Table  3 displays the degree of coordination among experts in each round.

Delphi round 1

Thirteen experts (81%) put suggestions in the first round. Items were modified or added based on experts’ opinions. In the first-level competencies, “motivations” was revised to “internal motivations”. In the second-level competencies, “thinking skills” was revised to “thinking cognitive skills”, “legal awareness” was revised to “legal and ethical awareness”, “self-mental quality” was revised to “mental health literacy”, “self-reflection” was revised to “self-reflective habit”, “sense of professional identity” was revised to “sense of professional acquisition”, and “sense of collective honor” was revised to “sense of collective belonging”.

Three experts suggested that the item “self-development” should be moved from “professional ethics” to “personal traits”. It was accepted with a reason stated by one of the experts, the realization of “self-development” indicates the evolution of an individual’s inner thinking and belongs to personal quality from the psychological point of view.

An expert suggested that “physical health literacy” should be added to the second level of “personal traits”. We accepted the suggestion that nurses’ own physical health is as important as their mental health.

Delphi round 2

In the second round, the altered PCC framework after the first round was presented to the experts. Six experts (43%) made revision suggestions, with 6 of the 23 items requiring further revision. In the second-level competencies, “mental health literacy” was revised to “psychological quality”, and “physical health literacy” was revised to “physical quality” because one expert pointed out that “health literacy” involves knowledge and skills that are inappropriate for the domain of personal traits. The team unanimously accepted this suggestion. In addition, “self-image” was revised to “etiquette and image”, and “psychological care awareness” was revised to “psychological care concept”.

Some modifications were made to more effectively direct the development of the PCC framework. Two experts strongly suggested deleting “character quality” because character is not to be distinguished as good from bad, and this item was repeated in the domain of professional ethics. After a discussion, the team consistently agreed to delete this item. One expert suggested changing the “legal and ethical awareness” to the “legal and ethical practical skills” and reorienting this item to the domain of skills because “practical skills” could be more specific to guide training content.

Delphi round 3

The opinions of the experts converged in the third round, and only one amendment was proposed. “Legal and ethical practical skills” was revised to “legal practical skills”. At the end of the third round, five domains, 22 subdomains, and connotations of the PCC framework obtained a consensus of 75% or more among the expert panel. The weights of each item were determined with a combination of the Delphi technique and the APH method, and all items reached CR < 0.10, as presented in Table  4 .

Combined with the results of literature review and qualitative interviews, this study preliminarily formed a comprehensive and multi-perspective description of the PCC framework, providing strong evidence support materials for further Delphi studies. This was consistent with the point made by Guo et al [ 38 ]. After three rounds of Delphi, the PCC framework included five domains (knowledge, skills, professional ethics, personal traits and internal motivations) and twenty-two subdomains.

For the study to be scientific and reliable, it is crucial that the experts chosen for the Delphi method should be represented, motivated, and authoritative [ 39 ]. First, the experts involved in this study had solid professional knowledge and rich work experience. They came from nine Chinese provinces and had worked for five or more years in nursing or psychological fields. Second, a questionnaire RR > 70% was regarded as an effective consultation [ 40 ]. All RRs of the three rounds were effective. Meanwhile, the experts provided constructive comments on the framework, suggesting that the experts were enthusiastic and active in the Delphi rounds. Third, the Cr scores in the three Delphi rounds were > 0.80, and Kendall’s W for the three rounds was statistically significant ( P  < 0.05), indicating that the results are acceptable and reliable [ 34 ]. Finally, external experts verified the final PCC framework, enhancing the validity of the research results.

A competency framework is developed to support healthcare development and ensure consistent high standards of care nationally [ 41 ]. The PCC framework, developed according to the native culture, could direct nursing administrators or educators to design competence-based training programs [ 42 ]. More importantly, this framework may promote clinical psychological care practices and improve mental health services.

Among the first-level items, the average score and the weight value of “knowledge” are the highest. Knowledge, as part of the iceberg above, is the most basic competence in psychological care. Nurses need to master the theory of psychological care, the identification of adverse emotions, the maintenance of mental health [ 43 ], and clinical nursing expertise. Additionally, nurses need to know well about humanities and social sciences [ 21 , 44 ]. Knowledge plays an important role in competence development [ 45 ]. As early as Ancient China, there was the idea that knowledge precedes practice [ 46 ]. Meanwhile, the Knowledge-Attitude-Belief-Practice (KABP) model in Western countries believes that knowledge is the basis for changing behavior [ 47 ]. In the field of psychological care, nurses’ behavior is related to the requisite knowledge, skills, and work experience [ 48 ]. It has been shown that knowledge is positively related to PCC and that nurses are more capable of providing services when they have a higher level of knowledge [ 49 – 50 ]. Therefore, to accommodate the future development of psychological care, nursing administrators need to conduct various trainings about PCC, invite psychological experts to give lectures and then strengthen nurses’ awareness of psychological care.

The weight of “skills” is comparable to that of “knowledge”, suggesting that skills give significant play to the PCC. Some items in skills, such as practical psychological care skills, practical clinical nursing skills, psychological crisis management skills and interpersonal skills, are consistent with previous findings in other countries [ 51 ], indicating that experts’ expectations of skills in Chinese PCC are in line with international standards. It is noteworthy that thinking cognitive skills and legal practice skills have the highest weights among the second domains, which may be related to the professional requirements for Chinese nurses. The Chinese standardized training program for new nurses encourages nurses to assess evidence-based evidence using critical thinking and scientific methods, thus enhancing their professional knowledge and ability [ 52 ]. Moreover, research indicates that enhancing practical legal skills of nurses could ensure patient safety and decrease care conflict [ 53 ]. Hence, to increase nurses’ psychological care skills, healthcare institutions could try to devise unique trains on various psychological care topics.

Each item on the dimension “professional ethics” has a weight of 0.250, suggesting that they are all equally essential. “Professionalism and ethical behavior are central to medical practice” [ 54 ]. Additionally, the realization of the Healthy China strategy calls for a large number of highly ethical medical professionals. For the item “equal and respect”, a previous study similarly suggested that nurses should respect all patients equally when communicating with them [ 55 ]. In addition, Chinese Medical Education believes that a positive professional attitude is the basic requirement for medical professionals [ 56 ]. Nurses’ competence to care for patients would be severely hampered if they treat patients poorly at work and lack empathy [ 57 ]. Enlighteningly, to direct nurses think about ethics dialectically and logically and to regulate their own roles and actions, hospitals need to carry out clinical psychological care practices from an ethical perspective. Of course, nurses themselves should also combine scientific skills with humanistic literacy.

The weight of “psychological quality” in the dimension of “personal traits” is 0.208, ranking first. Nurses are one of the most important forces in national health care, and their physical and mental health is a guarantee for patients’ life safety and mental health [ 58 ]. Nurses with mental health problems have a detrimental impact on their personal health and have a negative effect on the quality of nursing services and patient treatment [ 59 – 60 ]. In addition, both “self-development” and “self-reflective habits” have weights of 0.198. When developing the nursing competency framework, scholars from many countries believed that the nursing profession aimed to enhance care quality and provide more person-centered care through lifelong learning and reflective practic [ 32 , 61 , 62 ]. In this study, the experts advised that hospitals should create a humanized working atmosphere, pay attention to the physical and mental health of nurses, and be concerned about nurses’ personal career development aspirations.

“Psychological care concept” is ranked first in the dimension “internal motivations”, with a weight of 0.344. In the practical actions of socialist modernization, developing and adhering to correct values can help people perform duties [ 63 ]. Nurses who construct the concept of psychological care could keep themselves involved in high-quality psychological care. The item “sense of professional acquisition” is offered in the context of Chinese comprehensive reform, which aims to improve the medical care system and realize shared development [ 64 ]. Nurses’ sense of professional acquisition refers to the positive emotional state of being content with their career because they enjoy it and value it [ 65 ]. “Sense of collective belonging” is a localized item. Individualism is a fundamental feature of Western culture, and people with this cultural background are concentrated on “I”, while collectivist culture is a typical character in China, and individuals are more centered on “we” [ 58 , 66 ]. When nurses provide psychological care services, their own sense of collective belonging has a profound impact on their job efficiency, which in turn affects the quality of nursing care. To fully explore parts below the iceberg, nursing managers should help nurses understand the characteristics and social value of psychological care, stimulate their motivation for personal growth and strengthen their sense of collective belonging.

Implications for nursing, education, and future research

The results of this study provide a framework for curriculum development and performance assessment in PCC educational programs for nurses. This PCC framework applies to nonpsychiatric nurses because the psychological problems of patients in these departments are of concern. Therefore, nurse educators could use this framework to teach nurses after assessing the validity of cross-cultural adaptation in their respective nations. Furthermore, there is limited evidence as to the impact of nursing education programs on patient outcomes [ 67 ], and nursing work is needed to link competences to practice outcomes. Hence, nursing curricula should include educational outcomes that allow for an accurate evaluation of how these will relate to patient care. In the future, an assessment tool based on the PCC framework will be developed to assess the level of psychological care competency for nurses and the reliability and validity will be reported.

Limitations and strengths

Our study has its own limitations. The gender of recruited participants disproportionately distributed because of the natural of nursing profession. In addition, the study results are highly contextualized. The participants were from a hospital or university setting, and the validated and adapted nursing competencies framework describes nursing practice in hospitals, which does not take into account community care practice. Further research is needed to confirm the transferability of the findings to community care practice.

Based on the Iceberg Model, the PCC framework for nurses is more focused and comprehensive, which can clarify their competency requirements to understand the weaknesses of nursing staff competency and provide targeted training. Moreover, a strength of our Delphi study was the involvement of a relevant national expert panel with significant experience in clinical psychological care. Their expertise allowed for in-depth reflection on the relevance of PCC in China. The study provides useful insights into nursing competences related to psychological care.

This study used a literature review, semistructured interviews and the Delphi technique to develop a PCC framework for Chinese nurses. The results, which included 5 domains and 22 subdomains, were scientific and reliable. This framework could be used both to guide the design of the curricula and to actively involve nurses in self-assessment to understand needs for the development of psychological care. However, this PCC framework developed in this study has yet to be evaluated in clinical settings. An evaluation tool based on the PCC framework will be created in the future to analyze the reliability and validity of this framework. Furthermore, it is suggested that uniform training for nurses based on the PCC framework be conducted in healthcare institutions to improve PCC among nurses.

Data availability

Not applicable.

World Health Organization. Depression and Other Common Mental Disorders. WHO Document Production Services, Geneva, Switzerland. https://www.who.int/publications/i/item/depression-global-health-estimates . Accessed 3 January 2017.

Peng XD, Jin D, Zhou ZJ, Qiu YS, Xie HY, Liu HM, Liu TB. Analysis of reported death cases with severe mental disorder in community in Shenzhen from 2008 to 2017. Chin J Mental Health. 2018;32(09):738–40.

Google Scholar  

Rong L. Investigation and intervention study on the Psychological State of Inpatients in a General Hospital. Guangdong: Southern Medical University; 2018.

Gao J. Psychological problem, psychological nursing needs and its related factors of medical inpatients and surgical inpatients in general hospital. Shandong: Qingdao University; 2013.

Qin P, Webb R, Kapur N, Sørensen HT. Hospitalization for physical illness and risk of subsequent suicide: a population study. J Intern Med. 2013;273(1):48–58.

Article   CAS   PubMed   Google Scholar  

Kavalidou K, Smith DJ, O’Connor RC. The role of physical and mental health multimorbidity in suicidal ideation. J Affect Disord. 2017;209:80–5.

Article   PubMed   Google Scholar  

Wan Q, Ding X, Hu D, Han Y, Wang S, Liu Y, et al. A study of the epidemiology and risk factors for attempted suicide and suicide among non-psychiatric inpatients in 48 general hospitals in Hubei Province, China, 2015–2017. Gen Hosp Psychiatry. 2020;63:21–9.

Herrman H, Patel V, Kieling C, Berk M, Buchweitz C, Cuijpers P, et al. Time for united action on depression: a Lancet-World Psychiatric Association Commission. Lancet. 2022;399(10328):957–1022.

The Central Committee of the Communist Party of China and the State Council. Healthy China 2030. The Central Committee of the Communist Party of China and the State Council. https://www.gov.cn/zhengce/2016-10/25/content_5124174.htm . Accessed 25 October 2016.

Zhang W, An Y, Xiu H, Dou C, Wang Z, Wei Y, et al. Applying a psychological nursing care quality evaluation index in hospitalized patients: a pilot study. Nurs Forum. 2022;57(1):26–33.

Li L. Effect evaluation of specialized psychological nursing training in standardized training of new entry nurses. Liaoning: Jinzhou Medical University; 2022.

Malakian A, Mohammed S, Fazelzad R, Ajaj R, Artemenko A, Mayo SJ. Nursing, psychotherapy and advanced cancer: a scoping review. Eur J Oncol Nurs. 2022;56:102090.

Leng Y, Wu Y, Wang Z, Zhou X, Liao J. A qualitative study exploring barriers and facilitators to establishing nurse-led, multidisciplinary psychological care for trauma patients: experiences from doctors and nurses. BMC Nurs. 2022;21(1):191.

Article   PubMed   PubMed Central   Google Scholar  

Zhang R, Lai J, Wang H. An integrative model of In-Hospital and out-of-hospital nursing care for Non-suicidal Self-Injury: a narrative review. Brain Sci. 2023;13(3):466.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Black P, Boore JRP, Parahoo K. The effect of nurse-facilitated family participation in the psychological care of the critically ill patient. J Adv Nurs. 2011;67(5):1091–101.

Li TY. Investigation on psychological status of non-psychiatric inpatients and study on intervention in general hospital. Yunnan: Kunming Medical University; 2020.

Lan ZZ. Talking about the construction of a new model of psychological nursing work in China. Med Theory Pract. 2019;32(9):1450–2.

Boyer L, Pepin J, Dubois S, Descoteaux R, Robinette L, Dery J, et al. Adaptation and validation of a nursing competencies framework for clinical practice on a continuum of care from childhood to adulthood: a Delphi study. Nurse Educ Today. 2020;93:104530.

King RV, Burkle FM Jr., Walsh LE, North CS. Competencies for disaster mental health. Curr Psychiatry Rep. 2015;17(3):548.

Carroll M, Downes C, Gill A, Monahan M, Nagle U, Madden D, Higgins A. Knowledge, confidence, skills and practices among midwives in the Republic of Ireland in relation to perinatal mental health care: the mind mothers study. Midwifery. 2018;64:29–37.

Waite R, Hassouneh D. Structural competency in mental health nursing: understanding and applying key concepts. Arch Psychiatr Nurs. 2021;35(1):73–9.

Murphy JP, Rådestad M, Kurland L, Jirwe M, Djalali A, Rüter A. Emergency department registered nurses’ disaster medicine competencies. An exploratory study utilizing a modified Delphi technique. Int Emerg Nurs. 2019;43:84–91.

Green RA. The Delphi technique in Educational Research. SAGE Open 2014;4(2).

Bhandari S, Wahl B, Bennett S, Engineer CY, Pandey P, Peters DH. Identifying core competencies for practicing public health professionals: results from a Delphi exercise in Uttar Pradesh, India. BMC Public Health. 2020;20(1):1737.

Spencer LM, Spencer SM. Competence at work: models for Superior Performance. New York: John Wiley & Sons; 1993.

Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on conducting and REporting DElphi studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706.

Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Liu J, Qiu H, Zhang X, Zhang C, He F, Yan P. Development of billing post competency evaluation index system for nurses in China: a Delphi study. BMC Nurs. 2023;22(1):136.

Bai X, Gan X, Yang R, Zhang C, Luo X, Luo C, Chen S. Construction of a competency evaluation index system for front-line nurses during the outbreak of major infectious diseases: a Delphi study. PLoS ONE. 2022;17(7):e0270902.

Skulmoski GJ, Hartman FT, Krahn J. The Delphi Method for Graduate Research. J Inform Technol Education: Res. 2007;6(1):1–12.

Liu Y, Aungsuroch Y, Sha L, Gunawan J, Zeng D. Construction of evaluation indexes of nursing students’ quality and safety competencies: a Delphi study in China. J Prof Nurs. 2021;37(3):501–9.

Ye J, Tao W, Yang L, Xu Y, Zhou N, Wang J. Developing core competencies for clinical nurse educators: an e-delphi-study. Nurse Educ Today. 2022;109:105217.

Humphrey-Murto S, Varpio L, Wood TJ, Gonsalves C, Ufholz LA, Mascioli K, et al. The Use of the Delphi and Other Consensus Group Methods in Medical Education Research: a review. Acad Med. 2017;92(10):1491–8.

Dai F, Wei K, Chen Y, Ju M. Construction of an index system for qualitative evaluation of undergraduate nursing students innovative ability: a Delphi study. J Clin Nurs. 2019;28(23–24):4379–88.

Mi Y, Wu D, Qian JP, Qian GA. Constructing the evaluation index system of nursing staff’s health education competency based on delphi method and analytic hierarchy process. Chin Mod Prev Med. 2020;47(16):2895–8.

Shen L, Yang J, Jin X, Hou L, Shang S, Zhang Y. Based on Delphi method and Analytic Hierarchy process to construct the evaluation index system of nursing simulation teaching quality. Nurse Educ Today. 2019;79:67–73.

Fang QH, Li XW, Xiao L, Liu ST, Luo YY, Luo JH, Tan WX, Zhang LL. Competency characteristics of psychological care in clinical nurses: a qualitative study. Chin J Nurs Sci. 2023;38(24):76–80.

Guo X-E, Bian L-F, Li Y, Li C-Y, Lin Y. Common domains of nurses’ competencies in public health emergencies: a scoping review. BMC Nurs. 2023;22(1):490.

Zhao XS, Zhang BR, Zhang D, Guan RX, Hu LL, Zhang JP. Establishment of quality indicator system for nursing in pediatric continuous blood purification. Chin J Nurs. 2020;55(09):1366–71.

ADS   Google Scholar  

Zhao ZG, Cheng JQ, Xu SL, Hou WL, Richardus JH. A quality assessment index framework for public health services: a Delphi study. Public Health. 2015;129(1):43–51.

Simpson M, Millerick Y, Girdler-Heald L, Higginbotham K, Whittingham K, Masters J, Barton C. Developing the heart failure specialist nurse competency Framework. Br J Nurs. 2022;31(14):732–7.

Mills JA, Cieza A, Short SD, Middleton JW. Development and validation of the WHO Rehabilitation Competency Framework: a mixed methods study. Arch Phys Med Rehabil. 2021;102(6):1113–23.

Alderson J, Hamblin RP, Crowne EC. Psychological care of children and families with variations or differences in Sex Development. Horm Res Paediatr. 2023;96(2):222–7.

Lin Y, Lu YZ, Zhen Y. A Survey on the awareness and ability of Mental nursing care in nurses. Chin J Nurs. 2007;(22):13–4.

Yuan X. A mixed study on the current situation and influencing factors of nurses’ practice ability of palliative care in Hangzhou. Zhejiang: Zhejiang University of Traditional Chinese Medicine; 2022.

Huang JH. Health Pedagogy. Shanghai: Fudan University Press; 2011.

Ramjan LM, Maneze D, Krstevska E, Pathrose SP, Lewis P. Educational programs to improve nurses’ knowledge, attitude, and practice in adolescent and young adult health: an integrative review. Nurse Educ Today. 2022;118:105510.

Mahsoon AN, Dolansky M. Safety culture and systems thinking for predicting safety competence and safety performance among registered nurses in Saudi Arabia: a cross-sectional study. J Res Nurs. 2021;26(1–2):19–32.

Rong L, Zhou HZ, Wang QF, Wu Z. Role of psychiatric consultation-liaison nurses in mental health management. Chin J Nurs. 2017;32(15):73–6.

Moreno-Poyato AR, Casanova-Garrigos G, Roldán-Merino JF, Rodríguez-Nogueira Ó. MiRTCIME.CAT working group. Examining the association between evidence-based practice and the nurse-patient therapeutic relationship in mental health units: a cross-sectional study. J Adv Nurs. 2021;77(4):1762–71.

Cusack E, Killoury F, Nugent LE. The professional psychiatric/mental health nurse: skills, competencies and supports required to adopt recovery-orientated policy in practice. J Psychiatr Ment Health Nurs. 2017;24(2–3):93–104.

Chen YY, Cheng SZ, Chen LF, Zhang XY. Establishment of a core competence based standardised training model for junior staff nurses. Chin Mod Clin Care. 2022;21(11):57–62.

Hariharan S, Jonnalagadda R, Walrond E, Moseley H. Knowledge, attitudes and practice of healthcare ethics and law among doctors and nurses in Barbados. BMC Med Ethics. 2006;7:E7.

Core Committee, Institute for International Medical Education. Global minimum essential requirements in medical education. Med Teach. 2002;24(02):130–5.

Article   Google Scholar  

Wu C, Wu P, Li P, Cheng FX, Du YL, He SZ, Lang HJ. Construction of an index system of core competence assessment for infectious disease specialist nurse in China: a Delphi study. BMC Infect Dis. 2021;21(1):791.

Han SF, Zhang Q, Zhang PL, Guo JL, Shang LP, Jin J, et al. Reflections on the development strategy of nursing schools in China’s higher education institutions in the new era. Chin Nurs Res. 2022;36(08):1317–25.

Zhang YT, Liu RY, Jiao XP. Correlation among caring ability, job burnout and missed nursing care in oncology nurses. Chin Nurs Res. 2021;35(11):2046–9.

Lu XY, Shang DX, Wang SS, Guo YF. Construction of Psychological Capital Assessment Scale for nurses. Chin J Nurs. 2022;37(24):59–63.

Yang GY, Zhao MZ, Yao P. Research on the construction of psychological intervention model of nurses in Grade-A Tertiary hospitals based on Delphi Method and Analytic Hierarchy Method. Med Soc. 2023;36(07):139–44.

Arulappan J, Pandarakutty S, Valsaraj BP. Predictors of nurse’s happiness: a systematic review. Front Nurs. 2021;8(4):313–26.

Tolosa-Merlos D, Moreno-Poyato AR, Gonzalez-Palau F, Perez-Toribio A, Casanova-Garrigos G, Delgado-Hito P, MiRTCIME.CAT Working Group. Exploring the therapeutic relationship through the reflective practice of nurses in acute mental health units: a qualitative study. J Clin Nurs. 2023;32(1–2):253–63.

Chan TE, Lockhart JS, Schreiber JB, Kronk R. Determining nurse practitioner core competencies using a Delphi approach. J Am Assoc Nurse Pract. 2020;32(3):200–17.

Su YS. A new dimension of responsibility for values. Chin J Hubei Univ Econ (Humanities Social Sci Edition). 2022;19(05):4–9.

MathSciNet   Google Scholar  

Bai XL, He Y, Li JW, Bao CL, Meng LP, Liu Y, et al. Occupational acquisition and influencing factors among nurses. Chin Nurs Manage. 2020;20(01):67–73.

Zhang RL, Li MZ, Jiang H, Niu QH, Shen JJ, Liu YL. Research status and outlook about nurses’ professional benefit. Chin Nurs Manage. 2015;15(12):1479–83.

Yang CY. A comparative study of and reflections on differences in Cultural psychology between China and the West. J Xidian Univ (Social Sci Edition). 2007;17(2):114–20.

Kerr H, McSorley O, Donovan M. Registered nurses’ perspectives of the impact of a Post-registration Education qualification on patient care and clinical practice in Cancer Care: a qualitative study. J Cancer Educ. 2022;38(3):900–5.

Download references

Acknowledgements

The authors would like to thank all of the specialists and nurses who contributed to this study.

This study was supported by the Humanities and Social Science Research Project of the Ministry of Education (Grant No. 22YJA880084). These funding sources had no role in the design of this study and did not have any role during its execution, analyses, interpretation of the data or decision to submit results.

Author information

Qinghong Fang and Xingwen Li contributed equally to this work as joint first authors.

Authors and Affiliations

School of Nursing, Southern Medical University, Guangzhou, Guangdong Province, China

Qinghong Fang, Xingwen Li, Yuanyuan Luo, Zhihui Yang, Lin Xiao, Wenxuan Tan, Suting Liu & Lili Zhang

Department of Nursing, Taikang Tongji (Wuhan) Hospital, Wuhan, Hubei Province, China

You can also search for this author in PubMed   Google Scholar

Contributions

QHF, XWL, YYL and LLZ: design of the research. QHF, XWL, YYL, ZHY, STL and JHL: competence analysis. QHF, XWL, YYL, STL and WXT: development of the Delphi questionnaires and collection of data. QHF, XWL, YYL, ZHY, LX and JHL: analysis of the Delphi study’s results. All authors: development and writing of the final competence framework. QHF and XWL: writing the paper. LLZ: editorial revision of the paper. All authors approved the final manuscript.

Corresponding author

Correspondence to Lili Zhang .

Ethics declarations

Ethics approval and consent to participate.

This study was performed in line with the principles of the Declaration of Helsinki and its later amendments or comparable ethical standards. Ethical approval was given by the Medical Ethics Committee of Nan Fang Hospital of Southern Medical University (ID: NFEC-2023-037). Informed consent was obtained from all study participants.

Consent for publication

Competing interests.

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Fang, Q., Li, X., Luo, Y. et al. Developing a psychological care competences framework for nurses in China: a mixed methods study. BMC Nurs 23 , 129 (2024). https://doi.org/10.1186/s12912-024-01778-3

Download citation

Received : 22 November 2023

Accepted : 30 January 2024

Published : 19 February 2024

DOI : https://doi.org/10.1186/s12912-024-01778-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Psychological care
  • Competence framework
  • Mixed-method approach
  • Delphi technique

BMC Nursing

ISSN: 1472-6955

literary studies research methods

IMAGES

  1. PPT

    literary studies research methods

  2. PPT

    literary studies research methods

  3. PPT

    literary studies research methods

  4. steps for writing a good literature review

    literary studies research methods

  5. uses of literature review in research methodology

    literary studies research methods

  6. Introduction To Research and Types of Literary Research| Research

    literary studies research methods

VIDEO

  1. Introduction to Literary Studies Important Questions BA PROG, Eng Hons 1st Semester SOL DU

  2. How to present the research context when writing the methodology #shorts

  3. Research Methods: Writing a Literature Review

  4. Introduction to Literary Studies Important Questions BA PROG & BA Hons English First Semester DU SOL

  5. Introduction to Literature and Literary Studies

  6. Introduction to literary studies semester 1 Question Paper 📜 #dusol #dsc #duregular #ba

COMMENTS

  1. Research Methods

    Usually literary research involves a combination of methods such as archival research , discourse analysis, and qualitative research methods. Literary research methods tend to differ from research methods in the hard sciences (such as physics and chemistry).

  2. Is There a Method/Methodology for Literary Research?

    Is There a Method/Methodology for Literary Research? Authors: John Sekar Jeyaraj The American College, Madurai Abstract While English Studies as an academic discipline is roughly a century old...

  3. Research Methods

    Usually literary research involves a combination of methods such as archival research and discourse analysis. Literary research methods tend to differ from research methods in the hard sciences (such as physics and chemistry).

  4. Research Methodology- An Introduction to Literary Studies

    Research Methodology- An Introduction to Literary Studies. February 2020 Authors: Anila Pillai Veer Narmad South Gujarat University Urvashi Kaushal Sardar Vallabhbhai National Institute of...

  5. Methodologies

    Methodologies Methodologies (not to be confused with methods, which we discuss on the next page) are linked to literary theories. Methodologies are necessary to working with theories. They serve as the interfaces between theory (purely conceptual) and praxis (practical application).

  6. Research Methodology- An Introduction To Literary Studies

    This presentation of literary works that engage arts-based methods of research was created for Dr. Daniel Deslaurier's Integral Research: Arts and Imagination-Based Methods course in the Transformative Inquiry Department at California Institute of Integral Studies.

  7. Handbook of Empirical Literary Studies

    This handbook reviews efforts to increase the use of empirical methods in studies of the aesthetic and social effects of literary reading. The reviewed research is expansive, including extension of familiar theoretical models to novel domains (e.g., educational settings); enlarging empirical efforts within under-represented research areas (e.g., child development); and broadening the range of ...

  8. Research Methods for English Studies

    Introduces readers to a wide range of research methods for use in English Studies. With a revised Introduction and with all chapters revised to bring them completely up-to date, this new edition remains the leading guide to research methods for final-year undergraduates, postgraduates taking Masters degrees and PhDs students of 19th- and 20th-century Literary Studies.

  9. Literary Research Guide

    16. Welsh Literature. 17. American Literature. Ethnic and Minority Literatures. Early American Literature (to 1800) Nineteenth-Century Literature. Twentieth-Century Literature. 18.

  10. Course

    LITT3001 - Research Methods in Literature Studies About Examination arrangement Examination arrangement: Home examination Grade: Letter grades Course content This course will provide insight into central methodological issues within the field of literature studies.

  11. Is there a Method/Methodology for Literary Research?

    Keywords: literary research methods or methodologies, literary theories, research skills, oral history, archival, discourse analysis, textual analysis, interviewing, ICT Suggested Citation: Jeyaraj, John Sekar, Is there a Method/Methodology for Literary Research? (March 14, 2016). Methods and Outcomes of Research in English.

  12. Poetry and prose as methodology: A synergy of knowing

    Abstract. In this study, situated in the borderland between traditional and artistic methodologies, we innovatively represent our research findings in both prose and poetry. This is an act of exploration and resistance to hegemonic assumptions about legitimate research writing. A content analysis of young adult literature featuring trafficked ...

  13. Theory and Methods

    There are three method types: qualitative, quantitative, and historical. Likewise, some research uses mixed methods. Qualitative research is interested in the specific.

  14. Textual Analysis

    Here are some ways that textual analysis can be used in research: To explore research questions: Textual analysis can be used to explore research questions in various fields, such as literature, media studies, and social sciences. It can provide insight into the meaning, interpretation, and communication patterns of text.

  15. (Pdf) 'Applying' Theories in Literary Research

    Methodology is theoretical and ideological. Unless researchers are convinced by a particular methodology, they cannot derive their methods out of it. Literary theories are understood as ...

  16. Reviewing the research methods literature: principles and strategies

    Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research Stephen J. Gentles, Cathy Charles, David B. Nicholas, Jenny Ploeg & K. Ann McKibbon Systematic Reviews 5, Article number: 172 ( 2016 ) Cite this article 49k Accesses 26 Citations 13 Altmetric Metrics Abstract

  17. Strategies for Conducting Literary Research

    Chapter Four: Theory, Methodologies, Methods, and Evidence Chapter Four Objectives 57 Theory Guides Inquiry 59 Research Methods 66 Evidence 72 Chapter Five: Reading and Interpreting Literary Works Chapter Five Objectives 75 Reading Literary Works 76 Interpreting Literary Works 86 Chapter Six: Reviewing the Secondary Literature / Types of

  18. Literature Review Research

    Literature Review is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

  19. Literature review as a research methodology: An ...

    1. Introduction Building your research on and relating it to existing knowledge is the building block of all academic research activities, regardless of discipline. Therefore, to do so accurately should be a priority for all academics. However, this task has become increasingly complex.

  20. Chapter 9 Methods for Literature Reviews

    9.3. Types of Review Articles and Brief Illustrations. EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic.

  21. Nursing students' Health Literacy skills: a scoping review protocol for

    Methods and analysis This scoping review will be conducted following Arksey and O'Malley's framework. This study is based on the Joanna Briggs Institute manual. A systematic search will be performed by four researchers using the electronic databases of MEDLINE (via PubMed), the Education Resources Information Centre, the Cumulative Index to Nursing and Allied Health Literature, Scopus, Web ...

  22. 'It depends': what 86 systematic reviews tell us about what strategies

    Twenty-three studies scored 7, six studies scored 6, one study scored 5, one study scored 4 and one study scored 3. The primary focus of the review was on reviews of effectiveness studies, but a small number of reviews did include data from a wider range of methods including qualitative studies which added to the analysis in the papers [ 18 ...

  23. At the Human Longevity Lab, studying methods to slow or reverse aging

    "Knowledge gained from this research may allow scientists to develop methods to slow the process of aging and push back the onset of aging-related disease, hopefully extending the 'healthspan.'" Anyone is eligible to participate in the Northwestern research study, but the scientists are focused on studying people who are disadvantaged ...

  24. Introduction to Strategies for Conducting Literary Research

    The Complexity of Literary Studies Research. In a research-based course, your instructor might require you to write a research paper of 5-6 or 10-12 pages, but the knowledge you need to perform this task can fill up dozens of pages. ... Understand how theory relates to particular research methodologies and methods for gathering evidence; Learn ...

  25. Impact of Menthol Cigarette Bans: A Systematic Review and Meta-Analysis

    Meta-analyses increase statistical power and provide more robust summary estimates with greater generalizability than the results of individual studies. 9. Methods Data Sources and Search. A literature search of PubMed/Medline, CINAHL, PsycINFO, Web of Science, and Embase was conducted on May 20, 2020, and updated on November 3, 2022.

  26. Development and application of emotion recognition technology

    Effectively constructing an emotion classification model using neurophysiological, facial feature, and behavioral data recorded from portable devices, along with machine learning methods, showcases a novel research area. This study reviews relevant literature from the past decade to delineate current trends and hotspots in emotion recognition ...

  27. Developing and Retaining Homecare Nurses Through Employer-Based Tuition

    Tuition assistance reduced financial barriers to career advancement; 83% of recipients remained with their employer for at least 1-year post-studies but only 29% experienced career advancement. Psychosocial supports, career navigation and coaching to ease the licensing and role transition processes were identified as opportunities to support ...

  28. Developing a psychological care competences framework for nurses in

    A descriptive mixed methods study was designed consisting of a literature review and semi-structured interviews followed by three Delphi rounds. The experts (n = 16) involved were nurses, nursing managers and educators from nine Chinese provinces with a specific interest in psychological care. ... The research methods were feasible and the ...