e-ISSN: 2448-8062

ISSN: 0188-431X

Open Journal Systems

Checklist as a tool for critical reading of published research papers EXPERIENCES IN PRACTICE
How to cite this article:
Romo-Martínez JE. La lista de cotejo como herramienta para la lectura crítica de artículos de investigación publicados. Rev Enferm Inst Mex Seguro Soc. 2015;23(2)109-13.

Checklist as a tool for critical reading of published research papers

Jesús Eduardo Romo-Martínez1

1Epidemiología, Unidad de Medicina Familiar/Unidad Médica de Atención Ambulatoria No. 29, Instituto Mexicano del Seguro Social, Acapulco, Guerrero, México

Correspondence: Jesús Eduardo Romo-Martínez

Email: jesus.romom@imss.gob.mx

Received: September 12th 2014

Judged: February 16th 2015

Accepted: March 24th 2015


The growth of scientific knowledge requires health professionals to form a habit of reading medical journals with minimum skills of comprehension, analysis and criticism in order to enrich the exercise of continuous updating. Learning the basics of epidemiology plays a very important role at the time of reading a medical publication; from the analytical point of view, it is important to understand the basis of this branch of medicine that allow to generate a critique of what we read with grounds. A check list is a useful test for the evaluation through observation (in this case the reading of a document), which enumerates the characteristics, aspects, qualities, etcetera, whose presence (or absence) we aim to determine. It is a list of comparison (material) to be applied while reading medical publications (method); an objective tool to facilitate the understanding and analysis. This list is divided into five essential sections: an overview of the publication, type of study, methodology, presentation and interpretation of results, as well as the discussion and conclusions of the publication. It is imperative to develop capacities to evaluate critical aspects that allow us using valid information and then integrating them to our knowledge. Far from seeking to shorten the fundamentals and basic concepts of basic epidemiology, biostatistics and research methodology, this is an invitation to the reader of medical journals to take a close look at the fascinating field of these branches of medicine through the references.

Keywords: Critical reading; Scientific and technical publications; Checklist


Significant and abundant evidence in the field of biomedical, clinical, and public health research has been generated on a global level;1 this is a step forward that, thanks to technology and the expansion of the media, is available to health professionals and the general public. Given this algorithmic growth of technical and scientific knowledge (more than 20,000 biomedical journals and 2 million publications a year),2 it is necessary to make a special focus for those working in the field of public health and epidemiology.3

As health professionals, we are forced to form a habit of reading with minimal skills of comprehension, analysis, and criticism to truly enrich the exercise of continuously staying up to date.4 Publications or medical articles are, per se, the tool of first resort to achieve this objective.5 This tool moreover represents a very profitable activity at the time of research design, as both activities have many similarities and points convergence.6

The first thing you should turn to is the means by which we will get the information. To do this we must know the available information resources that are electronically systematized to facilitate obtaining publications and which are stored in Internet databases: Medline, PubMed, LILACS, EMBASE, OVID, EBSCO, Cochrane, et cetera. The next step is to choose the article, so we have proposed several strategies to select a publication under the assumptions of relevance and quality of the presence of a number of features (in most cases).7 Finally, the article should be read and reviewed to generate an idea and, even better, a critique of it.

Taking into account the principles of epidemiology plays an important role in critical conception when reading a publication, so from an analytical point of view it is important to understand the basics of numerical language,8 know the basics of biostatistics,9 understand the methodology of epidemiological research and other skills in order to formulate an enriched opinion and determine the clinical relevance of the material consulted.10
A checklist is a useful verification tool for evaluating through observation (in this case the reading of a document). It lists the features, aspects, qualities, etc., of which one is interested in determining the presence (and absence).

The aim of this work is to provide, through a checklist, an objective tool to facilitate understanding and analysis of medical literature.

Tool design

A tool is presented in checklist format (material) to execute during the reading (method) of medical publications. It is a guide characterized by "checklist" format, divided into five sections, allowing readers to identify the most essential points in order to develop the ability to form a critique and an opinion on any published work, and for evidence to strengthen their knowledge (Table I).

Table I. Checklist for reading medical journals
Overview of the publication
Was the article published in a recognized journal suitable for the theme? Yes [] No []
Was the article published recently (less than five years)? Yes [] No []
Is the author of the article a recognized expert in the field? Yes [] No []
Is the title of the publication clear? Yes [] No []
Does the title highlight the independent variable (cause) and the dependent variable (effect)? Yes [] No []
Is the abstract structured? Yes [] No []
Are the key words appropriate? Yes [] No []
Is the introductory framework consistent with the theme? Yes [] No []
Does the objective have a direct relationship with the title? Yes [] No []
Type of study (epidemiological)
Transverse (studies the prevalence of a disease) [   ]
Cohort (studies the natural history of the disease, the prognosis and causes) [   ]
Cases and controls (identifies potential causal factors) [   ]
Experimental (clinical randomized) (investigates the effectiveness of intervention) [   ]
Methodology used
Does it describe the date, the place, and the people in which the study was based? Yes [] No []
Does it describe the criteria for inclusion and exclusion of the studied population? Yes [] No []
Does it describe the calculation, or the selection of the sample, and is it valid? Yes [] No []
Could there be some kind of bias (selection, information, confusion)? Yes [] No []
Does it describe the ways of capturing information? Yes [] No []
Does it describe the control group (if applicable)? Yes [] No []
If it is a cohort study, was there adequate follow-up of a proportion of the sample? Yes [] No []
If it is a case-control study, are the controls properly matched with cases? Yes [] No []
If it is an experimental study, are the subjects distributed randomly in the treatment groups? Yes [] No []
If it is an experimental study, do they make comparison between treatments? Yes [] No []
Is there quality control of measurements? Yes [] No []
Is the hypothesis clearly stated? Yes [] No []
Does it mention the statistical test for qualitative variables (e.g. chi-square)? Yes [] No []
Does it mention the statistical test for quantitative variables (e.g. Student's t)? Yes [] No []
Does it mention the statistical significance of the p-value ?  (e.g. < 0.05)? Yes [] No []
Does it mention the statistical package used (e.g. Epi Info ™, SPSS ™, etc)? Yes [] No []
Presentation and interpretation of data
Is the information presented in graphics or pictures not repeated in writing? Yes [] No []
Are the graphics or pictures sufficient (suggested up to five total)? Yes [] No []
Do the graphics or pictures to demonstrate to the population studied, percentages, strata, etc? Yes [] No []
Are the graphics appropriate? * Yes [] No []
Are the outcome or outcomes (odds ratio [OR] or relative risk [RR]) suitable?‡‡ Yes [] No []
Are the outcome or outcomes of confidence intervals (CI 95%) in the right range? ‡ Yes [] No []
Are the p-value result or results statistically significant?§ Yes [] No []
Do the results reflect the population included in the methodology? Yes [] No []
Could the reader tabulate the information in a contingency table or "2 x 2"? Yes [] No []
Does it present results with measures of central tendency (mean, median, mode)? Yes [] No []
Does it present results with measures of dispersion (standard deviation, percentiles, ranges, etc)? Yes [] No []
Is there a statistically significant difference in the results? Yes [] No []
Was there a difference between the groups studied? Yes [] No []
Discussion and conclusions of the publication
Does the main outcome suggest new knowledge? Yes [] No []
Does it contrast the main result with experiences previously published? Yes [] No []
Are secondary outcomes compared with those previously published? Yes [] No []
Do the authors explain the results objectively? Yes [] No []
Are the results relevant from an epidemiological point of view, even if there is no statistically significant difference? Yes [] No []
Does it respond exclusively and promptly to the objectives? Yes [] No]
Are the strengths and limitations of the study made explicit? Yes [] No]
Does the study conclude with contribution, recommendations and main message? Yes [] No]
The nature of the graph or figure will depend on the information presented; for example, a pie chart for proportions, lines graph to observe a given phenomenon over time, an epidemic curve to observe the incidence of cases, a map of rates to see the geographical impact of a phenomenon, a Kaplan-Meier curve to observe survival, etc.
‡‡An OR or RR equal to 1 is a null result; from 0 to < 1 is protective effect; > 1 is a risk factor.
‡ CI 95% of OR or RR must respect the limit "1" to be considered appropriate; for example, an OR = 2.3 (95% CI 1.8-2.6) is adequate, while an OR = 5.8 (CI 95%, 0.02‑10.6) is considered not appropriate.
§For a p-value to be statistically significant, it must be < 0.05 or < 0.001 as the author chooses.

Overview of publications

Definitely a reading with critical view is not always easy,11 and the first step when we hold reading material in our own hands (or on the screen of our computer) will be to evaluate general aspects to decide whether to continue.

In most cases we turn to publications in order to stay up to date, so logically we must consider the source, taking into account the journal and if its theme is appropriate to our field and needs; check that the research is relatively current (less than five years); if the author is recognized in the subject (with other similar work); the institution where the research took place, and so on.

In any publication it is hoped that the title of material will project a short nature of the investigation. This must be clear and preferably show the cause and effect variables; in the best case, it will specify the type of study.

The abstract will be presented in a structured way, according to the sections of the publication and providing appropriate keywords to grasp the nature of the article.

The introduction of the work will provide an overview of the subject matter, definitions and background, inviting the reader to being reading. Finally, the objectives should be clear and directly connected with the title; this is expected to be in the infinitive (to describe, to analyze, to investigate, etc.).12

Type of study

Medical literature has multiple publications regarding the type of article, which is, broadly speaking, informative or of a research nature. For the informative article we have newsletters, reviews, letters to the editor, consensus information, and so on. In the case of research articles they are found in the form of original articles; however, it is worth noting that those of an epidemiological nature are best suited to exercise judgment based on an investigation, because medicine is mainly built on evidence, because they provide results obtained from populations.

Within the category of other types of research studies, such as case studies or case series, beyond providing scientific evidence, they share with us an outcome from the point of view of isolated experiences.13

Once a publication has been identified with epidemiological research, you should consider the divisions within this kind of studies, which are classified according to specific criteria, among which jump out the factor of exposure and its effect, temporality, and the position of the researcher (observational or experimental).6,10

Methodology in a publication

This section is the backbone of any item. It is related to the objective description of the way the investigation was conducted, showing the steps to the extent that it could be reproduced by a researcher.1 This section should contain information about the date and place where the research was conducted; define the study population; describe the criteria for their inclusion or exclusion; expose and ideally exemplify the calculation of sample size;14,15 consider the types of potential bias;1 appropriately explain the type of epidemiological study looking at the distribution of subjects included, comparison, follow-up over time (if applicable); mention quality control, etc.6,8,10

One aspect that deserves special attention has to do with the hypothesis. It is said that this is "null" when there is no statistically significant difference, and therefore it is likely that the result is due to chance; it is said to be "alternative" when there is a statistically significant difference (e.g. a result with p-value <0.05). This latter is how researchers mostly opt to provide relevant knowledge. This statistical result is derived from a special test, depending on the type of variable (qualitative or quantitative).17

Presentation and interpretation of results

To describe and analyze data, it is necessary to apply the concepts and methods of biostatistics. In epidemiological research, samples are often used from which one can infer characteristics of the populations studied.6 The findings should be presented in a logical order and the most important highlighted using the past tense; they will be supported by tables and figures appropriate to the information they provide and in turn saving on redundant descriptions in the text.1,6

Discussion and conclusions of the publication

The discussion should answer the research question, as this section deals with the evaluation and interpretation of the data.1 It should preferably follow a logical order to break down the main contributions of the study and put greatest emphasis on the primary outcome and comparison with other research and with secondary outcomes. We must take into account that even if there is no statistically significant result, this may be relevant in some situations. Also the author must state the strengths and limitations of the study and finally conclude with the contribution of research (if applicable), recommendations, and the main message.

Final thoughts

When evaluating a published work with these points, the goal is to develop capabilities to assess critical aspects which allow the use of valid information and to then integrate them into our knowledge, to consider whether such an article read offered an impact or anything new for current knowledge and if it was really significant to your need.

At no time was the goal to abbreviate the fundamentals and basic concepts of basic epidemiology, biostatistics and research methodology; rather, the opposite: to invite the reader in the spirit of staying current, to delve into the fascinating field of epidemiology and public health through the referenced bibliographies.

  1. Hernández M, Lazcano E. Salud Pública teórica y práctica. 1st ed. México: Manual Moderno; 2013.
  2. Rivas R, Talavera J. Investigación clínica VII. Búsqueda sistemática: cómo localizar artículos. Rev. Med Inst Mex Seguro Soc. 2012;50(1):53-8.
  3. Tricco A, Tetzlaff J, Moher D. The art and science of knowledge synthesis J. Clin epidemiol. 2011;64:11-20.
  4. Khan K, Kunz R, Kleijnen J, Antes G. Systematic reviews to support evidence-based medicine. Second edition. London: Royal Society of Medicine; 2011.
  5. Feinstein AR. Clinical epidemiology. The architecture of clinical research. US: WB Saunders; 1985.
  6. Bonita R, Beaglehole R, Kjellström. Epidemiología básica. Segunda edición. Washington, D. C: OPS; 2008.
  7. Talavera J, Rivas-Ruiz R. Investigación clínica VIII. Revisión estructurada de un artículo. Rev Med Inst Mex Seguro Soc. 2012;50 (2):163-6.
  8. Londoño F. Metodología de la investigación epidemiológica. 3a ed. Bogotá: Manual Moderno; 2004.
  9. Feinstein AR. Principles of medical statistics. New York, NY: Chapman and Hall/CRC; 2002.
  10. Hernández M, Garrido F, López S. Diseño de estudios epidemiológicos. Salud Publica Mex. 2000;42(2):144-54.
  11. Zaccai J. How to assess epidemiological studies. Postgrad Med J. 2004;80:140-7.
  12. González-Labrador G. Partes componentes y elaboración del protocolo de investigación y terminación del trabajo de la residencia. Rev Cubana Med Gen Integr. 2010; 26(2):387-406.
  13. Portney LG, Watkins MP. Foundations of clinical research. Applications to practice. Chicago IL: Appleton and Lange; 1993.
  14. Celis AJ. Bioestadística. 2a ed. México: Manual Moderno; 2008.
  15. Talavera J, Rivas R, Bernal L. Investigación clínica V. Tamaño de la muestra. Rev Med Inst Mex Seguro Soc. 2011;49(5):517-522.
  16. Hernández M, Garrido F, Salazar E. Sesgos en estudios epidemiológicos. Sal Publica Mex. 2000;42(5):438-46.
  17. Talavera J, Rivas R. Investigación clínica IV. Pertinencia de la prueba estadística seleccionada. Rev Mex Inst Mex Seguro Soc. 2011;49(4):401-5.

Enlaces refback

  • No hay ningún enlace refback.
Contenido de la revista

Tamaño de fuente

Herramientas del artículo
Envíe este artículo por correo electrónico (Inicie sesión)