Image by James Yang http://www.jamesyang.com
PhD2Published has several informative posts about writing journal articles, and more recently has featured a post outlining a potentially revolutionary collaborative peer review process for this kind of publishing. Todays post offers an alternative perspective; that of the journal article peer reviewer. Doing peer reviews provides important experience for those writing their own papers and may help writers consider what they should include based on what peer reviewers are looking for.
At some point in your scholarly career, you likely will get asked to review an article for a journal. In this post, I explain how I usually go about doing a peer review. I imagine that each scholar has their own way of doing this, but it might be helpful to talk openly about this task, which we generally complete in isolation.
Step One: Accept the invitation to peer review. The first step in reviewing a journal article is to accept the invitation. When deciding whether or not to accept, take into consideration three things: 1) Do you have time to do the review by the deadline? 2) Is the article within your area of expertise? 3) Are you sure you will complete the review by the deadline? Once you accept the invitation, set aside some time in your schedule to read the article and write the review.
Step Two: Read the article. I usually read the article with a pen in hand so that I can write my thoughts in the margins as I read. As I read, I underline parts of the article that seem important, write down any questions I have, and correct any mistakes I notice.
Step Three: Write a brief summary of the article and its contribution. When I am doing a peer review, I sometimes do it all in one sitting – which will take me about two hours – or I read it one day and write it the next. Often, I prefer to do the latter to give myself some time to think about the article and to process my thoughts. When writing a draft of the review, the first thing I do is summarize the article as best I can in three to four sentences. If I think favorably of the article and believe it should be published, I often will write a longer summary, and highlight the strengths of the article. Remember that even if you don’t have any (or very many) criticisms, you still need to write a review. Your critique and accolades may help convince the editor of the importance of the article. As you write up this summary, take into consideration the suitability of the article for the journal. If you are reviewing for the top journal in your field, for example, an article simply being factually correct and having a sound analysis is not enough for it to be published in that journal. Instead, it would need to change the way we think about some aspect of your field.
Step Four: Write out your major criticisms of the article. When doing a peer review, I usually begin with the larger issues and end with minutiae. Here are some major areas of criticism to consider:
– Is the article well-organized?
– Does the article contain all of the components you would expect (Introduction, Methods, Theory, Analysis, etc)?
– Are the sections well-developed?
– Does the author do a good job of synthesizing the literature?
– Does the author answer the questions he/she sets out to answer?
– Is the methodology clearly explained?
– Does the theory connect to the data?
– Is the article well-written and easy to understand?
– Are you convinced by the author’s results? Why or why not?
Step Five: Write out any minor criticisms of the article. Once you have laid out the pros and cons of the article, it is perfectly acceptable (and often welcome) for you to point out that the table on page 3 is mislabeled, that the author wrote “compliment” instead of “complement” on page 7, or other minutiae. Correcting those minor errors will make the author’s paper look more professional if it goes out for another peer review, and certainly will have to be corrected before being accepted for publication.
Step Six: Review. Go over your review and make sure that it makes sense and that you are communicating your critiques and suggestions in as helpful a way as possible.
Finally, I will say that, when writing a review, be mindful that you are critiquing the article in question – not the author. Thus, make sure your critiques are constructive. For example, it is not appropriate to write: “The author clearly has not read any Foucault.” Instead, say: “The analysis of Foucault is not as developed as I would expect to see in an academic journal article.” Also, be careful not to write: “The author is a poor writer.” Instead, you can say: “This article would benefit from a close editing. I found it difficult to follow the author’s argument due to the many stylistic and grammatical errors.” Although you are an anonymous reviewer, the Editor knows who you are, and it never looks good when you make personal attacks on others. So, in addition to being nice, it is in your best interest.
Tanya Golash-Boza is Associate Professor of Sociology and American Studies at the University of Kansas. She Tweets as @tanyagolashboza and has her own website.
Anna Tarrant, Peer Review, Pitching & Publishing, publishing, Tanya Golash-Boza, Writing
I. Thinking About Your Literature Review
The structure of a literature review should include the following:
- An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
- Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
- An explanation of how each work is similar to and how it varies from the others,
- Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.
The critical evaluation of each work should consider:
- Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
- Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
- Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
- Persuasiveness -- which of the author's theses are most convincing or least convincing?
- Value -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?
II. Development of the Literature Review
1. Problem formulation -- which topic or field is being examined and what are its component issues?
2. Literature search -- finding materials relevant to the subject being explored.
3. Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic.
4. Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.
Consider the following issues before writing the literature review:
If your assignment is not very specific about what form your literature review should take, seek clarification from your professor by asking these questions:
1. Roughly how many sources should I include?
2. What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)?
3. Should I summarize, synthesize, or critique sources by discussing a common theme or issue?
4. Should I evaluate the sources?
5. Should I provide subheadings and other background information, such as definitions and/or a history?
Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read are also excellent entry points into your own research.
Narrow the Topic
The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make your job easier if you first limit scope of the research problem. A good strategy is to begin by searching the HOMER catalog for books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text.
Consider Whether Your Sources are Current
Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.
III. Ways to Organize Your Literature Review
Chronology of Events
If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union.
Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies.
Thematic [“conceptual categories”]
Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it will still be organized chronologically reflecting technological developments in media. The only difference here between a "chronological" and a "thematic" approach is what is emphasized the most: the role of the Internet in presidential politics. Note however that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point made.
A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.
Other Sections of Your Literature Review
Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you but include only what is necessary for the reader to locate your study within the larger scholarship framework.
Here are examples of other sections you may need to include depending on the type of review you write:
- Current Situation: information necessary to understand the topic or focus of the literature review.
- History: the chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
- Selection Methods: the criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed articles and journals.
- Standards: the way in which you present your information.
- Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?
IV. Writing Your Literature Review
Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.
A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid.
Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information but that are not key to understanding the research problem can be included in a list of further readings.
Use Quotes Sparingly
Some short quotes are okay if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for your own summary and interpretation of the literature.
Summarize and Synthesize
Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work.
Keep Your Own Voice
While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording.
Use Caution When Paraphrasing
When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.
V. Common Mistakes to Avoid
These are the most common mistakes made in reviewing social science research literature.
- Sources in your literature review do not clearly relate to the research problem;
- You do not take sufficient time to define and identify the most relevent sources to use in the literature review related to the research problem;
- Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
- Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
- Does not describe the search procedures that were used in identifying the literature to review;
- Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
- Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.
Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination. Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques. London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach. Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students. 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review. Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.