Monthly Archives: October 2018

Writing a literature review: Identifying ‘gaps’ and pointing out your ‘contribution to knowledge’

Mind the Gap

Photo credit: Jeremy Segrott 

Introduction

I recently received a literature review chapter from a student, and in my feedback noted more needed to be done to helped identify the ‘gap’ in the literature that her work is seeking to fill. My student responded that she understood the request but wasn’t sure about how to go about doing that in her writing, which is fair enough. Rather than write an email to a single student on the topic, which would benefit one person, I felt it would be better to write out my thoughts on this matter here, in the hopes that what I have to say can benefit more than just one person.

I start with some more abstract critique of the concept of a ‘gap’ in the literature then follow with what I refer to as different ‘levels’ of gap/contribution identification. Finally, I finish with some more general advice for thinking about and writing literature reviews.

The feminists already critiqued it: The patriarchy of the ‘gap’

The first point I feel it’s important to raise is that the term ‘gap’ is a misleading one. It implies that there is some void in understanding which the gallant author is going to plaster over through their investigation, making a fuller, safer understanding of reality for everyone. There are several problems with this conception of the production of knowledge within academic research, most (all) of which the feminist literature has already pretty roundly critiqued. Please note that my academic specialty is not feminism. While I’ve read around this topic, my understanding of the nuances of all of the argumentation I’m going to reference is incomplete. Also, I read most of the material that I’m basing my summary off of years ago, and the sources are lost in the mists of time. I really should include citations below, but I wouldn’t know where to start to track down where the specific ideas originate from. If you would like to suggest relevant citations, please get in touch with me and I’ll consider adding them

As I understand it, the gist of the feminist critique of viewing the production of knowledge in terms of filling ‘gaps’ is that it sets up a relationship of opposition between the author seeking to contribute new knowledge and those who have contributed knowledge before them. If the requirement is that you find a ‘gap’ in understanding, then by definition as an author you are required to explain what those who came before you got wrong or didn’t understand. In the worst case scenarios, this results in text that implies the literature being reviewed is somehow ignorant of the facts and produced in error, and that the author seeking to explore the ‘gaps’ identified is basically filling in for the mistakes of those who came before them.

There are several problems with the scenario that develops in such cases. The first is that the author paints themselves as seemingly superior to those that preceded them, earlier research is characterized as misinformed and incorrect, and new research is lauded as a kind of inevitable march toward progress. This ends up representing the author as a patriarch (whatever their gender) with the informed perspective of the modern age who has the duty of correcting the mistakes of earlier researchers.

However, reality, with all its flaws, is almost never as clear-cut as that; earlier authors almost always help to make more current research possible, and influence what is and can be researched in various ways. Also, if the author is a student writing for assessment, they are likely not going to be able to make a strong case that the expert authors who precede them are universally mistaken in their understandings and that it’s the student, new to the field, who is going to provide the answers to contemporary questions and problems.

So what to do? Well, one option is to see the literature review as outlining your ‘contribution to knowledge’, which doesn’t necessarily require the framework of opposition implied in ‘gap’ finding. But I personally feel the phrase ‘contribution to knowledge’ sounds overly bombastic; I find it hard to take myself that seriously. I am, after all, the same person who put a plastic plate in the toaster oven when I was twelve years old. I don’t want the world’s knowledge to be riding on the adult that kid became.

Hopefully in this section I’ve at least outlined the tensions inherent in seeking to find a gap in the literature or outline the contribution to knowledge that you are seeking to make through your writing. There’s a kind of fine balancing act required between pointing out what previous research has accomplished, while at the same time leaving space in a review of the literature for your specific writing/investigation to make its own contribution. In the next part of this post I’ll consider some different strategies for identifying specific gaps or contributions. I’ll try to outline them from less to more critical, so you can hopefully see a scale of thinking about the topic emerge.

Identifying gaps in the literature or contributions that you hope to make

I’ve included some subsections to break apart my text a bit and to hopefully help you to see a progression from a generally less critical contribution to knowledge framing to a generally more critical contribution to knowledge framing. I think it’s worth pointing out that you don’t necessarily need to identify only one gap. Particularly if you are doing a complicated and involved investigation, like a master’s dissertation, there is likely to be more than one gap or contribution that your investigation can make. Pointing out multiple gaps/contributions can help to better situate your investigation in the literature you review and demonstrate greater critical thinking regarding the topic of your investigation and your framing of your investigation.

Disclaimer: All of the examples of text used below are made up (the indented text). I’ve done this purposely, as I don’t really want to look like I’m criticizing specific authors. I also don’t want to spend too much time looking up articles I read or wrote years ago; I feel like the examples as I’ve included them serve their respective purposes well enough.

The simplest gap: No one has ever done this before

Some may disagree, but I feel this framing of an investigation tends to be the least critical. You find some context that the literature has not previously investigated, and you investigate it. This is one of the more common framings that I see in the assignments I comment on and mark. It tends to look something like:

While English language learners’ attitudes toward differently accented English has been investigated in Japan and China, the attitudes of students in Thailand have not been described to date. Thus this research investigates Thai high school students’ attitudes toward differently accented English in order to address this gap.

This strategy can end up getting pretty specific. For example:

While the attitudes of Japanese learners of English toward differently accented English has been investigated at the junior and senior high school levels and in universities, primary school pupils’ attitudes have not been described in the literature. To address this gap in understanding of Japanese primary school students’ attitudes toward differently accented English, this investigation was conducted using 100 pupils from 4 primary schools located in the Tokyo area.

While the example above is an arbitrary one, I think it makes the point relatively well. This perspective of dividing the world into thinner and thinner slices of knowledge pie can get pretty silly when taken to extremes. Here’s what I hope is a light-hearted illustration of this:

While the shopping habits of white-shirted and black-shirted consumers have been described, the literature has not to date investigated the full variation of colors of clothing that consumers wear and the potential influence these colors may have on their shopping habits. To investigate this issue, the current investigation uses the Pantone color scale to correlate consumers’ shirt color with shopping behavior.

It’s pretty easy to image follow-up studies of pants colors, two-tone and multi-toned attire and shopping behavior, etc., all with an eye toward dividing the knowledge pie up into infinitesimally smaller pieces.

So what does a critique of this gap framing look like? Well, one issue is that, particularly in language research, context tends to be defined quite uncritically as bounded by country or nationality (‘Japanese’, ‘Chinese’, ‘Mexican’ learners) and/or by demographic category (primary, junior high school, high school, university, secondary, tertiary learners). Sometimes using framings such as these is useful and necessary, but I think it’s almost always in the best interests of the authors using them to include some acknowledgment of their shortcomings and to justify their reasoning for using them. For example, if someone is interested in issues of teaching English to Japanese high school students, then investigating Japanese high school students makes sense, but such a framing would be more practice-oriented, which I discuss further below.

Another critique is that the framing is a fragile one. If a reviewer/reader says that another author has already used this framing, then the argument falls apart that it is necessary to conduct the newer investigation at all. This can lead to considerable stress for students/researchers who while they write up an investigation worry about whether someone is going to beat them to the punch or if they’ve missed some paper in their review that already did what they are doing. The implication is also that once the investigation has been done, the case is going to be closed on the group investigated. However, people are hardly ever this simple, and researchers are constantly finding that what (at least initially) appear to be clear-cut demographically-framed findings aren’t as clear-cut as they first appeared following later investigations.

John Swales critiques this kind of framing with respect to English for specific purposes research in a 2000 paper that’s worth following up on if you’re interested in reading more on the topic. Here is the link to the journal article’s site, and here is the direct link to the article PDF via ResearchGate.

A higher level of abstraction: There are assumptions being made in the published literature that the current investigation examines

I made a (very small) name for myself investigating an aspect of task-based language teaching using this argument. In my reading of the literature on task-based language teaching, there was often a statement to the effect that teachers shouldn’t pre-teach language phrases, as that could somehow contaminate students’ language when it came time for them to complete a language task. However, this claim in the literature was always very carefully couched in hedged language and almost never included a citation to some literature that demonstrated the influence of the pre-teaching of language on students’ task performance, and so I gradually became interested in investigating the issue. To my knowledge, the jury is still out, by the way, just in case you’re looking for a topic to investigate yourself. I feel like I’ve since moved on in my own research interests.

Another example of this comes from my PhD research; one author, Gosden (1995), makes a claim that certain sections of papers are revised more than others during the process of review and revision. However, he makes that claim as a justification for only looking at those sections in his own research (rather than whole manuscripts) and the claim doesn’t include a citation. So, if one investigates entire manuscripts for the changes made to them, one could evaluate Gosden’s claim. That’s one of the (relatively minor) claims to a contribution to knowledge that I make in my PhD thesis.

This demonstrates an additional level of criticality compared to the ‘no one has ever done this before’ gap claim above, in that there is engagement with the arguments being made in the literature, and some effort to point out how the current investigation intends to contribute to those arguments. I feel it still has the flavor of generating some tension between the newer investigation and those that precede it, but it doesn’t necessarily have to directly fault older studies. For example, from my task-based learning research, I could write something like:

In task-based language learning guides, providing language for learners before they are asked to complete a task is generally discouraged, as it may influence students’ task performance (relevant citation). However, the influence of providing pre-task language for students has not been investigated to date, and so the current investigation seeks to examine how pre-task language influences task performance in the English language classroom.

There’s still a bit of pie slicing of knowledge going on, but this is more directly related to ongoing conversations in the literature than those in the previous section. Also, the specific demographics of the learners/participants aren’t necessarily important to the investigation; the issues being explored are what’s important in this instance.

Arguing for the practitioner-value of new knowledge: Getting around the it’s-already-been-done dilemma

In this framing of a contribution, context is more carefully defined to be very specific, and the argument that’s made is that the contribution to knowledge is a contribution to practitioner knowledge within that specific space. For example, in a book chapter I wrote about how doing extensive writing with high school students in Japan led to an improvement in their writing speeds relative to students who didn’t do the extensive writing (Implementing and Evaluating Free Writing in a Japanese EFL Classroom). When I moved to work at the University of Toyama, I was interested in whether the same exercise would result in similar improvements among my students here. (It didn’t.) In this case, the knowledge I’m seeking is of interest to me as a practitioner (teacher), as it can help to inform my pedagogy. The relationship I can draw to the literature is the evidence of the benefits of extensive writing generally, which I seek to apply to my particular teaching context specifically. This gets around the it’s-already-been-done dilemma because the knowledge pie is sliced according to practical applications for practitioners rather than according to more arbitrary demographic delineations.

Note that the person doing the research doesn’t necessarily have to be the practitioner; they could be doing research with the intention of informing the teaching practice of others. The point remains, though, that they are seeking to find locally useful insights through the investigation.

One common framing for this kind of investigation is ‘action research‘, although action research isn’t the only framing that could potentially be used (and tends not to be the framing I use in my own writing).

A higher level of abstraction still: Contributing to and answering debates in the literature

At this level of contribution, the author is engaging with various ongoing debates in the literature and seeking to make their own contribution to those debates. For example, in my PhD thesis I note that there is an ongoing debate in the writing for academic publication literature around whether authors whose first language is not English are treated differently from authors whose first language is English when their manuscripts are reviewed. There are those contributing to the literature who note that in reviews, first language is raised as an issue in reviewers’ comments on manuscripts, and there are others who claim authors’ first language is not an especially important factor when trying to write for academic publication. However, the research behind these debates tends to involve investigating the writing for publication of authors whose first language isn’t English. Authors whose first language is English tend to not be included in such investigations. So in my research, I include both Japan-based Japanese writing for publication and foreign residents of Japan writing for publication, which facilitates my examining the extent to which language background may play a role in the reviews of those authors’ work and evaluations of their manuscripts. I certainly don’t think my thesis offers a definitive answer to the debate, but I do feel that it contributes to it in some useful ways.

This is not an easy level of contribution to engage at, but it is one that I feel demonstrates critical engagement with the literature and a clear vision for how a given piece of writing fits into the larger literature in which it is couched.

Seeking new perspectives: Shifting the paradigm toward new ways of looking at things

My favorite example of this kind of contribution is ‘appreciative inquiry‘, which is framed as a critique of and response to action research (which I mention above). My memory of the reading I did around appreciative inquiry is that it critiques the assertion in action research that the action researcher should focus their investigation around a ‘problem’. Appreciative inquiry argues that the researcher should seek to understand the larger context/situation without first problematizing it. In so doing, the researcher can see the positive aspects of a given research context in addition to the aspects that can be improved. The argument made is that this makes a given appreciative inquiry project more appealing to practitioners and stakeholders, as they can feel their whole situation is being attended to, rather than the researcher seeing them as being/exhibiting a ‘problem’ that needs to be fixed.

This kind of framing seeks to show how a given issue is being explored in a new light, offering a contribution by asking a different kind of research question. One strategy for seeking to make this kind of contribution is to integrate concepts from another field into the one you’re writing for, such as incorporating theories around negotiation from the business literature into an applied linguistics investigation.

Summary

Please don’t feel that the list above is comprehensive by any means; these are my own ideas about this topic, and if they help you to think about how to frame your own writing, then I’ll feel like I’ve done my job. Giving a comprehensive account of how students/authors can seek to contribute to knowledge or fill a gap in the literature would be well beyond the scope of a single blog post.

The last topic I’ll cover here is the importance of consciously deciding what (not to) include in the boxes you draw around the literature and how to go about drawing those boxes in convincing ways.

Defining the literature: The importance of drawing boxes around what you’re covering (and what you’re not)

An issue that I haven’t addressed in my discussions above is how to go about framing the ‘literature’ you are seeking to contribute to. This is not an easy or trivial task, and would benefit from a lot more space and consideration than I can give it here.

Suffice to say, one way to demonstrate criticality in writing is how the literature reviewed is framed. Also, how the literature is framed in your review, to a large extent, either limits or opens up the degree of knowledge contribution that can ultimately be claimed in a given sample of writing. If the objective is to contribute to ongoing debates in the literature, then it’s going to be important to identify and characterize those debates in the review, and point out the specific questions the new investigation is seeking to contribute to. A given open question from the literature doesn’t have to be answered outright, but the paper can/should seek to add some additional information or perspective that will help to inform others’ understanding of the question addressed.

If seeking to shift a paradigm or a way of thinking about a topic, it is important to first outline current ways of thinking and to note how those ways of thinking are similar to and different from the direction pointed to in your writing. You don’t necessarily need to ‘invent’ a direction, either; appreciative inquiry authors have a kind of read-made framing they can borrow, contributing further to appreciative inquiry as a field through their specific investigation. The argument about how appreciative inquiry is different from action research needs to be restated (with appropriate citations), but a given individual author doesn’t need to reinvent the argument/define a new direction with every paper they write.

One way to characterize and describe the boxes you draw around the literature is through attention to the methods of research and the issues addressed. Is the literature primarily text focused or participant focused? Are the studies primarily trying to characterize opinions/impressions or to describe experiences? Is there a focus on change over time or a focus on a single set of responses? Not everything fits neatly into boxes, but you can acknowledge that the boxes you draw are flawed while still trying to group various similar studies on a topic together. Acknowledging the flawed nature of the categories you use to characterize the literature is itself a way to demonstrate your critical thinking about the topic of discussion.

I would also recommend you watch how other authors characterize the literature they review as you do your readings and see what from their framings you can incorporate into your own writing and thinking (with appropriate citation). If someone else has done the work of characterizing the literature for you (for example, the language learning motivation literature has been pretty thoroughly summarized by Dörnyei), then you don’t need to invent a new framing. Rather, you can note where the literature you review fits into the pre-existing framing and (ideally) note how your research contributes further to understanding somehow. If you can critique some aspects of the framing, that’s even better in my opinion.

Finally, one thing I didn’t cover here is the ‘comprehensive literature review’. That’s where an author/researcher picks a topic and keywords, then does a database search for all the literature, examining all/most of the hits returned. That’s not something I’ve encountered very often in applied linguistics, and isn’t something I would recommend that students or beginning authors try to cut their teeth on. It tends to be more popular in the library sciences and in the biological sciences in my experience, and is too far outside of my realm of expertise to comment on very extensively. I think it’s worth saying, though, that in an applied linguistics review you don’t have to use language to try to sound like you’ve covered everything. A generally more successful strategy is to claim that, largely the literature can be fit into a finite number of boxes, and to characterize the boundaries of those boxes, the broad topics/themes that they cover, and what from that literature your particular study is hoping to contribute to.

I think this is all I have to say on this topic for now. I hope you’ve found this useful. If there’s something you think I missed or that you feel I could/should elaborate on, please let me know.