Qualitative data analysis (2012)

Cloud created by:

Rebecca Ferguson
9 March 2012

Analysis is central to the research process, it is the pivot on which much else turns. In qualitative research it can take a variety of forms, though all analysis must be attuned to the particular research questions and data involved. In this session, Martyn Hammersley outlined a distinction between theme analysis and discourse analysis, and participants explored what is involved in the initial stages of carrying out the first of these, since this is probably still the most common approach employed by qualitative researchers.

Led by Professor Martyn Hammersley.

Extra content

I'm live-blogging Martyn Hammersley's workshop on qualitative analysis at the OU Nuts and Bolts of Research (Methods) Conference 2012 (Tuesday 20 March 2012, 11:00 - 12:30).

Martyn Hammersley
CREET/CHDL
Faculty of Education and Language Studies

There are 15 members in the audience. The facilitator provided handouts of the slides and reading notes: "Qualitative analysis: a brief reading guide". 

(11:00)
The session starts on time.

Clarifications on who the audience members are: mostly first year PhD students.

In terms of probationary report preparation, the first slide seems a good starting point. Description of the report process and its aims. Properties of the probation report. Research question. Literature review is sometimes included, but it can never be the final version you will use in your thesis. Methodology and a plan.

Focus on data analysis. It is not something you can leave to think about later on in the project. The kind of analysis affects the data collection, so some thought is needed in advance. It is hard to think about it, especially at the beginning. Research progresses so this has to be adjusted constantly.

Still, you can think of what are the analyses available, what kinds of data you might look at, what the tools are, etc. And think about this recurrently over your project.

The aim is to build up a working understanding of the data: a set of resources of what you can do with your data.

"I needs to be fuelled". It is not passive learning, but active, wondering at all times how this is useful to you. Analogy with literature review, it's not just reading, but reading for a purpose, to build a set of resources in your head that you can use along your research. Example from his PhD research in schools.

Analysis can not be done by following a set of rules or a recipe. You need to see how any methods or guidelines apply to your data. Thinking backwards and forwards.

What is the aim? Answer to research questions. Descriptions vs explanations.

Slide on methodological paradigms/philosophies that have influenced qualitative research. The list is there to give a sense of the diversity of quality research/analysis. It is a very complex set of different approaches to what you do and how you do it. The literature is endless. You can not read it all. You must get a feeling of what is there and then work out where you focus; find something that suits you and then go back to the names, to the abstract paradigms. It is better to adopt a more concrete approach, rather than tying yourself to an abstract philosophy.

Now to the more concrete…

You start with some idea/puzzle/group of people/question that interest you, and use those as resources to make sense of your data; but you need to carry on an open-ended exploration. You shouldn't let yourself be limited by your initial motivations, but use them to guide you.

Suspending your priors: "what would Martians think of this?" But that approach is a bit extreme. Balance.

How do you collect unstructured data in practice? And once you collect it, how do you organise it and start making sense of it? That is at the core of qualitative analysis.

Research questions are not the same as question in an interview. The questions in an interview are a means to obtaining data to answer a research question, but the assumptions behind the research question should not be reflected in the interview questions.

Data are not merely collected, they are produced. Field notes have to be written down, interviews have to be recorded and transcribed. Example from his work, when in the field: writing down the notes takes most of an evening. From notes taken while in the field, to an organised account of those notes plus memories and some interpretations. A 1-hour interview would take 5 or more hours to transcribe. If you want to do discourse analysis then you need to capture hesitations and pauses. So the ratio is 7 or 8 to 1, or even 12 to 1. All this has to be taken into account when deciding how much data to collect.

What is analysis? You are always interpreting data. In doing analysis you have to ask questions of the data, rather than just passively reading it. Why did the participants say this rather than that? Why did this person say this and not that person? Why did I ask that question at that moment? Reflection is also a part of the analysis. Your role in the process.

You start by generating ideas and resources without much limitation to see what emerges. Then you analyse it by looking at what is relevant to your research questions.

Forms of data for qualitative analysis. Data can be in different forms depending on the paradigm you follow. Most of it takes textual form, but some analyses need visual data or a combination of both. Some people are arguing for a broadening of the forms, so that the researcher engages with the data using all the senses. Why focus on one particular form?

Theme analysis crosses all data regardless of the sources and tries to develop conceptual categories that explain the phenomena.

Discourse analysis is usually focused on one particular type of data: documentary, naturally occurring conversation (e.g. doctor-patient, casual chats, classroom). Low-level detail: choice of words, speech acts, timing, turn-taking. Smaller amount of data because of this micro-analysis.  It produces interesting amounts of analysis from a small amount of data.

Types of discourse analysis: not covered, just there for informative purposes. There is a course at the OU with that title, check on it if interested.

Stages of qualitative analysis:

1. Coding the data: it's not just passively labelling fragments of data. The researcher has to come up with an adequate set of categories and assign them consistently to parts of the data. This is already making sense of the data, it's the beginning of interpretation.
 
2. Comparative method: once the data has been coded, what do the categories mean? are they exclusive? are there any gaps? is there a better way?

3. Checking interpretations and conclusions: what do I get from it? can this be generalised? is it consistent with my hypotheses?

(11:30)
Practical exercise

Handout of an interview transcript (material produced for an OU course for students to perform analysis). The idea is to apply the first step to this transcript and a video of the interview.

My impressions while watching the video and reading the transcript:
The 15-minute interview is about the children of the interviewee: number of children, where they spend their day, whether the interviewee is single, views on childcare (asking for personal opinion; very long answers here) and contrast with nursery. The interview is quite relaxed and friendly; the interviewer makes comments empathising or agreeing with the interviewee. Creating rapport (?). The transcript includes some disfluencies, but no pauses, interruptions or overlaps.

Some observations by Hammersley:
The interview is about a particular setting (the childcare setting) which exists independently of the interview. You have sound and video (two cameras and professional sound), and a transcript (that you generally have to do yourself). If you do theme analysis, you look at how she came to decisions, the opinions she presents, the assumptions that might be in the questions, etc. The transcript is not adequate for discourse analysis, for instance, as it doesn't have hesitations and precise timings for pauses, etc. You also don't know what is happening outside what is seen in the video. The point is to see how data collected/produced in some way might not be suitable for all sorts of analysis.

The projection of the video is followed by 10 minutes in which audience members are encouraged to do themselves an analysis of a couple of pages of the transcript trying to find out how she came to have the opinions she has regarding childcare.

First part explains the interview procedure and its aims. Second part asks about the particular setting of the participant with respect to childcare: number of children, spouse, work, time at home and at childcare, etc. Third part looks into views and opinions of the participant regards childcare.

(12:00)
Interaction with the audience after the analysis.

What categories have you found?
Audience member: home environment is repeated 4 times in the first 2 pages.
MH: it doesn't seem to come from the questions, but from the informant.

Audience member: the first page is about demographics.
MH: right, this is laying the grounds of the setting.

MH: You are not only looking at what set of criteria this person used to decide on childcare, but also at how these criteria relate to other beliefs and aspects of her life.

Audience member: trust?
MH: yes, that I wrote in big letters, to come back to it later on. It seems to be essential, but what does that mean to the participant in the setting she described (unknown, young age of children, anxiety, etc)? Peace of mind? We need to see how the criteria fit in the other aspects of her life.

Audience member: for the setting, big and small, social and educational seem to be relevant categories.
MH: yes, this is recurrent in anything related to children.

MH: so, once you have your first categories, you need to go back to the beginning and go through it again to see things that you might have missed. Because you change the first time you do it. You learn things, you get ideas, you discover new things and the second time you have a different view.

Boring versus interesting data; sometimes it is not about how you approach it, but what the data contains.

Computer programs can help you manage the data, especially when you have large amounts of it. They won't analyse it for you, but they can store it and retrieve it, and assist in several tasks. For instance, some programs can retrieve all data coded in the same way.

There are several packages. You need to think what is it that you need from it and how long it will take for you to get familiar with it.

And that is it...

Questions

Q: regarding categorisation, what do you do once you have the categories? You just assign fragments of data to them?
A: not all categories are the same. In the example, you had home environment, mixing with other children, enjoyment, education, proximity… As soon as I realised she was mentioning choice criteria, I was looking for others. Trust is a bit different, it relates to the nature of the decision, not the factors that influenced her decision. Now we have a little model: first you have the setting, the situation that define the problem making it difficult, the options they have; then you have the factors affecting the decisions, the criteria; and then you have an aim (trust, peace of mind) that let them know what options are better based on the criteria and the end result. Then you can write it up.

Q: is this the same process you use in your ethnographic field notes?
A: you can use this procedure for field notes or for analysing video or audio data. You can also use it to analyse yourself as an interviewer or your role in the acquisition of the data. The difficulty when you are analysing yourself is distancing yourself from your role in the data. You have to find an angle where you can do that, and this can be hard to do.

Q: to what extent should the transcription be verbatim?
A: what you have in the example is what you get in lots of qualitative research. Hesitations, for instance, sometimes can be irrelevant, but sometimes can have a meaning. For discourse analysis, for instance you would need most of the detail, as much as possible. For theme analysis, this becomes problematic as many details are not required. Word for word you do need at least for the initial analysis. If later on you are confident with your approach, but have so much data you can not analyse it all, then you can decide what parts you want to transcribe, annotating the resource with what is in the untranscribed bits. Still, because your mind changes along the research project, you risk missing something if you do selective transcription, as you might initially leave something out that you then might have considered relevant and it might get lost in the data. Especially at the beginning, you should transcribe everything you collect.

There is also the risk of collecting too much data. This is better than collecting too little. Still, having too much data can become unmanageable. Specially if you have not a clear criteria for selecting what to use and what not.

Back to the probation report…

With qualitative analysis you have to think that you can get out a lot out of a little data. This must be had in mind when deciding on how much data to collect.

Be realistic. Ambition is good, but you need to be able to do what you propose doing. You can leave things for doing later. 

Recurrent assessment is essential. At all times, you must ask whether the direction you are following is still realistic. Data collection can change that. Every instance of data collection can change the answer to that question. Data collection can go wrong, answers can be meaningless.

Uncertainty is essential. You make decisions early on, and they have to be the best decision with the information you have, but the good thing is that there is very little that can not be changed later on.

Q: how do you make sure that the quality of your analysis is good?
A: the answer is long, but in short, you go back to the literature and the methodological philosophies mentioned earlier. There are guidelines for each of these approaches. Still, these analyses and criteria are open to interpretation, so what counts as evidence, what is extensive analysis, etc., can be debatable. They can be too general, but then when you go to the specifics of your theoretical resources you should be able to tell what makes sense in your field. Once you get there, have a look at what other people do in the area and see if you can do something similar. Talk to supervisors. Your sense of what is good analysis evolves with time. First you have to be certain and satisfied with the quality of your analysis and your data, and the results they are contributing. Then you have to feel the same about your work when placed in the context of your field and when it is communicated to others. You have to think if there would be different interpretations, if someone coming from a different direction would have made a different analysis, if there is another way. You might not be able to provide a set of precise steps to prove that your analysis is of quality, but you can certainly make judgements of what's the most convincing way of presenting your arguments.

(12:35)
The session ends.

Brian Plüss
14:04 on 22 March 2012

Handouts of the session:

Brian Plüss
09:28 on 29 March 2012 (Edited 09:32 on 29 March 2012)

Embedded Content

Workshop Slides

Workshop Slides

added by Brian Plüss

QSR International - on-line tutorials on using NVivo and advice on qualitative research

QSR International - on-line tutorials on using NVivo and advice on qualitative research

added by Dorothy Faulkner

Contribute

Dorothy Faulkner
7:28pm 21 March 2012


Some of you asked whether there was any training on how to use the qualitative analysis software NVivo. There are no courses running at the OU currently but I have put up a link to the QSR International website. QSR International are the team who developed NVivo. There are several on-line tutorials that you can work through at your own pace. Also, once you have a basic understanding using the tutorials - a really good strategy is to ask for some informal tutoring from someone who already uses NVivo. Your Faculty may be willing to fund you to go on an external training course. The next QSR event is a two day training conference in London next week (27th - 28th March - £275 for students) se the training and events section of their website. This is repeated on the 24th- 25th April.  Alternatively, QSR run a number of on-line training workshops.

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.