Tuesday, March 2, 2010

Content Analysis: Next Steps

You're proposing to do a content analysis. Dr. Hassett-Walker has approved (or approved with revisions) your study. Now what?

Here's what you do next (i.e., beginning your study):

(1) Do the search of your selected content. Ex: Searching in Westlaw for past articles in the NY Times and the Washington Post, during the years 2000-2010. Your search terms are "university" and "campus" and "violence" and "victims" (for a study - my hypothetical example - about the effect of violence on college and university campuses on primary and secondary victims).

Note: your search terms will be similar, probably, to some of your code list words.

Suppose 8,547 articles come up. You don't need that many. You need 100, but will randomly select 200 to account for some articles (let's say every other one) being non-relevant or less relevant.

(2) At Randomizer.org, generate a random number list. I just did this, and told Randomizer to give me 1 set of 200 randomly generated numbers, ranging from 1 through 8,547. Here's part of the list it gave me: 49, 2387, 4163, 3955, 4424, 2349, 77, 2422, 3480, 128, 5972, 1579, 3784, 2312, 624, 5602, 876, 2977, 2948, 6856, 2736, 7490, 2117, 7590, 4569, 2844, 2071, 5033, 6005, 2112, 4073, 3580, 7596, 7362, 7066, 1793, 5914, 2815, 7584, 3384, 118...

(3) Next, I find those articles that correspond to the numbers in the random number list. I copy/paste them into a Word or Word Perfect document.

(4) Now I begin reading the articles. Working in Word (rather than on paper), I use the Word artist function to color-code red all words and phrasings related to violence and shootings (e.g., shooting, gun, shots fired, etc.). I color-code green all words and phrasings related to victims (e.g., victim, victims, witness, students standing there watching, etc.) I color-code yellow all words and phrasings related to college or university campus (e.g., campus, college, university, classrooms, buildings, etc.). I color-code blue all words and phrasings related to programs the university or college put into place to help victims.

(5) I set up a database skeleton in SPSS. Before I can enter any data into the database, I must create my variables. These will correspond to my code list. EXAMPLE of variable name (variable #1) and corresponding label: ArticleNum, which newspaper did it comes from (1=NYT, 2=Washington Post). Next variable name: VictimNum, number of times victim or victims mentioned in article. Next variable name: VictimLang, language used to describe victims. Etc.

Note: any variables that involve counting will be numeric variables. Any variables that will have me inputting words will be string variables.

Remember: You set up your SPSS database in variable view.

(6) Once I have my SPSS skeleton set up, I can enter my data (words, language, counts of words) into SPSS. I can either do this once I'm done reading all the articles, or as I go. Remember: you enter your data into your database in data view.

(7) Once you've entered all your data, you run your preliminary and "main" analyses: frequencies, descriptives, and perhaps a correlation analysis. Review your results with Dr. Hassett-Walker. Do you think you are finding support for your research question or hypothesis?

(8) Once you have determined which relationships are significantly related to each other, go back to the color-coded newspaper articles that make up your data. Search for the text coded with the code words that pertain to those statistically significant relationships. That's the text you want to read and perhaps copy/paste into a separate word document. Boil it down into a succinct one to two-paragraph summary/summaries that further explains the statistically significant relationships you found with the quantitative data.

Survey and Focus Group Methodology, Analyses, Results

For the students doing the survey and focus group option, once I've approved (or approved with revisions) your study, you're ready to start.

(1) Print out your consent form on Kean letterhead, which you'll get from me. You and I both sign in the appropriate places. Then you make around 50 copies (two for each student/human subject, one of which they sign and return to you, the other of which they keep).

(2) Make 25+ copies of your survey instrument as well. Your subjects will be filling this out, so it has to be in a form that's ready-to-use for them.

(3) Next step: subject recruitment. You should have already been thinking about this, and/or written it down as part of your method section. Where and when will you find your fellow Kean (NO OCC students!) students? Student Center? Library? Around campus? How will you approach them? If they say yes, you should hand them a consent form to sign. Once they've read, signed, and returned the consent form (keeping a copy for themselves), you'll need to take them to the location where you'll conduct the focus group. Plan and pick your location ahead of time. It should be somewhere you're allowed to be (empty classroom? Empty study room in the library?) that is quiet and has access to outlets, if your tape recorder needs to be plugged in.

(4) Equipment: Also plan this ahead of time. You need tapes that run long enough (1 hour should be good), maybe extention cord, extra batteries, and perhaps an extra tape recorder.

(5) Visualize where you will put the tape recorder, and how your subjects will sit around it. They should sit in a circle, with the tape recorder in the middle.

(6) During the focus group, periodically check that the tape recorder(s) are running, that they haven't timed out.

(7) Immediately after each focus group, spend some time writing down everything you can think about that happened during the focus group. What was the mood? Was everyone equally talkative? Were some students (probably) more talkative than others? What were some of the key points discussed? Write as if you might discover that neither of your tape recordings worked, so you want to be as thorough as possible in writing down what went on and what was discussed.

(8) Once you have all your surveys collected and focus groups conducted, you enter the survey data into a SPSS database that you create. (Email Dr. Hassett-Walker if you need help setting up a skeleton for this.) You'll also spend a few hours typing up everything that's on the tapes. (Play-stop-rewind, play-stop-rewind, play-stop-rewind.... Yes, it's a tedious process.) In the end, you should have a transcript that's perhaps 60-75 pages long. If you ever took a drama class in high school, what you produce will be similar to a script for a play. Note: You don't have to capture every "um" and "ah."

(9) As a reminder, the Krueger book on conducting and analyzing focus groups is on reserve at the OCC library.

(10) Once all your survey data are entered into SPSS and your focus group transcripts are typed up, you are ready to begin analyzing your data and summarizing your findings. You are doing quantitative (of the survey data) and qualitative (of the focus group transcripts) analyses.

(11) The quantitative analyses: this will be the focus of our lab visits in late March. That said, you already know what these are, as you've run them in Methods and Seminar: frequencies, descriptives, correlations and crosstabs with a chi square test, as appropriate. Also refer to p.11 in your course syllabus.

(12) The qualitative analysis: This will basically involve reading the transcript, and determining what group #1 said about question #1, what group #2 said about question #1, what group #3 said about question #1, etc. If it's easier, you can open up a separate Word (or Word Perfect) document and copy/paste text from the original transcript into that new document, under the subheading of (for instance), "All groups' feedback on question #1." So then you have all your text about question #1 in one place; then all your text about question #2 in one place; etc. You want to read through it, summarize it, shorten it so that you have a short, concise 1-2 (or longer) paragraph summary of the key points and themes from the focus group discussions, and a few quotes that "pop", that describe what your subjects felt about the issue that was question #1 (and question #2, and question #3, etc.). Ask yourself how the focus group discussions (your qualitative data) reinforce, condradict, or further clarify your survey findings.

In your final paper, each summary paragraph of the qualitative data will go after each relevant quantitative data table (e.g., of frequencies or correlations), and your paragraph summary of what's in the table.

(13) Example of part of a results section from a paper I wrote:

RESULTS
Fourteen (n=14) Kean University students took the survey and participated in one of several focus groups. Demographically, 79 percent of participants were women, and 21 percent were men. Half the sample (50%) was Caucasian, 14 percent was Black, and 14 percent was Asian. In terms of ethnicity, 21 percent self-identified as Hispanic. Mean age of the sample was 28 years. Socio-economically, nearly 29 percent of the sample self-identified as working class. Thirty-five percent of survey/focus group participants indicated they were middle class, and 35 percent self-identified as upper middle class.

"The guys basically went to an ivy league school”: Class Difference between the Women Performers and the Lacrosse Team Members

Most participants – just over 71 percent – agreed or strongly agreed that the women performers and male lacrosse team members were in different social class groups, as seen in Table 1, below.

Table 1: Women Performers and Lacrosse Team Members were in Different Social Class Groups (n=14)
Agreed or strongly agreed: n=10, 71.4%
Neither agreed nor disagreed: n=2, 14.3%
Disagreed or strongly disagreed: n=2, 14.3%
TOTAL: n=14, 100.0%

[NOTE: THE ABOVE TABLE ISN'T COMING ACROSS WELL IN THE BLOG. IN THE ACTUAL PAPER, IT'S A TABLE IN WORD.]

[HERE’S WHERE MY SUMMARY OF THE FOCUS GROUP FINDINGS COMES IN, INCLUDING SOME SPECIFIC QUOTES]
Some participants felt the women and men were in different class groups because the men “could hire expensive lawyers” whereas one of the dancers, Crystal Mangum, came from a more modest background, being the “daughter of a retired auto-mechanic… that, like, pretty much put her in a family of the working class.” In addition, the fact of the men attending Duke University (“ivy of the South”; and “the reputation of the school… ranked the top ten institutions in the country as far as status, and it’s not cheap”) meant that “these kids have money obviously.”

By contrast, the women worked in a profession (exotic dancing) with low occupational prestige. “The fact that they had to dance. I don’t think that would be a woman’s first choice of occupation.” Another participant explained, “dancers…are automatically associated with being strippers or hookers, or being paid for…you are automatically labeled or looked down upon.” Another participant felt that the men “would have disrespected them [the women] anyway because of their job and double disrespected them because of their ethnicity.”

Another participant said that the women likely danced because they had to in order “to pay bills, as opposed to lacrosse team in Duke University. Most likely, they had their parents pay for their education so I would think they’re in different social classes.” Another commented on the fact that the lacrosse team was able to hire the women (“it’s like a lot of money, like $500 right off the bat”), suggesting that the employer-employee relationship indicated a class difference between the men (employer) and the women (employee).

THE IDEA IS TO USE THE DISCUSSION FROM THE FOCUS TO FURTHER EXPLAIN OR CONTRADICT WHAT THE SURVEY FINDINGS (PRESENTED IN TABLE FORMAT) TELL YOU. SURVEYS TELL YOU COUNTS AND PERCENTAGES OF RESPONSES, BUT THEY DON’T REALLY GET AT THE “WHY” AND “HOW” AND “WHY DO YOU THINK THAT.” THAT’S WHERE THE FOCUS GROUP RESULTS COME IN.