Podcast, Research Methods

Research Interviews and Asking Good Questions

I have been thinking about interviews and how to ask better questions/interview for a while. Research questions unpack what is going on with the world around us. As an early career scholar, I want to unpack experiences, thoughts, and situations people are dealing with in the workplace (e.g.  networked professional lives, open online learning, mentoring relationships) to learn more about a particular phenomenon. I know good research comes from solid research preparation.

Last summer  I spent a couple of months, with my co-investigator Paul, digging into the empirical literature, academic findings, theoretical frameworks and debates around concepts and issues we want to unpack in our study. I appreciate his willingness to work and put the time up-front to prepare for our research interviews.

“Research designs begins with questions researchers and their partners want to answer about a particular problem, population, process, project, or topic they want to explore” (LeCompte & Schensul, 2010, p. 130).

We framed our research questions around issues addressed in other academic papers — you know, building on the shoulders of giants — and to unpack what is happening in the online and offline realm for higher education professionals. For our semi-structured interviews, we have a set of structured questions to guide open-ended discussions on relevant topics related to the themes, issues, and concepts we want to discuss (Kvale, 2007). By using the intensive interview techniques shared in Charmaz’s (2014) constructing grounded theory text, most of our questions are open-ended. This method was designed to encourage participants to reflect and share experiences, by starting questions with: “Tell me about…”, “Could you describe… or “Can you walk me through…”  Asking research questions to solicit for a comprehensive and an open response is everything.

This research design thinking not only developed our interview protocols, research questions, and data management plan, it also allows us to be fully immersed in our conversations while we conduct the interview now.  I think conducting a quality research interview is a skill. A skill that gets developed, honed and enhanced as you go. I always learn how to improve upon this each time I talk with a research participant. While being immersed in the interviews, I have kept this sage advice George (thanks!) offered when we were conducting interviews with a large number of open, online learners:

  • Give wait time to think before answering and tell them that you are doing that.

  • Listen to their replies and ask probing questions that aren’t listed below but go toward the issues we are trying to explore.

Now that we’re 60+ interviews deep with our project, I continue to think about this advice and understand what we are learning so far. I am also thinking about what we are asking, how we are approaching topics, and identifying where we might need to go as our questions reach a certain saturation point. If you have already graciously volunteered your time and shared for our study: THANK YOU SO MUCH!  If you are a higher education professional who would like to contribute and be interviewed for our research, we are still accepting participants for our study here: http://bit.ly/networkedself

UPDATED: Friday, August 11, 2017

R.I.P. #Turnaroundpod — it’s sad to hear that your podcast series is coming to a close. THANKS SO MUCH for producing The Turnaround Podcast, Jesse. It will be sad to see you go! Want to read more about this? Check out the Ask Me Anything (AMA) of Jesse Thorn on Reddit.

Recently, I started listening to Jesse Thorn’s  The Turnaround podcast (that partners with the Columbia Journalism Review -thanks for the transcripts!) This podcast flips the script and interviews people who typically interview others.

the-turnaround-cover_6

Image c/o The Turnaround! a Maximum Fun Production

These interviews unpack the art form of an interview and how to best investigate a story. Thorn asks how to best interview and also demonstrates how to summarize ideas and follow with an open-ended question for a response. Although most of these interviewers are producing interviews for public consumption and listening, there are some great takeaways from this 1:1 series about interviewing:

In addition to listening to podcasts or reading scholarly books about interviews, I thank and credit the @BreakDrink podcast production for providing me with the skills to conduct effective research. My “study” in podcasting (and research interviews) began just over 7 years when I received a DM from Jeff Jackson to see if I’d like to co-host a podcast. Although I was just starting my Ph.D. program, I think some of my early lessons for qualitative research actually came from the episodes where we invited brilliant people onto the Campus Tech Connection (#CTCX) podcast for an interview. Both my experience with podcast production and research interviews, have offered me a few insights for being a more effective interviewer:

  • Pre-Interview survey: Ask your podcast guest or interview participant a few questions about the topic in advance. For podcasts, we would have them complete a brief bio and see a few of the questions we might ask ahead of time. For interviews, we might have a pre-questionnaire or interview sign-up with requests for demographic information, topics about the research, or their role for the study research.
  • Organize and prepare: Do you work in advance! Create a shared doc (if on a collaborative team) or prep notes for each show production or segment of your research interviews. This would include the potential protocols, research questions, interview topics/issues, and information you would need for each recording. Review the pre-interview survey data and see how they might relate to your research questions.
  • Play with the technology to figure out what works for you: Technical tools have changed over the past 7 years of my podcasting/researching. I continue to learn as I go and as I collaborate with others. I now record with Audio Hijack+Skype/web conference/phone, edit in GarageBand/Audacity by splicing clips either for public consumption or to minimize for transcription costs, and find a secure cloud storage space for your audio files and notes.
  • Speaking of notes… ALWAYS TAKE NOTES: Besides recording the audio, I often scribe notes during a conversation or interview. These notes could include a quote, key point, idea, or issue. For the podcast, this might include a URLs and resources we would share with the show notes with the episode. For research, this ensured I was listening and noting what participants were saying and often it would spark a follow-up question or explore another aspect of our study I wanted to know about.  Pro-Tip: I use “analog” journals to write my research notes with pen and paper. I often return to my notes to make an annotation, highlight a concept, find another research question, and to review how the series of interviews are progressing.
  • Make time for reflection: After each episode of the podcast, I often would have a follow-up blog post with information and ideas shared. This practice I still do when I conduct a research interview, but often it’s a private act scribed in my journal or shared with my co-collaborators on a project.  This habit has me process what I am exploring, learning, and sorting out in my head.
  • Manage and archive your files: Be sure you create a system to label and itemize your digital files and notes. I am meticulous for organizing my life and projects (as I live in the digital) in particular ways. Set your own system so you can track where items are and code how these files/interviews are relevant to your project (or podcast). This will help you later when you go to code transcripts or you are interested in a particular issue/trend in your study.

References:

Charmaz, K. (2014). Constructing grounded theory.Thousand Oaks, CA: Sage.

Kvale, S. (2007). Doing interviews. Thousand Oaks, CA: Sage.

LeCompte, M. D., & Schensul, J. J. (1999). Designing and conducting ethnographic research (Vol. 1), 2nd Edition. Plymouth, UK: Altamira Press.

Advertisements
Fashioning Circuits, Research Methods

Participation Observation Method

In constructing the curriculum chapter for the Fashioning Circuit book* being developed by Dr. Kim Knight (a.k.a. @purplekimchi), I utilizing a few exploratory research methods to review the current workshop materials, lessons, and learning on the subject matter. The first method: Participant Observation. As I work through evaluation and assessment of the curriculum, I might as well share and get feedback on the process.

EFC Camp

Participation observation allows for the collection of information and qualitative data, rooted in the ethnographic research tradition. For this method, participation observers report on the physical, social, and cultural context to reveal relationships, activities, and behaviors of subjects. This is an effective method to gather information to support project design, data collection development, and to interpret other research. Data collection for this method includes note-taking, mapping-relationships, and media (video, audio or images) that might be translated into textual artifacts. Challenges to this method include diligent documentation and objective account from observers in the field, and this process can be time-consuming.

Specific responsibilities for Participant Observers include:

  • observing individuals as they engage in activities (as if you were not present and watching)
  • engaging in the activities to gain a better understanding
  • interacting in a controlled research environment
  • identifying and developing relationships with key informants and stakeholders

For the purpose of this research, I developed a field guide for our research team of three. Basics for the observation guide include listing the observer name/background, research setting, materials used, and concentration areas to focus on for the workshop observation. Other tips and general guidelines were provided to outline expectations for observing.

The research team divided and conquered today by taking notes related to the following categories:

  1. Lesson/Curriculum (Electronic Fashioning Circuits Camp)
  2. Lead Instructor/Facilitator (a.k.a. Dr. Knight)
  3. Learners/Students (participants in the workshop)
  4. Facilitators/Helpers (those supporting the workshop)

The observation guides were segmented by the 4 categories and included questions to prompt observers and focus their field notes.  The observation goal was to focus on the physical space and set up, participant attributes and involvement, verbal behavior and interactions, physical gestures, personal space, lesson understanding, instructional support, and individuals or examples that stood out from the workshop.

At the beginning of the day our group met to review the research context, expectations, behavior as an observer, and potential problems that might occur during the workshop. Another item we discussed was distinguishing interpretation (I) from observation (O), and labeling our notes accordingly (Kawulich, 2005). To help with strategic note-taking, I encouraged leaving space to expand on notes, using shorthand to follow up with later, writing observations in  sections, and encouraged our team of researchers to consider body language, attitudes, conversations, ambiance, and general interactions that might be relevant for the curriculum.

Participant Observatin Continuums

Image c/o Chapter 3: Participation Observation (Guest, Namey, & Mitchell, 2012)

During the day the three of us took notes on tablets, laptops, mobile phones, and pads of paper with the following platforms: Google docs, Word, Evernote (audio & images), etc. We reconvened the end of the workshop to process and discuss what we observed. This debriefing provided ideas for supporting a research team, specifically with regards to:

  • general observations, ideas, and questions about the workshop
  • how to create anonymous identifiers for research subjects in notes
  • expectations for field note-taking and organization submission for the lead researcher
  • roles and responsibility for how to effectively observe a single group within in a workshop, i.e. instructor, learners, and helpers
  • future planning needs and ideas for upcoming participation observation

I am truly grateful for the UT Dallas EMAC students, Jodi & Lari, who volunteered their time to observe and be a part this exploratory study. Their insights and ideas are very helpful for future field observations and research method development. Once everyone’s participation observation notes and artifacts are collected, I will share how to analyze this data.

Lily pad

*Interested in learning more about Fashioning Circuits? There’s a few social spaces for that! Check out the Fashioning Circuit’s website, Facebook page, Twitter handle or hashtag #FashioningCircuits. Feel free to follow along, and join the conversation.

 

References:

Guest, G., Namey, E. E., & Mitchell, M. L. (2012). Collecting qualitative data: A field manual for applied research. Sage.

Kawulich, B. B. (2005). Participant observation as a data collection methodForum Qualitative Sozialforschung / Forum: Qualitative Social Research6(2), Art. 43, http://nbn-resolving.de/urn:nbn:de:0114-fqs0502430.

Research Methods

Action Research Methods

Action research (Koshy, 2005), also known as participatory action research, is a method of research that can combine a framework for public, reflective inquiry.  It offers a way for academics and scholars to work with individuals or communities, such as learners or communities of practice, to investigate issues together to find a solution-oriented approach. For educational settings, this might often be the “teacher-as-researcher” research approach (Elliot, 1991; McNiff, 2013).

action_research

Image c/o New Mexico State University, College of Education Research and Budgeting

The premise for the action research method is that  researchers study with the subject(s), rather than research on or outside the subject(s) as a hierarchy or “expert” (Cousin, 2008). Action research is a journey that experiences transformative findings with linear, sequential relationship between the hypothesis, research activity and the research findings, i.e. a proposed solution.

Another branch of this research includes, community-based participatory action research (CBPR), involves researching a group or community where research is conducted:

by community members, so that research results both come from and go directly back to the people who need them most and can make the best use of them.

CBPR research will grow in education as it is impacted with emerging technologies and learning pedagogy design, while needing to meet standards and managing fiscal challenges. This research method allows for project ideas and measurement outcomes to be designed by the community population to shape the research agenda. Participation in research by the community often results in generating greater sociopolitical awareness and effecting systemic change in the community (Jason et al., 2004). From  other case studies described by Jason et al. (2004), CPBR has improved upon the quality of scholarly work, developed research outcomes, created intervention tools, and increased quality collaborations on projects.

References

Cousin, G. (2008). Researching learning in higher education: An introduction to contemporary methods and approaches. Routledge.

Elliott, J. (1991). Action research for educational change (Vol. 49). Buckingham: Open University Press.

Jason, L. A., Keys, C. B., Suarez-Balcazar, Y. E., Taylor, R. R., & Davis, M. I. (2004). Participatory community research: Theories and methods in action. American Psychological Association.

Koshy, V. (2005). Action research for improving practice: A practical guide. Sage.

McNiff, J. (2013). Action research: Principles and practice. Routledge.

 

Fashioning Circuits, Research Methods

Ethnographic Methods

In thinking about how to evaluate the curriculum and pedagogy for the #FashioningCircuits book with Dr. Kim Knight (@purplekimchi), I spent some time over the weekend reviewing  potential research methods for the project. Since I have used  ethnographic methods for a grant project Fall 2012 (studying culture and healthcare perceptions), I thought I’d revisit these as potential research methods and blog about it here.

Ethnographic Methods

Ethnographic is the description of groups, specifically these approaches appeal to those who are confident they have time to stay in a research setting (the field) for a sustained observation and informal interviewing (Cousin, 2008).

ethno

Typically ethnographic research derives from anthropology where communities or groups are studying for long periods of time (a year or so) for the “purpose of learning from their ways of doing things and viewing reality” (Agar, 1980, p. 6) where the researcher not only observes, but also interacts and talks with participants (Delamont, 2002, p.g. 8). Other data collection can be collected with communities also interact in online spaces and contexts. Ethnographers can study forums, wikis, blogs, or Twitter in the context of participant observation research to review the community and activity or historical research archive (Boellstorff et al., 2012).

A few selected strategies for research inquiry:

  1. Observation: Researchers will complete overt participant and observer review strategies which the ethnographer has negotiated access to the group to observe and participate in its activities (Cousin, 2008). Observations of this group will include notations about the environment, population, interactive order, and characteristics of the membership. In thinking about digital footprints, historical archives and other means of data collection for “observation” of this group is inclusive of blog posts, videos, tweets, and other online contexts (Boellstorff et al., 2012).

  2. Semi-Structured Interviews using the Ethnographic method (Schensul et al., 1999) provides researchers with a structured set of themes to organize the interview discussion. Unlike the structured interview, interviewers will be required to adapt, modify, and augment the prepared questions if they flow of the interview discussion suggests it (Cousin, 2008). The interviews discussions are “meaning making events” where the ethnographer utilizes “active interviewing” skills (Holstein & Gubrium, 1997). From a common list of questions, ethnographers will interview various members of the group. These interviews will be transcribed and evaluated for thematic information and content using a grounded approach.

References

Boellstorff, T., & Marcus, G. E. (2012). Ethnography and Virtual Worlds: A Handbook of Method. Princeton University Press.

Cousin, G., Cousin, J., & Deepwell, F. (2005). Discovery through dialogue and appreciative inquiry: A participative evaluation framework for project development. D. Taylor and S. Balloch (Eds.), The politics of evaluation, 109-118.

Cousin, G. (2008). Researching learning in higher education: An introduction to contemporary methods and approaches. Routledge.

Delamont, S. (2002). Fieldwork in educational settings: Methods, pitfalls and perspectives. Routledge.

Holstein, J. A., & Gubrium, J. F. (1997). Active interviewing (pp. pp-113). Sage Publications.

Schensul, S. L., Schensul, J. J., & LeCompte, M. (1999). Essential ethnographic methods. Walnut Creek.