Conference Participants:

Frances Harris, PhD
Bonnie Kennedy, PhD, OTR
Noralyn Jacques, PhD, OTR
Kathy Rust, MS, OTR
Marcia Scherer, PhD
Pimjai Sudsawad, ScD, OTR
Mary Taugher, PhD, OTR

Introduction

The Qualitative Research Conference (QRC) was organized with the goal of exploring ways in which qualitative research, as it has been developed in the general field of disability and, specifically, assistive technology (AT), could be made relevant to outcomes researchers. It also was an opportunity to introduce other less well-known qualitative and social science constructs to the disabilities research arena. We were particularly interested in examining and developing a qualitative research model(s) that could be incorporated within an AT outcomes measurement system.

Needs Assessment

Outcomes research in Assistive Technology is presently in its most formative period. Efforts to develop classificatory schemes, concepts, methods, and outcome instruments currently dominate AT outcomes research and practice. At present, ATOMS, one of two national consortium projects devoted to developing these objectives, is working to create an outcomes measurement system. The aim of this system, among other things, is to provide an interactive, critically structured database. It would be accessible to consumers, clinicians, policy makers and all those who have a vested interest in both directing and enhancing AT use.

As regard its general methodology, outcomes research is quantitative in nature. That is, research questions and results are tabulated and/or manipulated so that they may be expressed in a numerical format. However, classificatory schemes, which form the basis of those concepts underlying outcomes methodologies and instruments, contain numerous assumptions, attitudes, values, goals, etc. about disability and technology. The effect is that seemingly objective numerical data are quite subjective in reality and practice. Indeed, the results of outcomes research rest on the assumed integrity and utility of its classifications and categories. Outcomes results necessarily presuppose that the aggregate data produced correctly represent the individuals' experiences on which they were originally based.

An example from the anthropological literature: Schiller (1992) examined the seemingly neutral epidemiological categories used by NIH and the CDC to identify and communicate with AIDS risk groups, (i.e., Hispanic or Latino communities, IV drug abusers, the gay community). Through qualitative methodology, it was demonstrated that these categories did not resonate with target populations. For example: just because a man has sex with another man does not mean that he will automatically identify himself as "gay." IV drug users may not necessarily see themselves as having a drug problem. And Hispanic and Latino populations are not a single culture; they encompass diverse nationalities and cultures. One result was that public health messages not only were less effective with the target populations, they actually overlooked the fastest growing segment of the population to become HIV positive - heterosexual women.

In this instance, the crucial issue is that categories that appear commonsensical and authoritative must themselves be investigated as ways of perceiving and organizing the world. Diagnostic procedures and questions may reflect more about how medicine, a branch of scientific research, organizes its own knowledge practices and less about how the phenomena they are interested in understanding actually occur. A good outcomes measurement system necessarily depends on its capacity to gather information in order to accurately reflect the complexities of the behaviors and activities under research.

These issues profoundly affect outcomes researchers interested in understanding the impact of AT use on people with disabilities in ways that society as a whole considers valuable. Consider that the recently modified WHO ICF nomenclature related to disability serves as a guide for outcomes researchers who wish to calculate the effects of AT on its users. The ICF construct of 'participation,' an increase of which is generally regarded as an obvious outcome goal, contains many assumptions about the nature and desirability of societal, work, and familial relationships that may or may not be realistic, desirable, or applicable across all disability types and contexts. These categories and classifications need to be investigated to discover in what contexts "participation" reflects a meaningful goal. The various meanings and connotations of participation need to be uncovered so that researchers can develop instruments that accurately assess its multiple complexities and uses.

In short, qualitative methodologies may prove a powerful strategy for exploring this, as yet, minimally charted terrain. Qualitative research can deepen our understanding and awareness of how outcomes categories are constructed and the multiple, often confounding, meanings they may carry. Qualitative research also provides additional information about how categories are actually "enacted." That is, the complex ways in which the subjects of research (for example, an AT user's participation or wheelchair) may act out/create multiple identities (i.e., AT user as patient, client, consumer, parent, employee) or uses (wheelchair as mobility, as an extension of one's body, the different spaces they inhabit or activities they facilitate). People may use their various assistive devices in ways unanticipated by the clinician, vendor, or engineer who designed them. Without well-integrated qualitative research methodologies, the complementary quantitative research data may be inaccurate or, worse, undermine the research question and mislead those depending on the data.

The Conference

Structure

Frances Harris set the agenda for and chaired the conference. The goal for the first day was to engage people in informal discussions and responses to general themes around outcomes research and qualitative methodologies, essentially a survey of concepts, terms, and assumptions. During the second day we were to focus on formalizing ideas for future projects involving qualitative research.

Day 1: April 29, 2003

Initial discussions addressed the differences between qualitative and quantitative research and how to meaningfully integrate these differences within a particular research project. Since outcomes research is traditionally understood to be a quantitative enterprise, it was essential to gather views on how different qualitative methodologies and theoretical orientations could be meshed with outcomes research goals. All conference attendees, while acknowledging differences in the types of data gathered and methodologies, agreed on the essential compatibility and potential usefulness of qualitative systems to AT outcomes research.

Outcomes variables identified by the ATOMS team such as cost, satisfaction, function, use/non-use, quality of life, and participation were discussed. A qualitative study that examined the use of canes by the elderly and which demonstrated the range of meanings these assistive devices held for their users was cited as an important example of the kind of contribution qualitative research can make. Meaning, defined here as the range of significance a variable may hold, was noted to be an important contribution of qualitative methodology to AT outcomes research. Because assistive technologies may hold multiple, even contradictory, meanings for their users, these meanings may directly impact researchers' understandings and attempts to measure them.

Also, it was noted that outcome variables are not discrete, but "bleed into one another" and are interdependent. That is, 'use' is directly related to how well a particular assistive device 'functions'. We see this often in the abandonment process. 'Function' and 'use,' in turn, impinge on how we understand satisfaction and participation. How we look at variables, how we carve them up and connect them to one another was identified as a key area in which qualitative research can make a contribution.

Three qualitative methodologies used within the general field of disability and assistive technology in particular were discussed: focus groups, interviews, and participant observation. Two essential points were noted in this discussion. First, focus groups and interviews report information from the subject's point of view. In contrast, participant-observation, as a method, is concerned with both observing behaviors, as well as performing unstructured interviews. That is, what people say they do or feel, how they perceive their actions are better understood and interpreted in conjunction with an observer's perception of actions. Observation provides both a context and a different point of view for people's perceptions of their actions. It was concluded that a qualitative strategy such as participant/observation coupled with self-report measures, including interviews and focus groups, provides a strong methodology for qualitative research especially as it pertains to outcomes research.

Various strengths of different qualitative strategies were also noted. For example, focus groups may produce unexpected data. Such data is often useful for creating first generation ideas. One may extract rich information from these data and use it to form initial questions and more sophisticated quantitative queries. From that stage one can develop questions that can be graded along, for example, a Likert scale in order to allow verification of qualitative data.

Another qualitative methodology that was discussed as being potentially useful to outcomes researchers is derived from "actant network theory" (ANT), a product of science and technology studies, e.g., Callon (1997), Callon & Latour (1981), Latour (1987, 1996). ANT assists us in considering a set of entities (people, machines, ideas, relationships, etc) as they connect more or less directly to the focus of our investigation - in this case assistive technology. It treats all these entities as "actants" as they act and are acted upon in a network of practices. They define each other by their relationship to each other. This actant network represents all the things that construct our subject of investigations. It alerts us to the pre-programmed or prejudged responses and similarly loaded questions we are asking. The model begins to show us not only those actants influencing the subject but the dynamic relationship among these actants. Feedback from one set of actions potentially reconfigures the relationships. Classically, we see this at a micro level when students provide the answer they believe the teacher wants. The teacher, in turn, becomes more responsive to those students, altering their predisposition to answer in specifically defined ways. This, in turn, alters the approaches to learning of other classmates.

ANT is a powerful qualitative methodology to emerge from a multidisciplinary social science approach to technology and health issues. It has a pragmatic applicability to outcomes researchers' needs in that it considers how previously homogeneous categories may contain many voices, interrelated variables, and hidden discourses with far-ranging effects. It lays a transparency over quantitative data exposing the historical depth and complexities that inform its categories.

An important point to note is the quantity of data that qualitative methods can generate. Depending on the specific strategies selected and applied, qualitative research typically generates large amounts of data in the form of verbatim notes, transcribed recordings of interviews, focus groups, videotapes, jotted notes, detailed "field notes" of observational research, and/or a researcher's reflections in the form of a diary made during the research process. A single interview, for example, can produce 30 to 40 single-spaced pages of raw data. It is not uncommon for a single project to generate thousands of pages of data.

Mentioned earlier, a strength and challenge of qualitative research is the rich and extensive data that is generated through it. A realistic tactical and resource analysis of qualitative research data typically begins during the data collection process itself. Continuous review of data allows questions to be refined iteratively, and may suggest new lines of inquiry. This is appropriate to qualitative research goals, which are typically interpretive or exploratory in scope (rather than statistically predictive or descriptive), and, therefore, are open to modification and development as data is collected in both the structured and organic sense.

Once data collection is complete, the researcher begins to identify patterns, themes, and concepts from the variety of accumulated texts. This is done by examining research questions, both in terms of the aims and objectives of the project, as well as the issues raised by subjects themselves. There are a variety of tools (methodological and software) from which the researcher may choose. Depending on the goals of the project, the researcher will categorize/map/index and/or chart the data. These processes lead the researcher who will begin to interpret data and provide explanations by finding associations between themes and creating typologies.

Thursday, May 1, 2003

Discussions during the second day of the conference concentrated on developing a research agenda that would augment ATOMS research goals. Specifically, three different kinds of projects were suggested.

1. Phenomenological interviews using the three-interview method (cf. Seideman, 1998). It was suggested that a number of AT users be interviewed using this particular methodology in an effort to uncover the range of meanings and uses assistive devices hold in the context of their disability.

Phenomenology is a 20th-century philosophical movement that has had wide theoretical applications within the social sciences (e.g., Heidegger, 1962). In short, a phenomenological research strategy is concerned with uncovering an individual's subjective experience of their world. Through multiple and lengthy interviews, an informant's experience of their "world" (in this instance the "world" as a disabled person) is explored.

The purpose of using the three interview method (Seideman, 1998) is to: first, generate data regarding a subject's life history; second, to obtain life histories in detail with respect to their experiences as a person living with a disability, and third, to allow subjects to reflect upon the meanings of their experiences. In this case, the meanings uncovered by the users of assistive technology are rendered in terms that are extremely individual and personal. The contribution of such interviews in terms of outcomes research is twofold: to uncover heretofore undiscovered themes related to the use/non-use of AT, and to broaden/deepen our understanding of already existing categories.

2. A participant/observation research project based on actant network theory (ANT) was proposed. Such a project in assistive technology would choose several locations in which assistive technology is prescribed (such as a clinic, hospital, and school). Through participant observation and interviews the researcher would (for a minimum of 2-3 months) observe client-clinician interventions, interview clients and their families and/or caretakers, clinicians and administrators, vendors, insurance providers. Goals would include examining assistive devices historically, observing and comparing clinician goals and activities with those of vendors, designers, engineers, insurance providers, and social policy makers.

3. A third proposal was to run a series of focus groups for multiple stakeholders in assistive technology, including patients, clinicians, vendors and insurance agents. Goals would include bringing to the foreground differences in goals and understanding (for example, of responsibility for AT provision and use). The resulting data could be used in one of two ways: first, as raw data from which to distill key constructs and meanings which, in turn, could be used to build research instruments. Second, data could augment other qualitative research data to provide both depth and critical feedback and structure to other qualitative strategies such as participant observation.

Next Steps

This conference considered the general contributions of qualitative research methodologies and their overall applicability to AT outcome system goals. The next activity is to formulate specifically how qualitative data may be analyzed in concert with outcomes research goals. This involves a multi-part process. In order to make this realistically manageable, the first four parts of the first phase are identified, below.

1. Define a concretized and formalized understanding of how qualitative research in general is analyzed.
2. Integrate the formalized understanding of analytical process and program with quantitative methodologies/findings in service to AT outcomes research.
3. Provide a specific context in which qualitative research could be applied to a current AT outcomes activity.
4. Determine and apply appropriate resources to data gathering and analytic activities to allow the quantitative and qualitative programs to be complementary and fully productive as a partnership.

A small team, possibly two or three individuals could rapidly draft responses to the definition of an analytical program. This would satisfy points one and two, above. QRC participants could then review the draft for comment. The intent of the review would be to correct or amplify any issues left unclear or absent from the draft statement and to construct a response to point three. The draft would then act as a framework to develop a general structure to respond to point four.

Summary

Qualitative research methodologies can provide several levels of novel and complementary utility for AT outcomes researchers. The iterative, organic, but formalized program of data collection and analysis can clarify, create, and refine, existing categories and concepts and their relationships among each other for organizations, individuals, technologies, economies, and connections. They also can provide information that illuminates assumptions in existing quantitative instruments constructs. The innovative outcome of this complementary approach can produce better quantitative information and a context through which a broad audience of researchers, end-users, and policy analysts and decision-makers can access and comprehend the power and expertise of outcomes work.

References

Callon, M. (1997). Representing Nature. Representing Culture, Paris: CSI, Ecole Nationale Superieures des Mines.

Callon, M. and Bruno L. (1981). Unscrewing the big Leviathan: how actors macrostructure reality and how sociologists help them to do so. In K.D. Knorr-Cetina and A.V. Cicourel (eds), Advances in social theory and methodology: Toward an integration of micro- and macro-sociologies (pp. 277-303). Boston, Mass.:Routledge and Kegan Paul.

Heidegger, M. (1962). Being and Time. (J. Macquarrie & E. Robinson, Trans.). New York: Harper.

Latour, B. (1987) Science in Action: How to Follow Scientists and Engineers Through Society. Milton Keynes: Open University Press.

Latour, B. (1996), Aramis, or the Love of Technology. Cambridge, Mass: MIT Press.

Schiller, N.G. (1992). What's wrong with this picture? The hegemonic construction of culture in AIDS research in the United States. Medical Anthropology Quartery, 6(3), 237-254.

Seidman, I. (1998). Interviewing as qualitative research. New York: Teacher's College Press.

---

 

Copyright © 2001-2015 ATOMS Project at the University of Wisconsin-Milwaukee