Skip to main content

Back to Next Generation Data Collection Index

ATOMS Project Technical Report:
Next Generation Data Collection

Bonnie L. Kennedy, Ph.D., OTR

Table of Contents

Introduction

Rationale

Technology provides a significant opportunity for AT outcomes data collection through its increased efficiency, natural environment assessment capability and potential for acquiring new types of outcome information (Smith, 2001). For instance, handheld palmtop computers allow branching and individualized question sets for people to use while they are performing their daily activities in the community.  This new data collection method permits an ecological and real-time data collection approach. This report will review the literature, the World Wide Web, and abstracts of funded technology outcomes projects to identify the current and emerging mobile technology-based data collection methods for comparative analysis and a projection to where these new systems might allow us to go.

Scope

A new generation of outcome measurement for rehabilitation and assistive technologies (AT), based on mobile information technology applications, has the potential to generate ecologically enriched data for decision-making regarding AT. In this report, any mobile technology that delivers information to individuals is viewed as having the potential to collect information from individuals in their native environment. The proliferation of mobile technologies in the personal, enterprise, and scientific arenas creates a broad scope for gathering information on this topic. Personal digital assistants, mobile phones, and pagers are the primary personal information technologies that this report focuses on. In addition, specialized commercial and industrial hardware and software applications are briefly explored.

Data Collection Procedures

This report focuses on personal mobile technologies capable of capturing and transmitting multidimensional data that can describe AT use in the everyday context of the individual user. Using diverse search strategies, researchers tapped into sources of information on mobile technology. Multiple search strategies were used for each topic (See Appendix A). Each search strategy was pursued until no new information on the topic emerged, indicating that the search strategy was exhausted. Topics were searched across multiple databases during which each hit was evaluated for both its apparent applications to the measurement of AT outcomes in daily activity and for redundancy with previously collected information. Hits containing redundant information were omitted. The purpose of the data collection procedure was to seek out novel examples of technology and to not collect exhaustive information on the manufacturer of each type of technology. Such a task is rarely feasible due to the rapid turnover in mobile technology. Monthly and quarterly periodicals on mobile computer technology generally report on newly released products (see Pen Computing v10, 49, June, 2003 and PDA Essentials, v14, May, 2003) with rare comprehensive comparative reviews of a specific line of mobile technology such as PDA phones. Within a few months of publication, due to the rapid change in mobile technology, these reviews are no longer comprehensive. Year-end reviews (e.g. “the best of the year”) (e.g. Ultimate Mobility from LAPTOP, 2003) are helpful in getting an overview of the features of mobile technology, however it is not considered unusual for a product launched one year to be out of production the same year or the next.

Sources

Steps and iterations

Sources of mobile technology information were identified based on a preference for primary sources, peer reviewed sources, and high circulation mainstream commercial (e.g. periodicals) and public sources (e.g., NPR, public radio).  Primary sources included examining the products themselves along with the material and software enclosed.  Other sources identified were first hand accounts of product development (Butter & Pogue, 2002) and original equipment manufacturer (OEM) websites (see Appendix C).

Initially, a literature matrix approach (Garrard, 1999) was attempted for this review.  In this approach, articles were entered into rows of a table whose columns contained variables relevant to the project topic, such as gender, age, and disability of the human subjects in the sample. This method produced a large, hard to use table with only a small fraction of the articles and abstracts obtained for this project entered into it. It was abandoned as a strategy to uncover the potential applications of mobile technology to data collection for assistive technology. The matrix method appeared to lend itself better to convergent applications than to divergent projects such as this one, where the goal is to identify a broad range of technology options for a diverse variable set.

The second approach to analyzing the peer reviewed literature consisted of an open coding strategy (Strauss & Corbin, 1990) which is used to identify the broadest possible array of technology options. Searches focused on identification of points of entry to information about mobile technology. Entry points included search words such as “ambulatory monitoring” and authors such as “Shiffman, S.” Open coding consisted of creating a category relevant to the application of mobile technology each time a new application was identified in the searched material, such as “clinician reference software”. Whenever possible, categories of applications were filled until the category was saturated (i.e., no new information was obtained). Some search strategies yielded only a few references and the categories were not considered saturated. The results of each search were open-coded to identify key categories relevant to this project.

Strategy three was environmental gathering, which is the hunting and gathering occupation of “homo sapien techie”, occupant of the information age. This process included surveying the world-wide web, consumer technology retail outlets, technology catalogues and periodicals, conference expositions (e.g., American Occupational Therapy Association, Rehabilitation Engineering Society of North America), bookstores, and Starbucks. This strategy was designed to cast a wide net into the sea of technology to capture one specimen of each new relevant data collection artifact. Using this strategy informed the author on the technology landscape and provided vocabulary for points of entry into focused search engines.

Findings

The Future

Many implementation configurations will be formed from the integration of handheld computer devices with multiple input options and data transport options. Mobile technology will capture outcome data from diverse populations via a broad array of hardware, software, peripherals, data transmission technologies, and data storage. See Figure C1 (Appendix C) for a model of the data capture process.

The Present

Mobile technology affords the opportunity for the user in their native environment to exchange information in real time between themselves and the service providers or researchers. It is helpful to identify the information needed from the native environment to select the best fitting hardware and software for the type of data needed. Some mobile technologies lend themselves to self-report data while other technologies are designed exclusively for third person observer reports. The report on hardware begins with technologies that lend themselves to self-report or observer reports then moves on to observer only reporting systems.

Over the past 10 years, personal digital assistants (PDA) and handheld personal computers (HPC) merged, morphed, and diverged. PDA initially referred to palm operating system devices that originated as personal information managers (PIM). Microsoft® Windows® based operating system devices were called HPC to contrast with PDA in the marketplace. They were noted for greater memory on board as well as software applications that required large amounts of memory. The HPC raised the bar for the industry by including digital audio, color graphics, and color still or video images in handheld devices in addition to the typical software suite associated with the PIM (see Figure 1).

Figure 1. The Hewlet Packard Jornada running Windows CE 2.2
Full text description

The picture shows a Hewlett Packard Jornada running Windows CE 2.2 HPC (Handheld Personal Computers). It is a mini computer on which software can be downloaded and used.

Retrieved from: http://h10025.www1.hp.com/ewfrf/wc/product?lc=en&cc=us&dest_page=product&dlc=en&product=61677

Handheld computers generally refer to larger than palm sized devices that often have a keyboard and the capacity to run condensed versions of Microsoft Word, Excel, PowerPoint and Access. Handheld computers have morphed into full capacity personal computers that are smaller than laptops for greater mobility (see a report on tablet pc models in “Pen Computing” December, 2002). All of these hardware products support questionnaire or survey software. Depending on the study goals and design responses can either be provided by self-report or an observer.

Mobile phones can be used to capture interview data via a live two-way interaction or through automated calling and digital survey systems. Closed-ended questions can be answered with either voice recognition or the keypad input. This type of solution lends itself to the needs of visually impaired consumers. Narrative data could be collected through a combination of open-ended questions, voice recognition, keypad entry, and speech to text transcription.

Smart phones integrate typical PDA capabilities with technologies like text messaging, web browsing, and email to provide multiple input and data transfer options in single device. (See Appendix C, Table C6).

Pagers are not only used for signaling individuals to respond to study protocol, they also have the capability to send and receive text messages.  Some are even equipped with small key pads that enable the user to send complex messages for data collection.

Noldus (http://www.noldus.com/human-behavior-research/products/the-observer-xt ) data collection system is an observer report system used to measure behavior. Many interdisciplinary applications can be found in human and animal behavior literature as well as on the Noldus website.

Measurement of actions, activities, and tasks has long been of interest to the field of industrial engineering. Numerous software and hardware products can be found for collecting self-report and observer based data. In general, the products focus on measuring frequency and duration of a few situated items in a work place. These systems appear to have limited application to data collection in the native environments of diverse AT users because of the improvisational nature of everyday life. Public transportation can run late, sidewalks can be closed for repair, weather provides different challenges within and between days (rain, ice, snow) – all of which contribute to infinite variation in how activities in the natural environment are carried out. This demand for adaptation, well known to AT users, is not part of the underlying assumption of most current “time in motion” technologies. Depending on the research question and the relative standardization of the environment in which the use of AT is being studied, time in motion technologies may have their place in AT outcome measurement.

Actigraph: Mini-Motionlogger Actigraph (Ambulatory Monitoring, Ardsley, NY) imputes activity through algorithms employing information from accelerometer sensors placed on the body. Research questions related to frequency, intensity and duration of effort in using AT could be addressed with this technology.

Input devices

Diverse input technology for handheld electronic devices provide an array of interfaces for reporting data to the PDA. Most PDAs come with touch screens that are calibrated to the user and software that affords scaleable targets. By scaling text and graphics large enough, people with low vision or poor coordination can be accommodated. Built-in hard and soft keypads are also common ways to enter data. Hard keypads tend to be smaller the more portable the handheld computer is. On the other hand, soft keypads reduce the space available on the screen for information and images. Furthermore, keyboards as peripherals are available for most PDAs. Additionally, barcode scanners were used in order to enter data into a PDA in a study of wellbeing of people with arthritis. See Appendix C, Table C3 for samples of barcode scanners and other similar input devices.

Audio files can be created for narrative data on most PDAs. Business quality PDAs generally have built in microphones and speakers. Both PalmOS and Pocket PC support digital audio files. These files, like other files, are stored in the memory with a time and date stamp. The Sony CLIE can store hundreds of minutes of audio files and has interchangeable memory cards. Essentially, unlimited audio data could be recorded. These files could provide first person narrative or caregiver narrative to supplement quantitative data regarding participation in a life situation.

Similar to audio files, digital images can provide qualitative information about the fit between AT and the native environment of the user.  Combining audio and digital information from keypad or stylus and touch screen input, a rich description of person, AT, and environment interface could be captured. Digital pictures are easy to take and of sufficient quality for data collection. Again, memory is not an issue due to the interchangeable memory cards.

While a broad array of sensor technology can provide relevant information on performance and use of AT, to be useful, sensor data must be integrated with other types of information to be meaningful in this measurement process. Sensors are relevant as input devices to a data capture system anchored by a personal mobile information technology device. Sensors are electronic devices that capture a change on a surface (e.g. pressure, light, heat, moisture, movement, skin conductance) in terms of change in the voltage that can be coded into binary data and then processed by a computer program. Familiar applications of these sensors are for heart rate, blood pressure, motion, muscle tension or activation, galvanic skin response, and skin temperature.

Global Information System (GIS) mobile products are integrated into PDA hardware or stand-alone hardware that is small enough to be combined with a PDA in everyday activities. Garmin iQue3600 (www.garmin.com/mobile ) has integrated GPS with PalmOS5 in an open face PDA format. The PDA has PIM software, mobile office applications, USB hotsync, and voice recorder. Timex has a sport watch with a GPS system that logs distance and speed. Such a device might be used in combination with another device to gather appropriate data for the study of selected AT and users. Numerous GPS equipment and software developers have created products to run within the Microsoft Pocket PC environment (see Appendix C, Table C4).

An integrated mobile data collection system will require custom content to be sculpted in order to capture information relevant to specific types of AT users, for example:

In addition, protocol design must be structured to capture the target uses of AT in the environment while minimizing participant burden. This may be done with strategic sampling of participation for outcome measurement:

Needless to say, user interface with the data collection instrumentation is essential to reliable and valid data acquisition. The variety of input strategies for lightweight mobile computing and communication supports a broad array of AT users.  In addition to “off the shelf” consumer and enterprise technology, mobile computing and communication itself have become assistive technology that incorporates specialized products for the visually and hearing impaired (see Appendix C, Table C8)

Discussion or Interpretation

By sampling activities in the natural environment in real time, outcome data consumers will capture the experience and performance of life participation in a manner that allows the ability to discriminate effects of assistive technology (AT).  This paradigm of outcome measurement will be accomplished through a suite of mobile technology products.  The suite will be configured around a universally designed core element that taps participation in society through engagement in daily activities

Solution Design

A handheld PC / PDA that synchronizes multidimensional data from multiple simultaneous events and asynchronous sources will anchor the next generation of mobile technology data collection solutions. Custom software for the PalmOS or Pocket PC environment will integrate the necessary software applications. Input devices will be customized for the research question and the sample of AT users being studied.  For example, motion monitors that provide input to the PDA may be appropriate for studies of assistive technology that facilitate mobility, while being unnecessary for assistive communication devices.

This section of the report focuses on the content of a solution design for gathering data on naturally occurring everyday activity in a personal life space or native environment. The solution calls for a universally designed core that is applicable to the study of daily activity with assistive technology along with specialized modules that are applicable to particular circumstances.

Universal Core

The universal core will sample context, activity, subjective experience, performance appraisal, and usage of AT. Diverse proximal contexts of participation in life situations will be captured in the next generation mobile technology AT outcome solution. These contexts shape and locate the person-environment interaction that is participation in the sense communicated through the International Classification of Function (ICF) (World Health Organization, 2002). Important contexts to consider are as follows:

The core outcome tool must include self-reported subjective experience in order to measure authentic outcomes relevant to the participation of real people in real life situations. Some of the outcomes of interest such as quality of life, pain, and fatigue are only available as self-reported data. Variables likely to be important to a universal core are:

Finally, the universal core will depend upon multidimensional data because no one variable can capture the complex nature of participation in life situations. Multidimensional data may include one or more of the following in addition to the essential elements described above:

Implications for the Next Generation Outcome Measurement System

Outcome of assistive technology (AT) intervention is multifarious and no summative measure exists to capture the complexity of life with and without AT.  Ecological assessment of activity, context, and subjective appraisals can provide multidimensional real-time, natural environment outcome data. In combination with single use trait and cost measures this complexity can be approached.

Researchers will be challenged to assemble a data collection system through an iterative process that takes into account the integrated nature of software, hardware, human interface and environmental constraints (see Figure 2). The moving target that is current hardware and software will also challenge them. New versions and new models are nearly a weekly occurrence.  A research tool must be based on hardware and software that will remain stable long enough to repeat studies, as replication is an essential component of science.

Figure 2: Essential iterative process to assemble a data collection system
Full text description

This figure describes an iterative process needed to assemble a data collection system which is based on the integrated nature of hardware, software, human interface and environmental constraints. Software and hardware are constantly changing and a data collection research tool should be based upon software and hardware which remain stable long enough so that studies can be replicated.

Appendix B summarizes the elements shaping the collection of outcome data. It is a list of parameters that specifically influence each stage of data handling. This matrix illustrates the complex, intersecting nature of the parameters.

In order to analyze outcome data to satisfy the need for model development data must be in electronic format. Collecting data in electronic format initially reduces time delay for data entry, labor costs for data entry, and data entry errors associated with pencil and paper data collection.

Numerous options for electronic data collection of AT outcomes increase exponentially with the inclusion of parameter assumptions at each stage of the data handling. To determine the best combination of data collection procedures for a given person-environment-outcome, the outcome researcher may need a data collection selection decision tree (see Appendix D).  This could be done through an electronic branching questionnaire with contingent combinations embedded in the questionnaire. For example, if a visually impaired data collector in a remote rural setting is selected, the program would offer only options accessible to visually impaired persons linked by phone line or wireless connections.

Parameter assumptions for each stage of data handling:


References

Butter, A., & Pogue, D. (2002). Piloting Palm: The inside story of Palm, Handspring, and the birth of the billion dollar handheld industry. New York: John Wiley & Sons, Inc.
Garrard, J. (1999). Health sciences literature review made easy: The matrix method. Boston: Jones & Bartlett Pub.
Hasselkus, B. (2002). The meaning of everyday occupation. Thorofare, NJ: Slack, Inc.
Strauss, A., & Corbin, J. (1990). Basics of Qualitative Research: Groundd theory procedures and techniques. Newbury Park, CA: Sage.
World Health Organization. (2002). International Classification of Functioning, Disability and Health. Geneva: World Health Organization.
Zerubavel, E. (1981). Hidden Rhythms. Los Angeles: University of California Press.

Appendices

Appendix A: Online Literature Search Results

Appendix B: Intersecting Elements Shaping Data Collection

Appendix C: Devices and the Data Capture Process

Appendix D: Data Collection System Decision Tree

Back to Next Generation Data Collection Index