Presented at the meeting of the Association for Educational Communications and Technology (AECT), St. Louis MO, 18-21 February 1998, as part of a panel chaired by Dan Surry. Web location of the paper: www.cudenver.edu/~bwilson/howtouse.html.

For the past couple of years, our research team has experienced a growing sense of discomfort with traditional theories of technology adoption and diffusion, especially as they relate to the human endeavors of education and training. Adoption models of Rogers and Hall, for example, are marked by their descriptive (as opposed to explanatory) emphasis, and their frequent use of labeling, typing, and categorizing of various elements and participants of the change process. Constituents fall into X number of categories; innovations may be of X varieties; and so on. In spite of our persistent concerns, however, our group has struggled in efforts to develop a comprehensive alternative to existing models.

One possibility, however, is this: It's not so much the model that's of concern; it's the way the model gets used. This paper constitutes an exploratory exercise in addressing this possibility. In conducting this exercise, we do not critique a particular theoretical model of adoption or change. Rather, we critique a generic approach to using such models in practice. Our intention is to highlight how the use of models tends to channel and constrain our thinking, leading to occasional mis-reading of situations and local needs.

Consider how a new technology or method typically gets implemented in an organization. Table 1 below presents several tasks that an organization might complete as part of a decision-making process concerning new technologies.

Determine a need to assess the need.

Create a committee or hire a consultant to conduct an adoption analysis.

Develop a conceptual framework for conducting the adoption analysis.

Collect, analyze, and interpret data bearing on the anticipated costs and benefits of the contemplated change.

Evaluate the pros and cons and decide on a course of action.

If the proposed innovation is to be adopted, then develop and carry out an implementation plan.

Collect information about the program's success, and modify as needed.

Table 1. Tasks an organization might complete to determine whether a contemplated change or new technology should be implemented.

While these tasks will vary somewhat across organizations, the list above reflects an underlying rational-planning methodology implemented in many disciplines, including engineering, computer science, and instructional design. The remainder of the paper takes each task and examines how it relates to practice. By way of conclusion, we pull out some key themes that may help point the way toward future research on the theory/practice interface.

Determine a need to assess the need. This task often precedes formal analysis. Change becomes an issue through a variety of circuits in an organization. Management may take the leadership and bring a technology to attention. Sometimes awareness is prompted by the mass media or by general professional practice.

Before an organization decides to assess a new technology or idea, several questions come to mind that may affect the eventual success of the adoption effort, including:

-How are concerns and issues shared and expressed within the organization?

-How do concerns reach a status of being serious and substantive, rather than speculative and peripheral? How does an innovation "get on the agenda"?

-Whose opinion counts in defining the issues?

-What is the relative value of outside versus inside information?

Every organization has methods-formal or informal-for doing the "evaluation that precedes the evaluation." Those institutional methods reflect the values, locus of power, and communication channels within the organization. Not every legitimate need for change is countenanced and presented for consideration; similarly, some issues or concerns may come up which serve the interests of key individuals rather than those of the whole group.

Create a committee or hire a consultant to conduct an adoption analysis. Work organizations are not pure democracies. That is, not every decision is made by the group-as-a-whole. Instead, a measure of efficiency is gained by specialization of function and assignment. Often in the case of a contemplated technology change, a sub-group is established to work on the problem, or an outside consultant hired to study the issue and make a recommendation. Task forces hold considerable local knowledge but, because innovations are often unfamiliar, local people may not have enough technical knowledge about the contemplated change. Conversely, consultants can provide needed expertise but they lack contextual awareness of the organization's unique needs.

Either way, there is an inevitable separation between the people assigned to work on the problem and the rest of the group. This separation leads to a decontextualizing of the problem. Task forces convene in the executive meeting room, have lunches brought in, and discuss the problem away from distractions. Consultants visit the production floor, interview a sampling of people, and fly home to write their reports.

Because of the separated nature of this work, communication becomes critically important. Both the technologies as well as the cultural practices of communication play a role in establishing the legitimacy of work-groups and consultants as they represent the interests of the whole organization.

Even if committee members are strategically selected to represent different interests, a change occurs when a person joins the special project and begins work on the committee. Because members spend time together and learn together, they create their own identity and sense of purpose independent of their origins. So necessarily, committee members take on a dual identity-representing their department colleagues while at the same time forging new alliances with other committee members. A significant level of cooperation and submerging of individual interests is necessary for the group to go forward with its study.

Develop a conceptual framework for conducting the adoption analysis. As committee members (or consultants) begin their work, they need some initial ideas to guide their inquiry. People express themselves, brainstorming and feeding in ideas, until a framework for thinking about the problem is established. The framework may be informed by theoretical models, or may reflect the best collective thinking and local knowledge of the group. In either case, a starting schema or framework is essential, and allows the work to go forward in some kind of logical order.

Just as the guiding concepts serve to orient, they also inhibit or steer away from other possibilities. Through the lens of the conceptual framework, participants are able to "see" certain events while missing others. Moreover, the stronger and more well-defined the framework, the stronger the blinding effect. Well-articulated models grounded in research and literature are the most potent kinds of frameworks, yielding powerful outcomes. At the same time, these powerful models are most likely to miss data outside of their scope and vision. This consequence of theory-using has been acknowledged by philosophers of science (e.g., Feyerabend, 1975; Kuhn, 1970) as well as technology critics (Arac & Johnson, 1991; Noble, 1991; Postman, 1995). Indeed, the critics are right that theories and models are just another technology used for solving problems. Accordingly, theory use has good and bad consequences, intended and unintended by its users, as do all technologies.

So are we suggesting that because theories can distort and shape our perceptions, they should be avoided? Not at all. A good theory-one that fits-is better than no theory at all. But we are suggesting that every "theory consumer"-people like us who appropriate concepts and apply them to local situations-should understand the risks involved. Theories, by their intrinsic nature, guide us to see what they predict we will see. Good consumers are aware of this self-fulfilling feature and will be careful not to over-generalize or over-react to situations based solely on theoretical evidence.

Collect, analyze, and interpret data bearing on the anticipated costs and benefits of the contemplated change. Recognizing that every change has costs and benefits to an organization, the committee or consultant (the "change agents") will go out and collect data that could be useful for decision making. We want to know the pros and cons, the costs and benefits, the intended outcomes and the unintended or undesirable consequences. We have identified four key reasons to be cautious in identifying costs and benefits of technologies and innovations:

1. The total impact of a technology cannot be foreseen, even by the best analysis. No research in the world is going to uncover all the impact and consequences of a proposed technology. The world isn't like that; it's too messy. Even after the fact, a technology's impact on the larger context is impossible to fully assess.

2. Negative consequences are harder to detect. Which is easier to appreciate: the touted, planned-for benefits of a technology, or its unadvertised side effects? We believe that planning processes suffer from an inherent bias that over-values technology, because technologies are specifically designed to yield an advantage, to solve a problem. The incentives to identify all the costs are just not there for the designer, and by the time negative consequences have emerged, the whole world may have adopted the innovation! This "innovation bias" provides further reason to assume a conservative stance toward technology adoption.

3. Some of the most important outcomes may resist quantification. Which tends to get the more attention: Measured and observable outcomes or hard-to-measure perceptions and intangible aspects of the local culture? We believe there is a "quantification bias" that systematically over-represents easily measured outcomes at the expense of other, equally important outcomes that resist quantification.

4. Many technology advocates suffer from "change-agent-itis." Once people have created or initiated and refined an innovation, they often feel an intense sense of identification with it. Other people in the organization can perceive that sense of identification - innovations are often known on the organizational street as "Sharon's project" or "Jack's baby." Indeed, change agents can develop a parent-like relationship with the innovation, promoting it in spite of flaws, or believing they have worked out the flaws. Consequently, change agents often do not welcome comments and suggestions from the feedback loop. Comments and suggestions are often seen as obstacles or attacks from those who don't understand and appreciate the hard work that has gone on.

Given these inherent sources of bias, all favoring the innovation, the best strategy might be a conservative one. Allow a margin of error; insist that the proposal demonstrate a safe or comfortable advantage over the status quo. Just as a statistician would insist on a margin larger than the standard error, we might insist on changes that can yield substantial benefits, enough to outweigh unforeseen negative consequences that may accrue.

This recommendation, though, flies in the face of many management consultants who advocate faster and faster change cycles and greater responsiveness to competitive conditions. Many people need to see clearly a compelling need for change to get them to move. Maybe the answer is not a simple go-slow, conservative response to change possibilities, but rather a greater appreciation for the costs involved, especially for those who find change difficult. Facing up to the full human and ecological costs of technologies can have a sobering effect on decision processes, and prompt a greater effort to support people in their learning and performance needs. This is good, no matter how slow or fast a company chooses to engage in change.

Evaluate the pros and cons and decide on a course of action. Recall the separation between the larger system and those charged with performing the adoption analysis. This separation is most keenly felt at the point of offering recommendations and deciding on a course of action. The ethic of evaluation is always complex and at least partially undefined. Participants enjoy substantial wiggle-room in determining what's "fair for everybody," or "best for the organization," or "clearly in our best interest." Moreover, the process of negotiation often extends past the change agents to the larger group. At this point, the larger group engages in a learning process similar to the one engaged in by the change agents as they completed the analysis.

If the proposed innovation is to be adopted, then develop and carry out an implementation plan.

It is commonplace that implementation plans tend to be neglected in comparison to the designing and promoting of technologies in the first place. This task, if done thoroughly and thoughtfully, would reduce the stress and violence felt by many underrepresented voices.

Technologies and innovations are often thought of in static terms, as though they had fixed meanings, stable across time. In practice, these changes are anything but stable and fixed. As time proceeds, many organizations tend to forget what it was they were adopting. And just as the technology evolves over time, its impact on the larger system is felt, resulting in continuing change at all levels. Nothing stays the same in dynamic systems!

This instability isn't all a bad thing. Often a technology evolves because it needs to be better fitted to the needs of the organization. Or as knowledge advances, so too does the technology, to better solve problems with fewer side effects. The point is, we should keep an eye on the innovation and make sure we know what's going on at any point in time. If things change, we would like to know about it, and know how those changes relate to conditions within the organization.

Collect information about the program's success, and modify as needed.

The feedback loop is another neglected task in the adoption process, given more lip-service than practice. Again, it helps to think of the innovation in somewhat flexible, dynamic terms. Users should feel permission to co-design the solution as it is fine-tuned and adapted to the local setting. The adoption process, then, moves away from a technical application model toward one of integration, adaptation, and accommodation. Human-factors engineers call this "participatory design," where end users become co-designers of the innovation, participating not only at the end, but throughout the entire design-and-adoption process.

Proceeding through the common tasks of analysis outlined above, we see a number of repeating themes, which can be expressed as simple recommendations:

1. Be careful when moving out of context. Analyzing a problem necessarily involves a separation from original context and a viewing from the outside. This separation is something we cannot avoid or eliminate altogether. As part of a problem-solving process, a measure of decontextualization can yield valuable insights, but at the same time requires careful monitoring in order to minimize its negative effects.

2. Respect the diversity within the full learning organization. Organizations and groups have a self-organizing, continuous-learning quality. Change cannot simply be mandated without larger repercussions. Indeed, the idea of top-down control is largely an illusion. Granted, it's messy to include lots of people in decision-making and design processes. But as Churchill said of democracy, it's better than any alternative we've been able to think up! At frequent intervals, people need to be brought in and included in the change process. Open communication is a key in respecting the diversity within an organization, and can prevent the costly surprises encountered by top-down mandates for change.

3. Watch for the biasing effects of theory application. Every practitioner has many layers of knowledge useful for solving problems on the job. Some of that knowledge gets represented in textbooks and theories in the field. This theoretical or formal knowledge can be tremendously useful to the practitioner, but at the same time, it can serve as "blinders" to a larger reality. Good practitioners make best use of theory by keeping an eye out for the unexplained, the anomalous, the unaccounted for. By combining the best available theory with our own tacit knowledge, we are likely to meet the needs of the local situation.

We are aware that the brief critique presented in this paper is itself a model or framework, subject to the very problems we are discussing! In this respect, we are like the postmoderns, who recursively turn their critiques toward themselves. But in good postmodern spirit, we want to be clear about what we're not saying: We're not saying that innovations shouldn't be adopted; we're not saying that models shouldn't be used for planning and decision making. We are saying that informed participants make better decisions than naïve ones do; we need to be good consumers of adoption models. Not only should we take care in choosing good models to guide our practice; we should take care in how we appropriate and use those models as technologies themselves.

Our exploratory exercise stresses the importance of thoughtful practice over decontextualized theory. We have tried to show how critical it is to appropriate theory in an informed, adaptive way. We encourage future research in the adoption and diffusion of educational technologies to be fully grounded in practice, and sensitive to the variety of ways our theories are used.

Arac, J., & Johnson, B. (Eds.). (1991). Consequences of theory. Baltimore MD: Johns Hopkins University Press.

Feyerabend, P. (1975). Against method: The outline of an anarchistic theory of knowledge. London: NLB.

Kuhn, T. (1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press.

Noble, D. D. (1991). The classroom arsenal : Military research, information technology, and public education. London: Falmer.

Postman, N. (1995). The end of education : Redefining the value of school. New York: Knopf.