At a national level, there is a need for more research to develop models of technology adoption and diffusion as applied to complex, decentralized organizations with a multitude of partnerships. Organizations too often purchase hardware and software with little planning and/or analysis to understand how to address challenges such as resistance to change, alignment with infrastructure and with school mission, and involvement of those who must work with the new technologies. Some development models exist, but few are understood in the context of an elaborate, complex system with a high risk population and multicultural overtones. A model needs to be established, adapted, and evaluated in a real situation that reflects the "messiness" of large, systemic integration of new and changing technologies complicated by new and changing partnerships. Once developed, this model and the feedback it generates could be used nationally where large institutions work with multiple partnerships attempting to use technology effectively.
To accomplish this goal, we propose to adapt Farquhar and Surry's (1994) model of technology adoption with research on large-scale collaboration and apply it to the efforts of the Navajo Community College (NCC) in establishing the Navajo Learning Network (NLN). We would support the NLN by helping the college to implement the model and providing ongoing feedback that will assist them in their goals to provide high quality education for all educational levels. The end result of this developmental research would be an indepth case study that examines the effectiveness of the technology adoption model and the variables that influence its effectiveness in meeting students academic needs.
RELEVANT LITERATURE AND MODELS
The literature in technology adoption and diffusion reveals that two
approaches are
generally used to gather data and create models. One looks at technology
adoption from the
viewpoint of the user, the other focuses on the complex needs of the
organization. When a
new technology project is to be introduced to an large educational
system, there must be a
comprehensive plan of action which addresses the needs and
characteristics of all users
(teachers and students) as well as the manner in which they perceive the
new system. New
projects also need to consider organizational factors: the physical
environment and the
human support systems provided by the organization in which the learning
network project is
to be implemented. In the past, most technology adoptions have occurred
in a single
organization of varying size. When the project involves a number of
organizations, agencies,
and individuals, collaboration issues also need to be addressed. The
following is a review of
findings to identify barriers and breakthroughs related to individual and
organizational factors
in technology adoption, and models for large-scale collaboration.
All technology adoption needs to incorporate a learner support system, especially for at-risk students. Porter (1994) found that students participating in the New Directions in Distance Learning (NDDL) project in Vancouver, British Columbia, needed both support and direction to enable them to make the transition from traditional classroom environments to self-directed learning. Specifically, these students needed tools to help them monitor their progress and obtain timely feedback on their activities, including guidance in putting information together, reaching their teachers and tutors, planning their studies, and charting their progress to help them submit assignments on time. They found that when teachers provided this type of support and mediation via the NDDL electronic learning network, students' course completion rates increased dramatically.
Teachers' perceptions of the attributes of the learning network also need to be considered. In a national study of K-12 teachers who introduced technology and telecommunications into the classroom, Honey and Henriquez (1993) identified five critical barriers: network connectivity, equipment availability, teacher training, technical support, and time for individual experimentation, exploration, and practice. Similarly, in a study of a southern California school district implementing a large-scale telecommunications network, Levin (1995) identified an additional set of factors which influence the effective use of telecommunications in high school science classrooms: teacher perceptions of the new technology, potential of the network for curriculum integration, administrative and financial support, teacher-teacher communication, and teacher-student interaction. To this list of technological considerations, Wilson, Ryder, McCahan, and Sherry (1996) added affective considerations, especially intimidation by people perceived to be technical experts, lack of perception of a clear payoff for using the new technology, concerns about having to learn too much technical jargon and too many commands to make the transition worthwhile, and a need for hand-holding in the early stages of adoption.
In contrast, there have been breakthroughs among teachers using computers and other instructional technologies to enhance and enrich their teaching. The Office of Technology Assessment (OTA) has identified four interrelated individual and organizational factors that must be met if teachers are to use computers and networks to enhance and enrich their teaching:
Kell and others (1990) identify five success factors that are similar to the above list:
Finally, Talab and Newhouse (1993) cite Bichelmeyer's (1991) doctoral dissertation, in which she found that teachers adopt technology innovations according to a hierarchy of needs, with the most basic needs generally being fulfilled before the higher ones. Successful programs have matched the identified needs of the teachers with the resources which are available to them. From basic needs to higher needs, these comprise:
It is evident that there are three classes of independent variables here: technology variables (time, access, dependability), user perceptions (seeing a clear advantage to using the new system, intimidation by technology), and organizational variables (ownership, control, and a voice in the implementation process). These concerns can only be dealt with when there is a systemic plan in place, taking into account the context in which the new technology and reform effort will be used, where it will be used, who will use it, and how it will be used and maintained. At this point, a coherent model is needed to tie all these factors together.
A MODEL FOR ANALYSIS OF TECHNOLOGY ADOPTION
One starting point is a recently developed model of adoption
analysis by Farquhar and
Surry (1994). They state that if a new instructional product is to be
successfully adopted and
diffused through an existing system, then its developers must determine
how all participants
perceive the new innovation. They base their current model on Rogers'
(1983) model of
diffusion of innovations within an organization. There are three
critical elements of the
original Rogers model: the stages of the diffusion process, adopter
categories, and
characteristics of the adoption process.
Each adopting situation is unique; each requires an adoption analysis to enhance the likelihood that the innovation will be adopted and effectively utilized at the user sites. A small investment in analysis early in the implementation process can pay off later on in large increases in utilization.
Farquhar and Surry (1996) have developed an adoption analysis tool, that is, a method for identifying factors present at a user site that might impede or facilitate the adoption of an instructional product. This tool clarifies the context in which the innovation will be implemented, and informs key decisions throughout the development and implementation process. It consists of a series of data collection activities, with the intended outcome being a response to lack of adoption or utilization. This adoption analysis method is based on an examination of organizational and individual factors that influence adoption:
Exhibit 1, roughly adapted from Farquhar and Surry (1994), summarizes the critical individual and organizational technology adoption factors which generally apply to large-scale learning networks:
INDIVIDUAL FACTORS
User Characteristics
Perceived Attributes
Physical Environment
Support Environment
Since many partners are involved with the NLN, we must also focus on models of collaboration and coordination. One of the main inhibitors of systemic change is the resistance that individuals, groups, organizations and agencies bring to a common project from their past histories of service provision. Collaborators must be willing to listen and adjust their approaches to the needs of the current project and their new partners. They must be willing to adapt and in some cases even abandon old methods of behavior in order to forge a new comprehensive approach with their current collaboration. In the early stages, individuals and groups often attempt to couch new problems into past personal and organizational strategies, and the results are often similar to the parallel play of children. Give and take must occur and new structures and processes must be implemented for the new alliance to be successful. These structures and processes must be flexible to meet the changing project needs while simultaneously holding to the common mission. Melaville, et al. (1993) propose a five stage process that leads to ever-greater collaborative sophistication:
PROJECT DESIGN
The proposed study will use a developmental research approach, with
the NLN as a
case study. It will examine a unique, challenging site that is adopting
a new technology that
necessitates the creation of a complex human network. The technology
adoption will affect
and be affected by the political structure, with its issues of ownership
and authority;
instruction in the classroom, with its issues of design, control, a
networked learning
environment, and learner support; the culture, which seeks to protect the
Navajo heritage
while enabling students to reach state, local, and national standards;
and the collaboration
of all stakeholders, including tribal leaders, administrators,
legislators, outside agencies
and service providers, parents, and other members of the community.
BACKGROUND
The main campus of the Navajo Community College is located in Tsaile, a
small town
on the high and arid Navajo reservation in Arizona. A number of branches
of the college are
spread throughout the reservation, occasionally crossing state lines, as
with the Shiprock,
New Mexico, branch. Parts of the reservation are in Arizona, New Mexico,
and Utah. The
NCC has a plan and funding to link all the remote Navajo communities
together through a
telecommunication network called the NLN. This would bring lifelong
learning to isolated
communities that have difficulty in even obtaining quality telephone
service. An exciting
element and challenge in this project is focused on the degree of
collaboration necessary
from a large number of political and educational agencies. Individuals
concerned about the
success of this project include tribal officials, classroom teachers,
students, parents,
scientists, mathematicians, private vendors, and others from inside and
outside the
reservation.
The NCC has received funding to establish a Navajo Learning Network from the National Science Foundation, National Aeronautics and Space Administration, the Department of Commerce, and the Air Force Office of Scientific Research. This network includes Mac and PC workstations at Tsaile, AZ; Shiprock, NM; Tuba City, AZ; Crownpoint, NM; Chinle, AZ; and St. Michaels, AZ. These stations are, or will be within the year, connected to NCC and the Internet. Users will have access to e-mail and the World Wide Web. The NLN plans to support K-12 use of the Internet through their system and hopes to eventually connect to the Chapter Houses (political and cultural centers of the Navajo tribe) if assistance is provided from the tribal leaders. The NLN will provide assistance to individuals and institutions, emphasizing the areas of Navajo language, history and culture, and teacher education. NCC has been asked by its funders to coordinate with other educational projects such as the Los Alamos National Lab, the Rural Systemic Initiative, and other schools and colleges in the region.
In February 1996, a work session was held in Tsaile with the NLN staff and funding agencies, NCC staff, in-state institutions of higher learning, outside telecommunication projects personnel, faculty from neighboring schools, tribal officials, representatives from other electronic networks in the state, private vendors, and research consultants. The goal of the work session was to get feedback on the project and to develop partnerships crucial to a project of this magnitude and scope. The discussion began with NLN project leaders presenting an overview of the status and plans of the NLN. Most of the efforts that have taken place so far have centered on installing the electronic structure. This task has been made more complicated due to the remoteness of the locations, the lack of technological base, and political complications. The NLN plans to address training and instruction issues, but the current main concern is access. A technician has been hired to work at Tsaile with plans to train students for onsite assistance.
After the introduction of the project by the NLN, the discussion flowed around topics on coordination and collaboration, planning, training and support, curriculum and instruction, and the maintenance of the system. The group suggested strategies that had worked in their programs to address each of these concerns. At the end of the day, the participants discussed how they could partner and support the NLN. This work group established an initial baseline to document the status of the NLN in its efforts towards the adoption of the new technology. (See Appendix B.)
RESEARCH DESIGN
A Type 1 developmental research approach, in which a program, process, or
product
used in a particular situation is described, analyzed, and evaluated
while being developed, will
be used as the research methodology for this study (see Richey and
Nelson, [In Press]). A
collaborative and large-scale technology adoption model will be
concurrently developed and
evaluated through its application to the NLN.
Developmental research case study methodology is appropriate for this proposed study because it results in context-specific knowledge and serves a problem solving function. It describes and documents a particular design, development, and evaluation project; utilizes a range of traditional research methodologies in addition to standard design and development procedures; draws conclusions from research which are highly context-specific; and tends to serve as a dissemination vehicle for exemplary design, development, and/or evaluation strategies. Using the developmental research approach necessitates having the researchers play an active role in providing some technical assistance. Specifically, researchers will facilitate coordination/collaboration among stakeholders with different agendas. We will work with school staff who have implemented parts of the system, and with interested members of the tribal and educational community. In addition, we will observe and record progress toward restructuring of the current system as the schools adopt technology and new curriculum. Exhibit 2 details the approach RMC Research proposed for its developmental study. The four stages are loosely based on those outlined by Melaville et al. (1993), Rogers (1983), and Farquhar and Surry (1994).
| CRITICAL FACTORS | OUTCOME | INSTRUMENT |
Stage 1: Establish and
Maintain Collaboration
|
Collaboration and coordination with staff, tribal communities, K-12 schools, institutions of higher learning, funding agencies, vendors, telecommunication projects, professional organizations, community groups, federal agencies, private schools, laboratories, federal initiative groups |
Researchers journals and
field notes from meetings
with constituents
Interviews |
Stage 2: Develop a
strategic plan of action
|
Cohesive plan of action implemented with commitment from participants |
Researchers journals and
field notes from strategic
planning meetings
Follow up interviews Documentation (plans, memos, etc.) |
|
Stage 3: Adopt the technology and clarify its relationship with the
organization
a: Identify organizational adoption factors
|
Understanding and generating the necessary conditions for the successful adoption of the electronic network |
Researchers journals and
field notes
Surveys Interviews Classroom observations Server logs Documentation |
|
Stage 3 b: Identify individual adoption factors
|
Large number of skilled users on reservation |
Surveys
Interviews Classroom observations Server logs |
Stage 4: Dissemination
|
Knowledge of use of technology and feedback for continuous improvement |
Monthly interim reports NLN Home Page Final report |
Data gathered will focus on five research questions
Stage 1: Establish and Maintain Collaboration
Research Question #1. How does a complex organization such as the Navajo Nation and all its associated partners convene, explore how to improve services for their community, and agree to work together on a unifying theme?RMC Research will document activities and provide support to the NLN informing them with technology adoption models, assisting in facilitation and coordination, and providing ongoing feedback on status of the project. Initially, researchers will build on previous work group meetings with NLN staff to discuss information that would be helpful to them. Then RMC Research staff will meet with key constituents to help clarify partnerships, resources, and lines of communication. We will provide this feedback directly to NLN staff and to all partners and interested parties via the NLN Home Page on the World Wide Web. This developmental research will involve the use of contextual review procedures that provide a way of updating the project plan, building project teams, promoting clear communication, and providing opportunities for dissemination and diffusion of project products. After initial meetings, RMC Research staff will conduct a series of interviews of participants selected from each of the identified stakeholder groups over the duration of the project to determine whether collaboration is occurring with all groups and whether all stakeholders are beginning to see a clear advantage to implementing the system. Interviews will be structured, consisting of approximately 20 questions, with an average interview lasting approximately one hour. The number of interviews will be determined by number of participating groups and available time. Feedback from the interviews will be synthesized and provided by the following month so that all partners will have access to the same crucial information that will permit them to establish a shared vision of the project.
Stage 2: Develop a Strategic Plan of Action
Research Question #2. How do partners come to agree to focus on a specific innovation, design a prototype system that incorporates the elements of their shared vision, and develop the technical tools and interagency agreements needed to put their plan into action?Within the first six months of the project, RMC Research will conduct a series of meetings to determine the status of a shared vision, goals, objectives, and agendas by NCC and NLN leaders, together with members of partner organizations and tribal representatives. Any organizational changes that taken place in an effort to support the diffusion of technology will be examined and noted. Information from these meetings and follow up interviews, as well as any documentation from the project such as a strategic plan, memos, and minutes, will be used to support findings. Feedback will be provided to the NLN leaders and participants within a month, both in a paper-based interim report and in electronic form on the NLN Home Page.
Stage 3: Adopt the Technology and Clarify Its Relationship with the Organization
Research Question #3. How does the social system, with its range of potential adopters (from innovators to laggards ) and its diverse perceptions of the proposed innovation, implement the prototype, adopt it in the classroom, and effectively diffuse it through the social system?Within the first year of implementation, the adoption process will be studied by administering descriptive surveys, together with follow up interviews. Server log statistics and classroom observation will also be used to see how individual teachers are beginning to adopt the new technology and implement telecommunications-based activities in their classrooms. All of these activities will serve to identify suggested improvements in the project, the conditions which promote successful use of the project, the impact of the project on the Navajo Nation, and the conditions which are conducive to effective design, development, and/or evaluation of projects of this type.
Periodic surveys of all teachers, staff, and trainers at NCC will be conducted to determine their frequency and type of usage of the instructional network, to identify barriers and successes, to assess the formation of a shared knowledge base and the building of a virtual educational community of teachers and students, to document student performance over the duration of the project, and to find out what is and is not working. The survey instruments will be similar to those used by the national survey of Honey and Henriquez (1993) and the U.S. survey of K-12 telecommunications use by Heaviside, Malitz, and Carpenter (1995). (The survey by Heaviside et al. was pre-tested with respondents like those in the survey population, and was also extensively reviewed by the National Center for Education Statistics to check for consistency of interpretation of questions and terms and to eliminate ambiguous items or instructions.) Descriptive statistics will be used to report the results of forced-choice questions. Open-ended questions will have their results grouped into a small number of categories, with two raters sorting each set, to determine the inter- rater reliability of the sorting process.
Since self-reporting involves perceptions of participants rather than external observations by researchers, three other instruments will be used to provide validity. Survey information will be cross-checked with interviews and classroom observations. Server logs will be gathered periodically to determine patterns and types of use, types of resources accessed, and types of lesson plans and classroom activities generated by NLN participants. Again, this information will be fed back into the system on at monthly basis.
Stage 4: Dissemination
Research Question #4. How is information about the outcomes of the technology adopters disseminated and received by all interested stakeholders and what mechanisms are used to receive feedback for subsequent revisions and refinements of the innovation?As stated earlier, monthly updates will be provided in a report form and on the NLN Home Page to provide ongoing feedback. In addition, a final report will be written and delivered to NLN partners before the end of the third year. Final reports of developmental research usually take the form of a case study with retrospective analysis. The final report from this study will include a disciplined analysis of the problem, context, constraints, learners, and task, as well as a disciplined synthesis involving the design and creation of the learning network, its associated knowledge base, and its training component. This report will be made available over the World Wide Web through an electronic journal such as the CMC Journal or the Electronic Journal of Virtual Culture, as well as being submitted to ERIC. Presentations will also be given at national conferences, such as the annual meetings of the American Educational Research Association (AERA) and the American Association for Educational Communications and Technology (AECT).
Our developmental research approach to this complex instructional problem directly addresses the perceived needs of the NLN partners and their students, and provides a general model for developing and adopting similar innovations by decentralized educational organizations. Our contextual review procedures provide a clear means for enabling the diverse partners to collaborate in developing a shared vision and mission statement, building project teams, promoting clear communication, updating and revising the project plan and its concomitant organizational components, evaluating the success of the innovation, and providing opportunities for dissemination and diffusion of project products.
Stage 5: Going to Scale
Research Question #5. What is the efficacy of the proposed model?
The purpose of this study is to create an effective, replicable national model that deals with adoption of technology by complex social systems with numerous needs and partnerships. Though the proposed research on the NCC and its associated K-12 schools, tribes, and other social structures is site-specific, the goal is not only to support the adoption of a learning network that will provide quality instruction to the Navajo Nation, but to "fine tune" a model that will generalize to other social systems large in size and scope with diverse needs. The model will continually be evaluated for fit and effectiveness throughout the study. Researchers will examine the critical factors influencing technology adoption/diffusion through the NLN and relate them to the intended outcomes. In doing so, this research will inform any project that addresses technology adoption and diffusion challenges.
STAFFING
RMC Research proposes a team of experienced professionals to conduct the
research
inquiry. Dr. Dianna Lawyer-Brook is proposed as project director for the
evaluation. Dr.
Lawyer-Brook has been a project director, technical assistance provider,
curriculum
developer, and evaluator for many innovative technology-based projects.
She has directed
national and international projects to link students and teachers via
interactive television to
enhance student achievement in math, language arts, and geography, as
well as to facilitate
teachers skill acquisition in these areas. She has evaluated distance
learning and other
technology-related projects using a variety of methods, including survey,
content analysis of
journals, interviews, observations, and performance tasks. In her role
as project director, Dr.
Lawyer-Brook will be responsible for the conceptualization of the
research design and
analytic frameworks to be used for this project. She will also
administer the project, oversee
all tasks, and oversee the writing of all reports
Dr. James Kettering and Ms. Lorraine Sherry are proposed as members of the evaluation team. Dr. Kettering has evaluated many large and small-scale educational projects and has served as a technical assistance provider for several technology projects. Dr. Kettering also has a strong curriculum background and has many years of experience facilitating the planning, implementation, and evaluation of innovative projects and school improvement activities. He has conducted and coordinated multiple projects focused on American Indian concerns, most recently helping the Navajo community to implement a project that uses interactive television to provide Title I programs from a university to private school students on the reservation.
Ms. Sherry has evaluated two large-scale educational telecommunications projects. She has served on the Internet Task Force of a major university and directed the development and implementation of the Home Page for the Graduate School of Education. She has written several journal articles and given presentations at national professional conferences, reporting on her ongoing research with educational telecommunications.
A fourth member of the team will be a graduate assistant from the NCC. As such, this project will benefit from an onsite data collector and will support the development of Navajo evaluators. Resumes for proposed staff may be found in the Appendices.
ADEQUACY OF RESOURCES
RMC Research has proposed a budget that enables us to be efficient and
effective in
collecting and disseminating research. Details may be found on the
budget justification
pages.
RMC Research operates a fully equipped, modern office with current technological capability and human support. Our budget includes salaries and benefits for 2.0 FTE professional staff and 0.33 FTE secretarial support. It also includes travel to the site, attendance at a national conference for dissemination purposes, printing, postage, telephone, use of computers, and other incidental costs. All of RMC's rates have been approved through federal audit procedures.
MANAGEMENT PLAN
The proposed project director, Dr. Lawyer-Brook will devote 75% of her
time to this
study. Her responsibilities will include budget oversight, research
design, supervision of
data collection and analysis, survey design and analysis, and report
generation. Dr. Kettering
(75%), Ms. Sherry (33%), and a research assistant located onsite (33%)
will serve as
teammates to Dr. Lawyer-Brook and will offer support in all phases of the
study.
Timelines for completion of all major tasks are presented in Exhibit 3. Benchmarks are listed.
RMC Research is assertive in its belief in affirmative action. In all of our work and in selecting all employees, we ensure that persons who are eligible to participate in our projects are selected without regard to race, color, national origin, gender, age, or disability.
A description of RMC Research's projects related to this proposal may be found in Appendix E. RMC Research has a solid track record of completing projects in a timely fashion without cost overruns.
STAGE 1
Meeting with National Learning Network
staff
July 1996
Interviews on collaboration issues
August 1996-May 1999
STAGE 2
Meetings on vision and strategic plan
August -December 1996
Follow up interviews
Ongoing
Review documentation
Ongoing
STAGE 3
Periodic surveys, interviews and classroom
observations to determine individual and
organizational adoption of the technology
January 1996-May 1999
Examination of server logs every three
months
January 1996-May 1999
STAGE 4
Monthly reports on the Home Page
August 1996-May 1999
Final Report
June 1999