This survey research study evaluated the extent to which high school science teachers and their students in a Southern California school district were using computers and networks in their classrooms during the 1992-93 school year. The findings suggest that installing hardware is only the first step in a complicated process of integrating telecommunication use into the high school science curriculum. Even with extensive training and some networking hardware in place, technological barriers such as time and access, administrative barriers such as financial and technical support, and affective variables involving teachers' perceptions, teacher communication, and curriculum integration continue to inhibit the widespread use of telecommunications in high school science classrooms.

The issue of district-wide training in, and implementation of, telecommunications infrastructures in school districts is an important one, especially since these represent major projects with significant grant funding and national scope. Many researchers want to know more about how to improve training, how teachers respond to it, what challenges they face, and what their felt and perceived needs are. Though Levin's study (1992-93) predates the Boulder Valley Internet Project (BVIP)--also funded by the NSF--by two years, our current results replicate hers.

The title of the study is misleading--the issue is telecommunications in particular, not technology in general. Levin's definition of telecommunications limits it to the use of a computer and telephone lines; it excludes videoconferencing and other analog techniques. A reworded title, "Teachers using telecommunications: Barriers and breakthroughs", would better describe the problem under investigation, namely, "to look at a school district in the process of building a telecommunications infrastructure, to observe the impact of [two independent training] projects on the use of telecommunications in the district, and to document how high school science teachers and students are currently using technology and telecommunications" (p. 55).

The goals of the two training projects were clearly stated. The CoE (Community of Explorers) project was intended to link five high schools, train about 15-20 science teachers to use telecommunications, and create a virtual community of teachers and students who could then exchange curriculum materials through network communications. The STEP (Supercomputer Teacher Enhancement Program) provided telecommunication training as staff development to 40 math/science teachers from under-represented minorities, with the hope that they would integrate new-found resources into their curriculum.

The study of these two projects was exploratory and observational - it was not directly linked to any substantial theoretical base, nor were any hypotheses posed. Some findings were compared with the national study of Honey and Henriquez, in which the teachers were self-selected. This limitation was not imposed on Levin's study: her survey of all high school science teachers in the district addressed previous experience, computer use and training, and teachersŐ expectations of using telecommunications in the future.

Follow-up interviews with several CoE and STEP researchers and district personnel addressed administrative issues such as financing, hardware/software requirements, and teacher training. Follow-up interviews and classroom observations were conducted to provide triangulation and to verify that the survey data were consistent with interviews and observations.

At the time their study was conducted, electronic communication within schools was generally limited to e-mail and Eudora, local area networking, and advanced applications such as gopher, telnet, Fetch and ftp, and Mosaic. There were few studies of widespread telecommunications use, though the research is rich in studies of educational networking, distance education, the use of analog technologies, and the process of institutional change. Of those few studies, Levin cites Honey and Henriquez (1993), her own previous research, and that of Margaret Riel, a prominent AERA researcher and telecommunications advocate. Since the paper was dated 1995, and the most recent citation was 1994, the federal survey on telecommunications in 1500 public schools throughout the USA by Heaviside, Malitz, and Carpenter (1995, January) is conspicuously absent. Also, since this paper deals with institutional change, other related literature of a more theoretical nature should have been included, notably Rogers (1983) on Diffusion of Innovation, Farquhar and Surry's (1993) work on Adoption Analysis, and the series entitled Beyond Implementation edited by Michael Fullan.

One of the major strengths of Levin's study is that her sample comprised the entire population under investigation - all high school science teachers in the district. This attempts to eliminate the self-selection bias characteristic of the Honey and Henriquez study. However, the response rate was just 53% (78 out of 147), even though Levin attempted to guard against non-response bias by sending the Technology and Telecommunications Surveys (TTS) to all participants by US mail, and by instructing them to use the inter-district mail service to return the completed surveys to the district office. Hence, we cannot be sure that the self-selection bias has been completely eliminated. Note, however, that Levin conscientiously avoided the even lower response rate that plagues e-mail surveys by using surface mail instead.

By sending the survey to all science teachers in the district, Levin was able to contact the approximately two-thirds of the population who were not telecommunications users. Thus, the variables for the entire population included gender, number of years teaching science, highest degree earned, perceived expertise with computers, inservices attended, frequency of computer use, access to computers and modems, use of computers for various purposes by teachers and students, and perceived most valuable computer and telecommunications resources. Additional variables for the telecommunications users included level of expertise with telecommunications, types of formal or informal training in telecommunications, type and location of Internet connection, perceived value of several kinds of informal assistance, and most widely used telecommunications resources. Note that many of these variables are technology-relevant, not specifically telecommunications-relevant. This may be the reason why Levin titled the paper "Teachers using technology", rather than "Teachers using telecommunications"--even though the expressed goals of the two training projects were telecommunications-specific.

The variables addressed in the survey were measurable--number of years experience, percent of teachers having computers at home, percentage of teachers using computers for word processing, simulations, telecommunications, and so forth. Frequency of computer use was measured on a subjective scale, "daily--often--sometimes--never", which is notoriously difficult to quantify. These were simply tallied and reported as charts.

Other intervening variables--predominantly affective variables - were not quantifiable, and were not addressed. A list of "challenge statements" to be answered on a Likert "agree-disagree" scale would have been helpful to further investigate perceived barriers. Some useful statements could include: "I feel intimidated by the technology", "I feel I lack the support of my colleagues in revising my curriculum to include Internet resources", "I donŐt feel as comfortable as my students in navigating the Internet", and the like.

Both reliability and validity are weak points of this paper. The issue of reliability was not addressed at all. Since a single survey was given to the entire population, equivalence and stability reliability do not apply. The author states, "[t]welve high school science teachers from five schools were also interviewed and observations were conducted in their classrooms to verify that the data collected in the survey was consistent with what was said in the interviews and observed in classrooms. (p. 55)." No contradictions were reported. Levin does not state how many interviewers were used, whether or not the interviewers were trained, whether or not the interviews were taped and transcribed verbatim, nor whether inter-rater reliability was investigated.

Validity--and construct validity in particular--refers to "whether the test measures what we intend it to measure". Self-reporting may be valid from the respondent's own viewpoint, but it may not truly represent the actual situation. For example, Levin states that "[t]he teachers participating in the CoE and STEP projects had formed communities both on and off the network", using both e-mail and telephone communications. Crook (1994) states that one of the hallmarks of a virtual community is its longitudinal continuity, i.e., the existence of a common base of shared knowledge and experiences. Though communication is going on, does it actually represent a knowledge-building process? Was there any evidence of a shared knowledge base? Levin did not explore this at all. If not, then even if teachers are reporting their frequency of communications accurately, the inference that this represents the development of a virtual community--one of the expressed goals of the CoE project--has little validity.

Data collection comprised gathering the 78 completed paper-based surveys from the district office and examining reports from interviewers and classroom observers. Beyond this, there was no further description of the process of data compilation, cross-checking of the three different data sources, or analysis. Results are presented graphically, so we can only assume that the data were entered into a spreadsheet which then generated a bar chart for each question. Short write-ups follow each chart. These were accurate but not clear - it was difficult to follow some of the calculations from the write-ups. For example, on p. 63, Levin states that "[s]eventy-five percent (18) of the TTS respondents who completed the telecommunication section of the survey, use the Internet"; the chart on p. 58 states that "only 31% of respondents used telecommunications"; and the statement on p. 56 reads, "[t]hirty-one percent (24) of the respondents identified themselves as telecommunication users".

The section titled "Interpretation of the survey findings" contained both data and interpretations. When inferences were made, Levin presented them together with factual data. On p. 65, she reports that survey respondents appeared to have positive outlooks on the use of technology and telecommunications in education, followed by six examples of the way in which computers brought about a positive impact on classroom activities. However, the last paragraph on p. 61 contains the following ambiguous statement: "While the highest occurrence of computer use in science classrooms was by students using simulation programs, the computer was used as a presentation tool by 31% of the respondents. This indicates a fairly high use of the computer by the teacher instead of by the students in the classroom". It would be advisable to keep interpretations and inferences separate from factual information by presenting them in two separate sections titled "Findings" and "Discussion".

One very important finding was that less than half of the TTS respondents who currently used telecommunications had integrated network resources into their curriculum. Four reasons for this non-integration were identified through teacher interviews - time, access, difficulty in matching resources with curriculum content, and difficulty in using effective strategies to enhance teaching with Internet resources. A second finding was that financial and administrative support was irregularly distributed throughout schools within the district. This brought about inequity in access, since computers were not available in all classrooms; few classrooms had computers available for student use; and phone lines and modems were located in areas which were difficult for students to access. These findings are in agreement with those of other researchers studying telecommunications implementation projects in other districts (Honey & Henriquez, 1993; RMC Research, 1996; Heaviside et al., 1995).

Generalizability to equivalent populations was dealt with by Levin's comparison of her results with those of Honey and Henriquez. This is one of the strong points of this paper. The differences between these two studies, especially with regard to Internet use, can be explained by the fact that Honey and Henriquez's respondents were self-selected, and thus show a greater frequency of advanced Internet uses. Results from a third study (RMC Research, 1996) - also district-wide and not self-selected - indicate that the predominance of e-mail use is about the same as that reported by Levin, but the frequency of advanced uses lies in between Levin's and Honey and Henriquez's results. This can be explained by the greater amount of time (two years rather than one year) the BVIP teachers have had to gain comfort and expertise using various Internet tools and resources. The level of non-integration of telecommunications into the curriculum is about equivalent in both the Levin and the BVIP studies.

Levin's conclusions are tempered with the practical meaning of her reported results. She notes that the NSF funding, along with voter referendums within the district, provided the seeds for telecommunications links in selected schools, which then began to spread throughout the district. Though computer expertise was evenly distributed among the science teachers, the telecommunications network was underutilized. Levin states, "[b]uilding the infrastructure was not enough to successfully implement telecommunications use in high school science classrooms" (p. 68). She identifies five major barriers to more widespread telecommunication use - equipment availability, telecommunications access, teacher training, technical support, and time. These are the same barriers that are cited throughout the current literature on this subject.

In sum, Levin's work is worthy of the refereed journal in which it appears. The report and its ancillary information - charts and bibliography - are sufficient, clearly presented, and interesting,with one exception: the write-ups following each chart could have been a bit longer and clearer. Weak points include scant information about data collection and compilation, reliabity, and validity; lack of study of any affective variables influencing telecommunications adoption and diffusion; and lack of descriptions concerning any shared knowledge base, technical support, or learner support for those science teachers who were beginning to use telecommunications. Strong points are Levin's use of the entire population rather than a sample, the replication of her results by other researchers, and the generalizability of her findings to other districts which are currently integrating telecommunications into the classroom with both local funding and grant money.

References

Farquhar, J.D., & Surry, D.W. (1994). Adoption analysis: An additional tool for instructional developers. Education and Training Technology International, 31 (1), 19-25.

Heaviside, S., Malitz, G., & Carpenter, J. (1995, January). Advanced telecommunications in U.S. public schools, K-12. Washington, DC: US Department of Education, Office of Educational Research and Improvement. (Available from Judi Carpenter, 202-219-1333.)

RMC Research Corporation. (1996). Boulder Valley Internet Project - Initiative II Interim Report. Available from RMC Research Corporation, 1512 Larimer Street, Suite 540, Denver, CO 80202.

Rogers, E.M. (1983). Diffusion of innovations (Third Edition). NY: The Free Press.

Back to Research Critiques
Back to REM7500 page