Background, Purpose, and Rationale

• to enable our programs, and later the School of Education as a whole, comprising students, faculty, and staff, to join the Internet culture,
• to develop a knowledge-building environment for graduate classes and seminars, and
• to conduct research on technology adoption and supporting learners as they gained expertise in use of the Internet.

To gain an early focus on our users and their tasks, we tapped the expertise of our members who had already performed needs analyses and research on technology adoption (Wilson, Hamilton, Teslow, & Cyr, 1995; Sherry & Morse, 1995). Two concurrent research projects were planned: a focus group interview with thirteen students in the IT doctoral program who had been actively using e-mail and the Internet for nine weeks, and a questionnaire that was distributed to most of the ILT classes and the IT seminar. Two follow-up research projects were also planned: an e-mail interview in the fall of 1995 with the thirteen IT seminar students, and interviews with four faculty members in the ILT department. Participation was voluntary, confidentiality was assured, and all participants were informed about the eventual use of all data collected.

Using these converging lines of inquiry, our team addressed five research questions:

• What is your current level of Internet use?
• Do you have access to the appropriate technology?
• How would you like to use the Internet?
• What challenges do you face?
• What aids and supports do you feel would be helpful?

The results of the four research projects allowed the ILT/IT faculty to make some decisions about the types of aids, supports, courses, and workshops that students, faculty, and staff might find most useful. We found that our community of learners had good e-mail access. Some respondents were already participating in e-mail discussions as part of the doctoral seminar. Most respondents preferred formal courses and short workshops to tutorials, brochures, or booklets. We also found, as in Pelton and Pelton's (1996) study of preservice teachers' attitudes toward technology, that although access was becoming easier and more widespread, many participants were hesitant to include the use of e-mail in the teaching/learning process. Some students find it too time consuming to fight" the technology or to find the information they need. One of the faculty members feels that the World Wide Web (WWW) is not a credible place to either place a paper or to find published research of fellow academics.

Based on these results, the Internet Task Force dropped its efforts to produce on-line tutorials and paper-based job aids and began to concentrate its efforts on building a home page on the WWW. The home page included on-line information for accessing instructional and reference materials, getting Internet help, and keeping participants abreast of the latest changes to the academic programs. Task Force members also gave demonstrations followed by hands-on practice in the IT doctoral seminar and began to coach other students face-to-face and on-line. Concurrently, our faculty advisor developed a set of policy statements and strategic plans for Internet use within the ILT/IT program. Faculty began to offer courses in telecommunications and HTML authoring.

Two years have passed since the initial survey results were published in the TSS Newsletter and in the AECT In-CITE'96 proceedings paper. Meanwhile, both the overall usage and the usability of the Internet have changed dramatically. Currently, many classes and seminars require students to participate in Colorado Education On-line (CEO) conferences and use e-mail to discuss relevant topics between class meetings. Some classes have collaboratively produced annotated bibliographies on-line. Students are creating web pages and electronic portfolios. Faculty and students are authoring on-line documents and scholarly products such as literature reviews, proceedings papers, and journal articles.

Because of the evolution of the Internet and the changes in our learning community's patterns of use, a follow-up survey is currently underway. The planned 1997 survey will target the entire School of Education rather than just the initial population. Since the only change to the survey instrument will be in the demographics section, we will compare responses over a two year period to see how the reasons for use, attitudes, and ratings of usefulness of aids and supports have changed in the interim. The information obtained from the 1997 survey will help the Technology and Special Services (TSS) Division, which now houses the ILT program, to plan for improved use of e-mail and the WWW.

To fully compare the data from both surveys, it is necessary to re-analyze the 1995 survey data using SPSS rather than simply looking at the descriptive statistics that were obtained from the Excel spreadsheet. This paper presents a more detailed data analysis of the original 1995 survey that will set the stage for a planned comparison of both the 1995 and 1997 surveys later this year.

Design of the Study

• To what extent is telecommunications used by community members?
• For what reasons is telecommunications used by community members?
• What challenges to the use of telecommunications are perceived as most important?
• What aids and supports are perceived as most useful by community members?

The results of the analysis of these data would then guide the Internet Task Force regarding interventions, aids and supports - what to continue to develop, what to drop, and what to recommend to the School of Education regarding new courses, workshops, and assistance by GAs. Since that time, the university's Computer Information and Network Services (CINS) has also become interested in the results of this study as part of a needs assessment to determine what types of support new and continuing users of the university networks tend to prefer.

The design of the study is similar to the used by Pelton and Pelton (1996) in exploring the attitudes of preservice teachers regarding technology. Data were collected on student and faculty demographics and patterns of Internet use, reasons for use, attitudes, and perceived usefulness of the aids and supports that the Internet Task Force and the ILT faculty were considering implementing.

The questionnaire consists of four sections:

Section 1.	6 demographic items to obtain a profile of participants' status in the division, access to technology, and use of e-mail;
Section 2.	14 Likert scale items assessing participants reasons for using various Internet resources;
Section 3.	11 Likert scale items assessing participants perceptions of obstacles they face in using e-mail and the WWW ; and
Section 4.	8 aids and supports proposed by the Internet Task Force, to be rank-ordered in order of perceived importance.

The questionnaires were administered at the beginning or end of class, and were filled out by both the students and the professor. Completed questionnaires were returned to the Internet Task Force's faculty advisor in sealed envelopes. They were then coded and analyzed by the author, using an Excel spreadsheet. In 1997, the Excel file was copied into SPSS and analyzed.

Measurement Strategies

Based on the literature review, Ryder and Wilson conducted the focus group interview with 13 students who were enrolled in the spring 1995 IT doctoral seminar. These students had already spent nine weeks exploring the Internet, developing home pages, conducting searches, downloading files, and participating in LISTSERV discussion groups. The interview was an important source of information about students' motives, attitudes, and perceptions of the Internet as a learning tool. Ryder organized group interview comments around several key factors that affect students' choices to use or to avoid e-mail and the Internet, namely, clear payoff and value, technophobia, cultural/personal compatibility, proper scaffolding, and finding a voice and having something to say.

Based on the variety of responses, we fully expected that we would find students and faculty scattered throughout many stages of the adoption-decision process (See Rogers, 1995). Thus, we spent several sessions brainstorming a set of questions to get at the information we needed, namely, to identify our learning community's primary objectives for using e-mail and the Internet, the barriers and challenges that stood in their way, and the types of aids and supports that would prove most useful for the majority of our population.

Starting with insights gained from the literature review and his past experience, one member of our task force (Teslow) generated about ten pages of potential items. Each of the six members of the team evaluated the questions, and deleted items that were ambiguous, repetitious, or not closely related with the objectives of the survey. With direction from our faculty advisor, I then constructed a test instrument consisting of items on which we had reached consensus, and pilot tested the questionnaire on the group. After a second round of revisions and further cuts, our team finally agreed that the instrument was ready to be distributed.

Validity - especially content and construct validity - of the items in sections 2 and 3 was an important issue that was discussed at length among members of the Internet Task Force. Goodwin (in press) states that obtaining construct validity evidence involves delineating the meaning of the construct in terms of a theoretical framework, developing hypotheses from this same theoretical basis, and testing the hypotheses empirically (p. 9). Since this was an exploratory study, there were no hypotheses per se. However, the results of the questionnaire cannot be considered in isolation from Ryder and Wilson's literature review, their results from the initial focus group interview with the thirteen doctoral students and the follow-up e-mail interview with the seven students who responded, and McCahan's results from the three faculty interviews.

Ryder and Wilson's (1995) analysis of the group interview responses revealed that there were five key factors that affect students' choices to use or avoid e-mail and the Internet. They named these five factors as follows:

• Clear payoff and value: some compelling need for students to engage in the discomfort attending the learning of new technologies;
• Overcoming technophobia and technophatigue: a belief in the value or necessity of learning and using a new technology, coupled with feelings of incompetence or inability to learning the new technology;
• Cultural/personal compatibility: especially when technology conflicts with people's learning styles, self-concepts, and lifestyles;
• Proper scaffolding: a support structure that includes a non-judgmental, social support system, lots of hand-holding, good access, and removal of technical hurdles; and
• Finding a voice and having something to say: not just knowing how and where to get help, but knowing what content to put into messages and home pages, and how students want them to look.

Overall, student and faculty comments were consistent with the survey responses (Wilson, Ryder, McCahan, & Sherry, 1996). Cultural/personal compatibility was explored with qualitative methods, and is not included in the 1995 questionnaire. The type of scaffolding that was considered useful by the respondents to the questionnaire was dealt with in Section 4, where participants were asked to rank order a set of eight proposed supports for training and performance using e-mail and the Web.

In a similar study using factor analysis with Varimax rotation, Pelton and Pelton (1997) expected four factors to emerge from their 42 items measuring preservice teachers' attitudes toward technology: confidence, value, isolation, and external factors. Their first factor, confidence, accounted for 18.6% of the variance in the set of items. Their factors 2 through 8, which were retained, accounted for 12.0% down to 6.9% of the variance. Their second through fourth factors were usefulness in education, personal usefulness, and value. The rest of their factors differed from those in the UCD survey because the other items were measuring different constructs. Their first factor, confidence, is closely related to Ryder and Wilson's second factor, overcoming technophobia and technophatigue; and their next three factors are related to Ryder and Wilson's first factor, clear payoff. Pelton and Pelton did not consider the remaining factor, finding a voice and having something to say, when designing their instrument. Based on these two empirical studies, we expected at least two, and at most five, factors to emerge from the analysis.

One way to estimate content validity is to have an instrument evaluated and revised by a team of experts who can judge the extent to which the sample of items represents the defined domain. All of the Internet Task Force members were fairly expert in using e-mail and the Internet, had access from home or work, and were familiar with the CINS computer systems. Because of the diverse backgrounds of the team members, they brought multiple perspectives and widely differing viewpoints to the discussion.

Data Analysis Strategies

In 1995, we had not done a formal reliability analysis on the instrument. Internal consistency, especially of the affective measures, was critical if we were to combine items into subscales later on. This became the first step in the re-analysis of the data. The coefficients alpha for the 14 reasons for use in Section 2 and the 11 attitude measures in Section 3 are presented in Tables 1 and 2. N=72 because one of the respondents had missing data.

Since the generally accepted rule of thumb for factor analysis requires that there be ten times as many subjects as there are items in the scale (here, 110 subjects for 11 items on the challenge scale), but there were only 73 respondents to the survey, there was some violation of this guideline. Interestingly, Pelton and Pelton (1997) used N=60 for 42 Likert scale items assessing attitudes toward technology, which was a much more flagrant violation of the rule of thumb than mine. In the 1997 survey, we will remedy that by surveying all ILT/IT classes and a stratified sampling of the School of Education classes.

I was also curious whether there was any relationship between participants' attitude scales and the aids and supports that the Internet Task Force had either created or was planning to develop in the foreseeable future. For example, would on-line performance support be considered valuable by participants in general, or only by a small group of participants who had already attained a relatively high comfort level with e-mail? Tying the attitude scale together with the rating of relative usefulness of aids and supports was a task that required a correlational rather than a factor analysis. Since the supports for training and performance using e-mail and the Web were ranked by some respondents and rated by other respondents, they had to be treated as ordinal data. Hence, a Pearson correlation between the emerging challenge factors and the rankings of the eight performance supports was not applicable.

Under Crosstabs, SPSS produces four measures of correlation between two sets of ranked or ordinal data. The Spearman correlation uses the difference in ranks of each pair of scores and the number of pairs of scores to calculate the degree of correlation between two columns of data. The program also calculates the significance at the .05 level for the degree of correlation that it produces. KendallŐs tau-beta, Kendall's tau-c, and Goodman and Kruskal's gamma are also measures of association based on a comparison of the values of both variables for all possible pairs of cases. (See SPSS 6.1 Base System User's Guide Part 2, pp. 118-119.) Since the values of each of these correlations are fairly close to one another, only the Spearman correlation will be reported here.

Results

Section 1. Demographics

• What is your current level of Internet use?
• Do you have access to the appropriate technology?

These were answered by totaling the responses to the six demographic questions. The sample of 73 participants was gathered from most of the ILT/IT classes, with all ILT/IT professors administering the questionnaire to at least one class. The questionnaire was administered at the beginning of the spring semester, 1995. There were 66 students and 7 faculty members. 61 had e-mail addresses; 12 did not (2 used to have accounts; one expired and one was canceled). 63 used e-mail; the remaining 10 did not. 35 learned to use e-mail on their own; 29 learned in various classes, both in the UCD School of Education and at other colleges; the rest learned from friends or colleagues, at work, or at school. 60 have computers and modems at home; 13 do not.

Respondents used e-mail primarily from home and work, and less in the UCD computer laboratories. When asked where and how often users accessed their e-mail accounts, the responses were as shown as percentages in Table 3:

Section 2. Reasons for use

• How would you like to use the Internet?
• What challenges do you face?

The factor analysis for Section 2, reasons for Internet use, is shown below. The variables USEFUL_A through USEFUL_N represent the fourteen reasons for use found in Section 2 of the questionnaire.

FACTOR ANALYSIS
Analysis number 1 Listwise deletion of cases with missing values
Extraction 1 for analysis 1, Principal Components Analysis (PC)
Initial Statistics:

PC Extracted 4 factors.
VARIMAX rotation 1 for extraction 1 in analysis 1 - Kaiser Normalization.
VARIMAX converged in 7 iterations.
Rotated Factor Matrix:

• Factor 1: share and disseminate information (use e-mail, transfer files, communicate, disseminate information). This factor accounts for 41.5% of the variance.
• Factor 2: find and organize information (literature searches, instructional materials, electronic publications). This factor accounts for 11.7% of the variance.
• Factor 3: collaborate (observe how schools are using the Internet, collaborate with scholars from other universities). This factor accounts for 8.7% of the variance.

Factor 1 combines local access and connectivity together with information dissemination, whereas Factor 2 and Factor 3 represent finding information and collaborating. Thus, though I originally suspected that there would be four factors, in the final analysis there were only three that were important. (Note: USEFUL_E stands for "consult with your advisor"; this was the only item that loaded above 0.5 on Factor 4.)

Factor 4 was close enough to the cutoff point that it could be ignored for the purpose of this analysis. It could potentially be dropped, especially since staff members and faculty do not have advisors. When an item analysis was run on the fourteen items under reasons for use, the item analysis showed three things:

• The corrected item-total correlation is low. This means that USEFUL_E is not measuring what the rest of the scale is measuring.
• The squared multiple correlation is low. Item E doesn't predict much of the variability.
• If the item is deleted, the internal consistency of the measure (Alpha) goes up, but the change is negligible (Alpha=.8799 is raised to .8846 if USEFUL_E is deleted).

Section 3. Challenges to Use.

FACTOR ANALYSIS
Analysis number 1 Listwise deletion of cases with missing values
Extraction 1 for analysis 1, Principal Components Analysis (PC)
Initial Statistics:

PC Extracted 3 factors.
VARIMAX rotation 1 for extraction 1 in analysis 1 - Kaiser Normalization.
VARIMAX converged in 7 iterations.
Rotated Factor Matrix:

• Factor 1: clear payoff and value (communicate, worth the time, want more use in class, pays off professionally). This factor accounts for 32.5% of the variance
• Factor 2: technophobia and technophatigue (too much jargon, I need hand-holding, intimidation by techno-gurus in the program). This factor accounts for 17.2% of the variance.
• Factor 3: finding a voice and having something to say (I have a hard time expressing my thoughts in writing). This factor accounts for 8.7% of the variance.

These three factors corroborated Ryder and Wilson's groupings and matched three of their five factors: clear payoff, technophobia and technophatigue, cultural/personal compatibility, proper scaffolding, and finding a voice and having something to say. Moveover, we can strip the negative/positive polarization from Factors 1 and 2; rename Factor 1 as "perceived value" and Factor 2 as "self efficacy", thereby linking these empirically derived factors with two important constructs found in Bandura's theory of self-efficacy as a mediator of performance and achievement (Bandura, 1982; Driscoll, 1994). Factor 3 has been identified by Fishman (1997) and Berge (1997), and may be renamed "written communication proficiency".

Section 4. Relative Usefulness of Aids and Supports

Significant Spearman Correlations (p<.05) were found for the following pairs:

Recall that the items were polarized so that the higher the value on the Likert Scale, the more positive was the respondent's attitude regarding each item. Thus, we can interpret the above results as follows:

• As self efficacy increases, formal classes are considered more useful.
• As self-efficacy increases, booklets describing hardware, software, and online resources are considered less useful.
• As written communication proficiency increases, online tutorials specific to the university network are considered less useful.
• As self-efficacy increases, individual attention by graduate assistants is considered more useful.

Discussion and Critique

To reiterate, the significant correlations between usefulness of aids and supports and the empirical attitude factors that emerged from this re-analysis of the 1995 data look very different from the results that were obtained from the superficial analysis in 1995. The original analysis indicated that support efforts should be aimed at workshops and classes with guided, hands-on practice, demonstrations, and individual mentoring by Internet Task Force members.

This new analysis does not show the emphasis on workshops that appeared in the initial analysis, but it does indicate that booklets and online tutorials may not have significant usefulness for faculty and students within the IT emphasis areas. Moreover, duplication of CINS's efforts to produce paper-based job aids would be an inefficient use of the limited time and resources of the Internet Task Force. The rapid evolution of the Internet and the quickly changing hardware/software resources offered by the university also means that paper-based job aids could become obsolete very rapidly.

Formal classes and individual attention by graduate students (either within class or as a follow-up to class instruction) continue to be considered valuable by the members of our community.

Clearly, the results of the survey have never, and will never, stand alone as the sole means of testing the attitudes and reasons for Internet use of our learning community. After the 1997 questionnaire results are coded and analyzed, they will be disseminated both on-line and in the TSS Newsletter. Follow-up interviews and focus groups are planned for next year.

Meanwhile, it is clear that the results of our research are in consonance with those of other researchers. Pelton and Pelton (1997) suggest that strategies to enhance new user experience with computers, either in introductory computer classes or content specific courses, contribute to the formation of positive attitudes and self efficacy. This suggestion can easily be applied to introductory telecommunications and web authoring courses, and to any School of Education courses in which on-line conferencing and e-mail messaging are an integral part of instructional activities.

As with any research study, there are always threats to both internal and external validity. We are aware that the unrepresentative sample compromises the generalizability of our findings. The IT division is small compared with some of the other divisions in the School of Education and contains a fair number of high-end computer users. The 1997 survey will encompass the entire School of Education. Nongeneralizability of the dependent variable(s) is another threat, since two affective measurement scales might give very different results.

Another problem that arose when the data were being analyzed was the presence of some ambiguities in the questions themselves, resulting in bad data. Some respondents were confused as to whether 1 represented "high" or "low" importance when ranking the relative importance of aids and supports. The 1997 survey will address any language ambiguities. It will also address gender differences - a variable that was not considered in the 1995 survey. Some respondents who did not have advisors (which would also include the faculty and staff members who participated in the survey) did not circle a choice on item E ("consult with your advisor") in Section 2. It was suggested that USEFUL_E ("consult with your advisor") be dropped from the 1997 survey instrument. At present, the item is retained; the final analysis will determine whether or not it should be dropped.

The best place to resume this ongoing study, once the 1997 data have been analyzed, will be to do a more extensive literature review, including some of the recent empirical studies that are now appearing on-line, at professional conferences, and in professional journals. Since the emphasis of current studies is changing from technology-based considerations to the use of the Internet to support teaching and learning by communities of learners, it is important to keep abreast of the most recent research efforts, as well as to remain in touch with prominent researchers in the field, via e-mail and LISTSERVs.

Another important area for further research is to survey non-users as well as users to develop a more systematic understanding of the beliefs about the Internet's value and the hurdles faced by non-users and those who might wish to become users, as distinct from novice and current users (cf. Katz & Aspden, 1997). A third area of research, which extends well beyond the limitations of a survey, is to explore the adoption of the Internet as a systemic change effort within the School of Education, and to examine both the social and cultural implications of the rapidly increasing use of educational telecommunications within our learning community.

References

Berge, Z. (1997). Computer conferencing and the on-line classroom. International Journal of Educational Telecommunications, 3 (1), 3-21.

Driscoll, M. (1994). Psychology of learning for instruction. Boston: Allyn and Bacon.

Fishman, B.J. (1997). Student traits and the use of computer-mediated communication tools: What matters and why? Chicago, Illinois: Paper presented at the Annual Meeting of the American Educational Research Association.

Goodwin, L.D. (1997, in press). Changing conceptions of measurement validity. To appear in the Journal of Nursing Education.

Katz, J., & Aspden, P. (1997). Motives, hurdles, and dropouts: Who is on and off the Internet, and why. Communications of the ACM, 40 (4), 97-102.

Pelton, L., & Pelton, T.W. (1996). Building Attitudes: How a technology course affects preservice teachers' attitudes about technology. [On-line.] Available: http://www.coe.uh.edu/insite/elec_put/html1996/04math.htm#pelton

Rogers, E.M. (1995). Diffusion of Innovations, Fourth Edition. NY: The Free Press.

Ryder, M., & Wilson, B.G. (1995). From local to virtual learning environments: making the connection. [On-line.] Available: http://www.cudenver.edu/~mryder/aera95.html

Sherry, L., & Morse, R. (1995). An assessment of training needs in the use of distance education for instruction. International Journal of Educational Telecommunications, 1(1), 5-22.

Sherry, L. (1996, October). Supporting a networked community of learners. Tech Trends, 28-32.

Sherry, L. (1995). TSS E-Mail Survey Results. Technology and Special Services Newsletter, [On-line]. Available: http://www.cudenver.edu/public/education/sherry/pubs/survey_results.html [November 13, 1997].

Wilson, B.G., Hamilton, R., Teslow, J.L., & Cyr, T.A. (1995). Technology making a difference: The Peakview Elementary Study. Syracuse, NY: ERIC Clearinghouse on Information and Technology.

Wilson, B.G., Ryder, M., McCahan, J., & Sherry, L. (1996). Cultural Assimilation of the Internet: A Case Study. Indianapolis, IN: Paper presented at the AECT In-CITE '96. Also available [On-line] at: http://www.cudenver.edu/~bwilson/cultass.html

Appendix A