Integrated Technology 1
Filetype:
pdf
Filesize: 66701
Integrated Technology
1
Running Head: Implementing Integrated Technology in Science
The Effects of Differentiated Technology Integration on Student Achievement in Middle
School Science Classrooms
Pamela Cantrell, Ph.D.
Raggio Research Center for STEM Education
University of Nevada, Reno
Mailstop 432
Office: 775-327-5281
Email:
pamcan@unr.edu
Paper presented at ASTE 2006 Annual International Meeting, Portland, OR January 12-14
Integrated Technology
2
Abstract
Thirty-nine middle school science teachers participated in a professional
development program designed to increase their skills in technology integration.
Teachers were also facilitated in conducting teaching-as-research in order to generate
rigorous evidence on the effects of integrated technology on student learning. Science
units and assessments were developed by groups of teachers and were subsequently
taught in their classrooms. The technology integration component for each unit was rated
for level of technology integration using a rubric with four levels. Comparisons of
student performance on units with varying levels of technology integration were
conducted. Students participating in units with the highest level of interactive technology
integration scored significantly higher than their peers. Data were further analyzed for
effects using disaggregated demographic variables of gender, special education, social-
economic level, and ethnicity.
Integrated Technology
3
The Effects of Differentiated Technology Integration on Student Achievement in Middle
School Science Classrooms
The Rural Science Teachers Teaching with Technology (RST3) project was
designed as a model for professional development aimed at increasing middle school
teachers’ pedagogical content knowledge of technology integration in science instruction.
The RST3 model is based on national standards for science (NRC, 1996) and educational
technology (ISTE, 2000) that call for the integration of technology into science and all
content areas, and on the national standards for professional development (NSDC, 2003)
that call for systemic, research-based programs.
The federal No Child Left Behind Act of 2001 (NCLB, 2002) calls for the use of
more rigorous scientific research that provides strong evidence as the foundation for
educational programs and interventions. Strong evidence is defined by the U. S.
Department of Education (2003) as randomized controlled trials in two or more typical
school settings. The goal of this program was first to facilitate a professional teaching and
learning community (TLC) that would foster growth in teachers’ motivation to increase
their knowledge and skills for integrating technology into their science classrooms and
then to produce scientifically-rigorous evidence showing the effects of integrated
technology on student learning as one measure of the effectiveness of the model. The
research component for this project encompassed a variety of areas including both
teacher and student outcomes. However, this paper will only briefly describe the larger
RST3 program and the professional TLC model with the major focus being on the results
of differentiated technology integration on student achievement scores. In terms of
Integrated Technology
4
student effects, we sought to answer the following two questions based on mean student
achievement scores on teacher-generated assessments that were standardized across
school settings:
1.
Does the level of integrated technology affect student achievement scores?
2.
Are there differences in effects of the levels of technology integration on
student scores when data are disaggregated by gender, IEP, SES and
ethnicity?
Theoretical Framework
Knowing that student attitudes toward science over the middle and high school
years show a general decline that is more significant for rural students (George, 2000),
we hoped to facilitate rural teachers in presenting science content using integrated
technology as an alternative method that could engage students in more rigorous and
compelling learning. Instruction in classrooms where technology is used infrequently
tends to be more teacher-centered (Waxman & Huang, 1996) while technology-rich
instruction fosters more student-centered activities and more time on task (Waxman &
Huang, 1996; Worthen et al., 1994). In a meta-analysis of 20 studies that examined the
effects of technology use on students’ cognitive, affective, and behavioral learning
outcomes, Waxman and his colleagues (Waxman et al., 2002) found a modest positive
effect. In addition, the findings showed no significant differences across contextual
categories, thus allowing for generalization across a wide variety of conditions and
student, school, and study characteristics. Several studies have also found positive
effects for technology use on standardized test scores (Bain & Ross, 1999; Mann et al.,
1999; Wenglinsky, 1998). The use of technology in science education has been widely
Integrated Technology
5
studied and reported in the literature (Bell & Bell, 2003), and guidelines for
implementing technology in science classrooms have been proposed (Flick & Bell,
2000).
A decade ago it was recognized that the role of computer-based learning
technologies in the classroom shifted from the perspective of “the computer as an agent
of change” in the 1980s when it was supposed that computers were expected to have a
major and direct impact on student learning and skill acquisition, to the perspective of
“the computer as a tool” by the 1990s (Bracewell & Laferriere, 1996). Using the
computer as a tool carries implications for effective use of technology embedded within
appropriate practices and activities in the classroom and raises questions about how best
to facilitate teachers in the acquisition of appropriate pedagogical content knowledge for
integrating technology in science classrooms. Professional development making use of
situated learning theory may be one way to increase teachers’ understanding of
pedagogical content knowledge for integrating technology into science classrooms.
The RST3 professional teaching and learning community model is grounded in
situated learning theory that has its roots in Vygotsky’s sociocultural theory (Reiber &
Carton, 1987). Situated learning suggests that learning is influenced by social and cultural
interactions as opposed to a traditional view in which the learner is expected to acquire
and internalize isolated facts and information individually. To situate learning involves
placing learners within a set of conditions in which they live the subject matter in the
context of real-world (authentic) challenges, thus acquiring knowledge that transfers from
the learning environment to the realm of practice. Lave and Wenger (1991) suggest that
situated learning takes place within a participation framework, not in an individual mind,
Integrated Technology
6
and necessarily involves other learners, the environment, and complex ambiguity that
must be dealt with. Learners start as novice participants and progress to expert level by
gaining skill and mastery. Gradually, they become full participants in the sociocultural
practices of their community (Orange, 2002). Brown, Collins and Duguid (1989)
describe situated learning as cognitive apprenticeship—an enculturation supported
through social interaction and the circulation of narrative that includes the following four
features: 1) collective problem solving, 2) displaying multiple roles, 3) confronting
ineffective strategies and misconceptions, and 4) providing collaborative work skills.
Situated learning theory suggests that learning must take place in authentic settings
involving authentic activities and applications, and that learning is a social enterprise, all
of which were incorporated into the RST3 model.
The RST3 Model
A 3-credit graduate course was developed to include the tenets of situated
learning theory to help increase teachers’ pedagogical content knowledge for technology
integration in science. This course served as the basis for the professional development
model. Professors and graduate students with expertise in science content, educational
technology, pedagogy, and assessment facilitated the course sessions. Each teacher
participant was given a technology package that included a laptop computer, an LCD
projector, an electronic microscope, a flash drive, and a one-year high speed Internet
connection at home. Five week-end class sessions took place over the course of a year
with instruction and facilitation provided for appropriate pedagogical practice for
technology integration, inquiry science teaching, and assessment. The technology
package provided the means by which teachers brought computers to every class session,
Integrated Technology
7
which helped situate the learning within an authentic context of technology integration.
Teachers participated within small groups that fostered social interaction and dialogue
and were provided with rigorous and complex tasks requiring the use of their computers
and the Internet, thus advancing their understanding as they negotiated new meanings
within each new situation. Such activities provided a model for what they were expected
to then implement in their own classroom.
The RST3 professional development course included four strands: technology
integration strategies, unit content development, assessment development, and teaching-
as-research. Each of these strands will be discussed briefly below.
Technology Integration Strategies
A significant block of time during the professional development component was
spent on demonstrating and discussing the various levels of technology integration
ranging from what we defined as low-level such as using the Web as a source of reading
material or using computers for word processing, to high-level such as interactive
software or Web sites with dynamic content that required students to enter values for
variables that resulted in changing outcomes that could serve as a data source for later
analysis. Teachers were provided with Web sites and software showing examples of
interactive technology with varying degrees of interactivity that they explored and
evaluated. They were provided with a methods textbook that generated many
discussions. During each session teachers would share new integration ideas and
resources they had located during the time between sessions.
Integrated Technology
8
Teaching-as-Research
Teaching-as-research is the systematic use of research methods to develop and
implement teaching practices that advance the learning experiences and learning
outcomes of students and teachers (Mathieu, 2000). It involves learning foundational
knowledge, then creating goals for better student learning, defining measures of success,
developing and implementing best practice methods, collecting and analyzing data, and
making data-based teaching decisions in harmony with selected goals.
A critical feature of the RST3 model was to work together with teachers in some
of the research activities as a model for how they might, on an ongoing basis, examine
their own practice and determine the effectiveness of any new classroom interventions or
teaching strategies they might implement by applying data collection and analysis
strategies. The essence of situated learning is when experts and novices work together
for a common product or goal within an authentic context. This shared productive activity
is also identified as one of the five standards for effective pedagogy and student outcomes
developed by the Center for Research on Education, Diversity & Excellence (Tharp et al.,
2000). We included the teachers in several decision-making conversations relative to our
research direction for this project. We also provided them with opportunities to apply
research techniques in their own practice and bring results back to share with the group.
Unit Development
Unit development was accomplished by teams of teachers from at least three
schools in order to maximize the number of school settings across which data on the
same unit could be collected. Topics for the units were selected by the teachers from
their regular curriculum. Content for each lesson was reviewed by teacher peer groups
Integrated Technology
9
and university educators for rigor, accuracy, and alignment with state and district science
standards.
An integrated technology component was designed for each unit that could be
included or excluded with minimal impact on the science content. The goal was for the
technology component to function only as a tool for teaching and learning the science
content and not for the technology to be the content. Technology components were
designed in this way so that they could be used or not, depending upon the random
assignment of a class to the control or experimental group.
Assessment Development
Teachers were provided with assessment training including test item development
and test blueprint design that would be used for the end-of-unit test. The test blueprints
specified that each numbered item was to be of a specific type drawn from the cognitive
and science content domains. The levels for the cognitive domain items were patterned
after the specifications for the Trends in International Math and Science Study
assessment (TIMSS, 2003) and included items for factual recall, conceptual
understanding, and analysis/synthesis. In addition, three types of questions were
developed—true/false, short answer, multiple choice, and a final scenario question
requiring problem solving skills including analysis and synthesis of science concepts.
Teachers then used this test blueprint to develop end-of-unit tests for every unit, so while
the content of each test varied by unit, the format for the all tests was standardized by
using the blueprint. For example, item #7 on the blueprint was to be a multiple choice
item (type) that tested conceptual understanding (cognitive domain level) of a given
concept from the content domain.
Integrated Technology 10
Method
Participants
Middle school science teachers were recommended by their respective school
district administrators for participation in the RST3 program. Thirty-nine middle school
science teachers and their students from six rural Nevada school districts participated in
the RST3 project. A total of 5043 students in the treatment group were disaggregated by
the demographic variables of gender (Males, n = 2626; Females, n = 2357), special
education (IEP) (Yes, n = 672; No, n = 4371), socio-economic status as measured by
participation in free or reduced lunch (SES) (Low, n = 1701; High, n = 3156), and
ethnicity (Black, n = 120; Hispanic, n = 822; Asian, n =123; American Indian, n = 167;
White, n = 3809).
Instrumentation
The three teacher-developed assessments included the end-of-unit tests as
described above. The unit test included 19 items worth a total of 25 points. In addition,
an interview protocol of ten questions worth a total of 100 points and covering the same
content as on the unit test was developed for each unit. Teachers also designed a
performance assessment worth 100 points for each unit. A standardized rubric was
developed for the performance assessment by the teachers and the researchers and was
used for scoring purposes.
In order to determine the level of interactive technology integration for the
technology component of each unit, the researchers developed a rubric (Appendix). The
rubric provided a scale ranging from 1-4 for rating the level of technology with 1 being
the least interactive and 4 being the most (Leverington & Cantrell, 2006).
Integrated Technology 11
Design and Procedure
Once unit plans and assessments were completed and taught, the technology
components were rated on the degree of interactive technology using the rubric described
above. Teachers were asked to self-score their units using the rubric, and their scores
were compared to those of the staff assessment specialist who also rated the units.
Discrepant scores were resolved by a third staff member. Of the 22 units developed,
four were rated at Level 4, while seven were rated at Levels 3 and 2 and four were rated
at Level 1.
Teachers taught a total of four units to their students across two semesters. For
the first unit, teachers’ classes were randomly selected for either the control or the
treatment (technology) group. This meant that teachers might be teaching a given unit to
one class as part of the control group and another class as part of the treatment group. In
discussing the possibilities for random selection with the teachers, they felt they could
better control extraneous variables by teaching their units both ways, and we honored
their preference. For unit two, teachers switched the group assignment for each class and
continued to alternate through the completion of all four units. This methodology also
meant that students were in each of the groups for half the units as well.
Teachers administered end-of-unit test to students and submitted the test papers
for all students to the researchers. Each item on every test paper was entered in our data
set for analysis. Teachers randomly selected three students from each of their classes for
one-on-one interviews using the developed protocols. Once the interviews were
completed and scored, teachers sent the scores to the researchers. Teachers administered
and scored the performance assessments and submitted the scores to the researchers.
Integrated Technology 12
To answer the first research question to determine if the level of integrated
technology affected student achievement scores, one-way ANOVAs were used to analyze
assessment scores for the whole student group as the dependent variables and level of
technology integration as the independent variable. Only scores from the treatment group
were used since students in the control group were not provided with instruction using
integrated technology. Factorial ANOVAs were used analyze scores for each of the
assessments to answer the second research question by disaggregating the whole student
group into the demographic variables of gender, IEP, SES, and ethnicity. Level of
technology integration was used as the second independent variable. Because the test
blueprint was used for all unit tests and the data entered accordingly, all factual items and
all conceptual understanding items were totaled into individual scale scores thus resulting
in the four dependent variables from the test: total test score, Factual Items Scale,
Conceptual Items Scale, and the analysis score. The interview score provided the fifth
dependent variable. The performance assessment task was delivered in some classrooms
to individual students while in other classrooms students were permitted to work in
groups that cut across the demographic variables. This situation was not discovered until
the results were submitted by the teachers. Due to obvious threats to validity, this
assessment was excluded from analysis.
Results
A total of 25 analyses were completed to answer both research question and the
level of technology integration was significant in every one. Post hoc tests revealed that
the rubric differentiated four significantly different levels in only four of the 25 analyses
(16%). In the remaining analyses only three significantly different levels were found
Integrated Technology 13
with levels 3 and 4 being statistically the same in six of the analyses (28%) and levels 2
and 3 being statistically the same in 14 of the analyses (56%). We therefore collapsed
levels 2 and 3 into a new level 2, renamed level 4 as level 3, and re-analyzed the data
with three levels of technology integration.
Whole Group Achievement Scores
The level of technology interactivity did make a significant difference in
achievement scores on the unit tests for students in the treatment group (F
(3, 5040)
=
115.25; p <.001). Post hoc tests indicate that scores fell into three significantly different
levels. Descriptive statistics are found in Table 1. The factual items scale (F
(2, 4848)
=
124.81; p <.001) and the conceptual items scale (F
(2, 4772)
= 90.79; p <.001) also showed
significant differences among the three levels of technology interaction. Students who
were taught level 1 units scored significantly lower than students who were taught level 2
units, and these students scored significantly lower than students who were taught level 3
units. Both the analysis scale (F
(2, 4855)
= 19.71; p <.001) and the interview scores (F
(2, 694)
= 50.13; p <.001) showed two significantly different levels with student scores for level
1 units being significantly lower than student scores for level 2 and 3 units in both
instances.
________________________________________________________________________
Insert Table 1 about here.
________________________________________________________________________
Demographic Variables
To examine the effects of the levels of technology integration on gender, IEP,
SES, and ethnicity, factorial analysis of variance was conducted using each of the
Integrated Technology 14
demographic variables and level of technology integration. Because the level of
technology integration was previously shown to be significant across all assessments and
scales, we sought only to examine any resulting significant interactions. No significant
interactions were found with the gender data and further analysis was not conducted on
this variable. The results for the remaining three demographic variables are provided in
the sections below and descriptive statistics relative to significant interactions are found
in Table 2.
Special Education (IEP)
Scores for students with and without individual education plans were analyzed
and two significant interactions were found. Student achievement on the Factual Items
Scale (F
(2, 4845)
= 6.45; p =.002) and on the Conceptual Items Scale (F
(2, 4769)
= 5.49; p
=.002) depended on whether or not students qualified for special education and on which
level of technology integration was used in the units they were taught. Graphs of the
interactions are found below. For the Factual Items Scale (Figure 1), a greater difference
in means occurred between special education and non-special education students on level
2 units than for level 1 and 3 units. Scores for both groups improved at least slightly as
the level of technology increased. For the Conceptual Items Scale (Figure 2), special
education students scored lower on level 2 units than on level 1 units and again, the
difference between their scores and those of non-special education students was greatest
Integrated Technology 15
at level 2. Means for non-special education students increased steadily as the level of
technology integration increased.
Level of Technology Integration
3
2
1
E
s
ti
m
a
t
e
d M
a
r
g
i
n
a
l
M
e
ans
7.0
6.5
6.0
5.5
5.0
4.5
IEP
No
Yes
Figure 1. Interaction of Factual Item Scale and Level of Technology Integration by IEP
Level of Technology Integration
3
2
1
E
s
ti
m
a
te
d M
a
r
g
i
n
al
M
e
a
n
s
7.0
6.5
6.0
5.5
5.0
4.5
IEP
No
Yes
Figure 2. Interaction of Conceptual Item Scale and Level of Technology Integration by
IEP
Socio-economic Status (SES)
Scores for students who qualified for free or reduced lunch and those who did not
qualify were analyzed and one significant interaction was found for the interview scores
(F
(2, 672)
= 4.34; p =.013) as shown in Figure 3 below. Low SES students—those
Integrated Technology 16
qualifying for free or reduced lunch—out-performed high SES students when interviewed
by their teachers on level 1 units, but scored below their peers on interviews for level 2
and 3 units. Both groups show increasing means for interviews as the level of technology
integration increased.
Levels of Technology Integration
3
2
1
E
s
t
i
m
a
ted M
a
r
g
i
nal
M
eans
90
80
70
60
50
SES
High
Low
Figure 3. Interaction of Interview and Level of Technology Integration by SES
Ethnicity
Scores for students from five ethnic groups—Black, Hispanic, Asian/Pacific
Islander, American Indian, and White—were analyzed, and one significant interaction
was found as shown in Figure 4. Student achievement scores for the Factual Items Scale
(F
(8, 4834)
= 2.38; p =.018) depended on their ethnicity and level of technology integration
for the units they were taught. All ethnic groups except American Indian students scored
increasingly higher as the level of technology increased. Asian students scored second
from the bottom on the Factual Item Scale on level 1 units, but scored highest on level 3
units. American Indians out-scored all other groups on the Factual Items Scale on level 1
Integrated Technology 17
units, but fell below all their peers on units at levels 2 and 3. Also, their mean scores for
level 2 units fell below the mean for their level 1 units.
Levels of Technology Integration
3
2
1
E
s
ti
mated Mar
g
i
n
al
Means
7.5
7.0
6.5
6.0
5.5
5.0
4.5
4.0
ETHNICITY
Black
Hispanic
Asian_PI
Am Indian
White
Figure 4. Interaction of Factual Item Scale and Level of Technology Integration by
Ethnicity
Discussion
Student Achievement
Whole
Group
Our overall results seem to indicate that as level of interactive technology
integration increases, student achievement also increases. This makes sense given that
according to the criteria for Level 1 on our rubric, most student time was spent reading
online text documents that did not differ much from their textbooks or other reference
books. On the other hand, Level 3 activities required students to engage with the content
using interactive technology. They used Web-based applets requiring them to enter
values for variables, then could immediately view the effects of a change in values of
given variables on the system. At Level 3, the students also were required to use
technology to collect and analyze data and prepare interactive presentations to
Integrated Technology 18
communicate results to their peers. Higher order thinking skills and problem solving
strategies were required of students with Level 3 units. Engaging students in more
interactive technology seemed to influence their understanding of the science concepts to
a significant degree based on their assessment scores.
Demographic Groups
We were pleased there were no significant gender effects, which indicated that
males and females performed equally well at each of the three technology integration
levels across all assessments. Several researchers have found that attitudes toward
computer use depend upon gender (Arenz & Hiheon, 1990; Chen, 1985) in the past two
decades. At that time, males seemed to exhibit more self-confidence and less anxiety
about mastering computers. A more recent study (Bimber, 2000) indicates that although
the Internet is often considered as “genedered” and that a gap does exist between males
and females in terms of access and attitude, the gap seems to be the product of SES and
other factors and not gender itself. Educational differences between males and females
are shrinking, so it is likely that the technology gap will shrink as well.
The level of technology integration affected the achievement scores for special
education students on the factual and conceptual items of the end-of-unit test. Compared
to their peers, level 2 units were more problematic for special education students. This
may be an artifact of the numbers of students in the level 2 group, however. Because we
collapsed two of the original levels into the new level 2, there were many more students
being taught level 2 units than were taught units at levels 1 and 3. While this
preponderance of students did not seem to affect the achievement scores for not-special
education students, it certainly increased the variability of scores for the special education
Integrated Technology 19
students. In another professional development program we conducted that strengthened
technology integration in middle school science classrooms, special education
performance gaps were narrowed (Cantrell et al., 2005).
Performance gaps in science are well-documented in the literature and continue to
persist. In this study, those gaps narrowed relative to factual items on the unit test as the
level of technology increased. The only exception was in the performance of American
Indian students. This seems to indicate that the more technology is used to engage
students higher order thinking and problem solving skills relative to science content, the
greater their factual recall will be. While it would have been more exciting indeed to
have this same result with the conceptual and analysis items on the test, this is at least a
start. American Indian students performed lower on level 2 units than they did on level 1
units. Once again there were three to four times more students in the level 2 group, and
the variability of their scores was much greater. Even so, their performance scores
remained fairly flat. This situation may be related to the well-documented digital divide
(Levy, 1999). This report indicates that only 76% of American Indians have home
telephones, 27% have access to computers at home, and 19% have access to the Internet.
All these percentages are far below the national averages. This situation underscores the
need for schools to provide as much opportunity as possible for these students to access
and increase their skills in using telecommunication technology while at school.
Professional Teaching and Learning Community
The RST3 teachers came to us with a full spectrum of technology proficiency
ranging from novice to expert. This helped set the stage for a powerful and effective
professional TLC. Early on a system of distributed expertise (Brown et al., 1993) was
Integrated Technology 20
modeled then embraced by all members of the TLC such that at any given moment, any
one of us became the master or the apprentice, the student or the teacher, the novice or
the expert, depending upon the task at hand. With teachers working in curriculum
development groups that spanned schools, new faces became new friends, and a spirit of
camaraderie soon developed. Communications were facilitated online through email and
WebCT platforms.
As the year progressed, we realized that our efforts to acquaint teachers with high-
level interactive technology integration were not resulting in the inclusion of similar
experiences for the technology component in the first units that were developed. We
therefore designed a tool to help address the situation. We distributed a new 4-level
rubric describing the range of technology integration that began with descriptions at
Level 1 that were very similar to the technology components they had designed, and
ending with Level 4 that illustrated the most desirable activities. We asked teachers to
peer review the technology components and to rate them using the rubric. The units
developed after the implementation of the rubric were more closely aligned with our
goals. Teacher comments in interviews, journals and final evaluations indicated a high
level of satisfaction with the experience. Many comments were directly related to the
sense of community they felt, and the network of peers they now feel they can turn to
with questions about technology. There were also many comments about the hard work
involved with this project and the high degree of professional growth they experienced.
We learned several important things from this project. First, in future projects we
will not require teachers to develop and teach four units across one academic year. One
unit would have been sufficient. Also, teacher change is not accomplished overnight, and
Integrated Technology 21
while we knew that from the literature, this was nevertheless an eye-opening experience.
To see the kind of change in teacher behavior brought about by this project required far
more person-hours and emotional engagement than we anticipated. But that aside, this
was also one of the most rewarding experiences we have had. One participant summed
up all our feelings when he said, “I’m so glad this project is over. Now I don’t have to
withhold the technology from half my kids—they can all use it!”
Integrated Technology 22
References
Arenz, B. W., & Hiheon, J. L. (1990). Gender differences in the attitude, interest and
participation of secondary students in computer use., Paper presented at the
annual meeting of the American Educational Research Association. Boston.
Bain, A., & Ross, K. (1999). School reengineering and sat-i performance: A case study.
International Journal of Educational Reform, 9(2), 148-153.
Bell, R., & Bell, L. (2003). A bibliography of articles on instructional technology in
science education. Contemporary Issues in Technology and Teacher Education,
2(4).
Bimber, B. (2000). Measuring the gender gap on the internet. Social Science Quarterly,
81(3).
Bracewell, R., & Laferriere, T. (1996). The contribution of new technologies to learning
and teaching in elementary and secondary schools: SchoolNet.
Brown, A. L., Ash, D., Rutherford, D., Nakagawa, K., Gordon, A., & Campione, J. C.
(1993). Distributed expertise in the classroom. In G. Salomon (Ed.), Distributed
cognitions: Psychological and educational considerations (pp. 188-228). New
York: Cambridge University Press.
Brwn, J. S., Collins, A., & Duguid, P. (1989). Situated learning and the culture of
learning. Educational Researcher, 18(1), 32-42.
Cantrell, P., Peckan, G., Itani, A., & Velasquez-Bryant, N. (2005). Using engineering
design curriculum to close science achievement gaps for middle school students.
Paper presented at the Frontiers in Education, Indiannapolis, IN.
Chen, M. (1985). Gender differences in adolescents' uses of and attitudes toward
computers. In M. L. McLaughlin (Ed.), Communication yearbook 10 (pp. 200-
216). Beverly Hills, CA: Sage.
Flick, L., & Bell, R. (2000). Preparing tomorrow's science teachers to use technology:
Guidelines for science educators. Contemporary Issues in Technology and
Teacher Education, 1(1), 39-60.
George, R. (2000). Measuring change in students' attitudes toward science over time: An
application of latent variable growth modeling. Journal of Science Education and
Technology, 9(3), 213-225.
ISTE. (2000). National educational technology standards for students: Connecting
curriculum and technology. Eugene, OR: International Society for Technology
Education.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.
New York: Cambridge University Press.
Leverington, M., & Cantrell, P. (2006). Developing a rubric that specifies levels of
interactive technology use in the classroom, Paper presented at the meeting of the
American Educational Research Association. San Francisco.
Levy, K. (1999). Fact sheet: Native americans lacking information. Retrieved December
12, 2005, from
http://www.ntia.doc.gov/ntiahome/digitaldivide/factsheets/native-
americans.htm
Integrated Technology 23
Mann, D., Shakeshaft, C., Becker, J., & Kottkamp, R. (1999). West virginia story:
Achievement gains from a statewide comprehensive instructional technology
program. Santa Monica, CA: Milken Exchange on Educational Technology.
Mathieu, R. D. (2000). Teaching-as-research: A concept for change at research
universities. Paper presented at the Research & Teaching: Closing the Divide,
Southampton, UK.
NCLB. (2002). No child left behind act of 2001, pub. L. No. 107-110,115 stat. 1425.
NCLB Act of 2001 Retrieved 9/23, 2003, from
http://www.ed.gov/nclb/
NRC, N. R. C. (1996). National science education standards. Washington, DC: National
Academy Press.
NSDC. (2003). Nsdc standards. Retrieved April 10, 2003, from
www.nsde.org/educatorindex.htm
Orange, C. (2002). Quick reference guide to educational innovations: Practices,
programs, policies, and philosophies. Thousand Oaks, CA: Corwin Press, Inc.
Reiber, R. W., & Carton, E. A. S. (Eds.). (1987). The collected works of l. S. Vygotsky
(vol. 3). New York: Plenum.
Tharp, R. G., Estrada, P., Dalton, S. S., & Yamauchi, L. (2000). Teaching transformed:
Achieving excellence, fairness, inclusion and harmony. Boulder, CO: Westview
Press.
TIMSS. (2003). Timss assessment frameworks and specifications 2003. Retrieved July
23, 2004, from
http://timss.bc.edu/timss2003.html
USDOE. (2003). Identifying and implementing educational practices supported by
rigorous evidence: A user friendly guide. Washington D.C.: Institute of Education
Sciences.
Waxman, H. C., Connell, M. L., & Gray, J. (2002). A quantitative synthesis of recent
research on the effects of teaching and learning with technology on student
outcomes. Naperville, IL: North Central Regional Education Laboratory.
Waxman, H. C., & Huang, S. L. (1996). Classroom instruction differences by level of
technology use in middle school mathematics. Journal of Educational Computing
Research, 14, 147-159.
Wenglinsky, H. (1998). Does it compute? The relationship between educational
technology and student achievement in mathematics. Princeton, NJ: Policy
Information Center, Educational Testing Services.
Worthen, B. R., Van Dusen, L. M., & Sailor, P. J. (1994). A comparative study of the
impact of integrated learning systems on students' time-on-task. International
Journal of Educational Research, 21, 25-37.
Appendix
Rubric for Classifying Levels of RST3 Interactive Technology Components
Think back over the unit you are rating and recall all the different ways that you
integrated technology in the teaching and learning process for this unit as it was delivered
in your classroom. You may have added or deleted technology integration strategies
when you taught the unit regardless of what the authors of the unit intended. Consider
the technology component as a whole when using this rubric and select the level that
Integrated Technology 24
most closely describes the experience for your students. Italicized terms in the rubric are
explained in detail below the table. If all criteria within a level are not met, move to the
level below.
Old New
4 3
At least 50% of the technology integration was interactive AND included
some sort of data collection and analysis using technology. At least 25%
involved students using technology to generate class presentations
(PowerPoint, Excel, digital photography) for communicating findings.
Not more than 25% devoted to online or electronic document-reading.
3
At least 30% of the technology integration was interactive and included
some sort of data collection and analysis. At least 10% involved students
using technology to generate class presentations (PowerPoint, Excel,
digital photography) for communicating findings. Not more than 50%
devoted to online or electronic document-reading.
2
2
At least some of the technology integration was interactive, but little or no
data collection and analysis occurred using technology. At least 10%
involved students using technology to generate class presentations
(PowerPoint, Excel, digital photography) for communicating findings.
More than 50% was devoted to online or electronic document-reading.
1 1
Little or none of the technology integration was interactive. No data
collection or analysis occurred using technology. Students used
traditional methods to generate class presentations for communicating
findings. Most of the technology integration was devoted to online or
electronic document-reading.
Explanation of Terms
Interactive
•
Students or teachers can directly manipulate the data or information presentation
•
Scenarios (i.e., screens, views, sounds, text, etc) change in response to
student/teacher input
•
Offers varying levels of support, such as verbal descriptions, screen captions, etc,
to support on-screen materials
Data collection and analysis
•
Students collected some sort of numerical data that required them to represent the
data in graphs or charts that were generated electronically. Students then had to
make sense of the data in some way—discussions, conclusions, etc.
Document Reading
•
Whether in a textbook or online, document reading involves a printed text that is
read word by word with the assumption that students will “learn.” This category
also includes teacher-generated PowerPoint presentations that include text and
images, animations, colors and cutesy! It includes online research where students
are given a list of URLs to visit (WebQuest-type activity) when most of the time
is spent reading static text or looking at pictures.
Integrated Technology 25
Table 1.
Descriptive Statistics for Assessments
________________________________________________________________________
N
M
SD
_______________________________________________________________________
Test—Total Score (25 points)
Level
1
1159 14.26 5.49
Level
2
2966 16.35 4.88
Level 3
918
17.42
4.56
Test—Factual Items (9 points)
Level 1
1048
5.56
1.89
Level 2
2902
6.44
1.89
Level 3
901
6.79
1.58
Test—Conceptual Items (9 points)
Level 1
954
5.66
1.96
Level 2
2919
5.91
1.84
Level 3
902
6.74
1.78
Test—Analysis Scale (7 points)
Level 1
1034
3.41
2.34
Level 2
2917
3.97
2.52
Level 3
907
3.89
2.24
Interview (100 points)
Level 1
168
34.68
56.49
Level 2
426
19.93
76.25
Level 3
103
15.98
81.39
_______________________________________________________________________
Integrated Technology 26
Table 2.
Descriptive Statistics for Assessments and Level of Technology Integration by
Demographic Variables for Significant Interactions
________________________________________________________________________
N
M
SD
_______________________________________________________________________
Individual Education Plan (IEP)
No
Factual
Items
Scale
Level 1
900
5.66
1.87
Level 2
2507
6.62
1.80
Level 3
802
6.88
1.53
Conceptual
Items
Scale
Level 1
819
5.75
1.97
Level 2
2519
6.07
1.77
Level 3
803
6.84
1.74
Yes
Factual
Items
Scale
Level 1
148
4.97
1.94
Level 2
395
5.32
2.10
Level 3
99
6.07
1.77
Conceptual
Items
Scale
Level 1
135
5.16
1.87
Level 2
400
4.85
1.96
Level 3
99
5.89
1.89
Socio-economic Status (SES)
High
Interview
Level 1
121
54.40
35.48
Level 2
260
79.32
18.32
Level 3
71
82.38
16.29
Low
Interview
Level 1
47
61.86
32.27
Level 2
150
73.07
19.35
Level 3
29
78.07
15.64
Ethnicity
Black
Factual
Items
Scale
Level 1
19
11.95
5.12
Level 2
85
15.85
5.41
Level 3
16
16.84
5.01
Integrated Technology 27
Hispanic
Factual
Items
Scale
Level 1
173
5.18
1.84
Level 2
480
6.25
1.84
Level 3
128
6.59
1.82
Asian/Pacific
Islander
Factual
Items
Scale
Level 1
25
5.00
1.80
Level 2
70
6.49
1.96
Level 3
24
6.88
1.83
American
Indian
Factual
Items
Scale
Level 1
36
5.97
1.43
Level 2
94
5.64
2.02
Level 3
25
6.12
1.20
White
Factual
Items
Scale
Level 1
794
5.67
1.90
Level 2
2174
6.53
1.88
Level 3
708
6.85
1.53
__________________________________________________________________
