Does educational
software make a difference? A summary
of: Campuzano, L., Dynarski, M., Agodini, R., and Rall, K. (2009). Effectiveness of Reading and Mathematics Software
Products: Findings From Two Student Cohorts—Executive Summary (NCEE 2009-4042). Washington, DC: National Center for
Education Evaluation and Regional Assistance, Institute of Education Sciences,
U.S. Department of Education.
S. Krashen, May, 2014
Hat-tip: Susan Ohanian
Hat-tip: Susan Ohanian
Campuzano et. al.
investigated whether software for reading and math had any effect of achievement,
as measured by standardized tests. The reading software was used in grades 1
and 4, and the math software was used in grade 6 (pre-algebra) and for algebra
(mostly grade 9).
The study lasted one
year, and was replicated the second year.
The first year, 16 softward products were tested in 33 districts, 132
schools, and with 428 teachers in either classes that used the softeware or comparison
classes that did not.. The second year included 10 products, 23 districts, 77
schools, 176 teachers, and 3,280 students. A variety of well-known standardized
tests were used. For reading, the SAT-9 was used in grade 1 and the SAT-10 in
grade 4 for reading and grade 6 for math.
The study used the Educational Testing Services’ End-of-Course Algebra Assessment for grade 9
(algebra I). Other tests were also used to confirm that the groups had similar
levels of competence in reading and math beore the treatment began.
With the exception of scores
on the ETS algebra test, scores were converted to normal curve equivalent (NCE)
units. Algebra I scores for the ETS test are reported as percent correct.
The results
NO IMPACT OF USING SOFTWARE
There was no significant
difference between test scores of students who used the software and those who
did not at the end of the first year.
First year results
READING
|
software
|
comparison
|
grade
1
|
50.2
|
49.5
|
grade
4
|
42.1
|
41.7
|
MATH
|
||
grade
6
|
52.2
|
50.8
|
grade
9
|
37.3
|
38.1
|
Results presented in
normal curve equivalents, except for grade 9, in percentiles.
NO CONSISTENT EFFECT OF
EXPERIENCE WITH SOFTWARE
Some teachers (n = 115) taught both years with the
software. For reading, these teachers' scores
were not significantly higher the second year. For math, the 6th
grade scores got significantly worse, and the 9th grade scores got
significantly better.
Difference between first
and second year scores for teachers who used software for both years
READING
|
year
1
|
year
2
|
grade
1
|
0.86
|
-1.28
|
grade
4
|
2.65
|
4.67
|
MATH
|
||
grade
6
|
-0.44
|
-1.24
*
|
grade
9
|
-0.34
|
2.58
*
|
* statistically significant
difference
LITTE DIFFERENCE AMONG
THE PROGRAMS
A third analysis was the
impact of ten of the software programs used in either the first or second year.
Only one difference out of ten was statistically significant and in this case
software group's test score was only two NCE's (about three percentiles) higher
than the comparison group.
Impact of reading programs
grade
1
|
|
Destination
Reading
|
1.91
|
Headsprout
Early Reading
|
0.29
|
PLATO
|
0.5
|
Waterford
|
0.42
|
grade
4
|
|
Academy
of Reading
|
-0.16
|
Leap
Track
|
1.97
*
|
· statistically significant difference
·
Impact of math programs
|
|
grade
6
|
|
PLATO/Achieve
now
|
-0.58
|
Larson
Pre-Algebra
|
2.37
|
grade
9
|
|
Larson
Algebra I
|
-0.1
|
Cognitive
Tutor
|
-1.28
|
CONCLUSION
Out of 18 comparisons, 15
showed no difference, two showed a significant but small effect in favor of the
software, and one showed a small negative effect. We would expect one comparison out of 20 to
be significant because of chance.
The results are consistent
with the conclusion that these educational software programs did not enhance
learning. The results also indicate that experience with the programs did not
improve their effectiveness. Nine of the
ten programs investigated had no positive effect. The only one that was
significantly effective did only slightly better than the comparison group.
It would be interesting to see descriptions and examples of the software and how it differs from the comparison group materials. Anecdotally, when I taught middle-school mathematics, the available software was just a different format for the same material, and it had the expected disadvantage of not relating to individual students affectively, something that was important to students and often made the difference between success and failure. Of course, that was several years ago, and one would expect there to be improvements. But I don't know how they could make up for the relationship between the student and the teacher.
ReplyDelete"But I don't know how they could make up for the relationship between the student and the teacher"
ReplyDeleteExactly that's the thing that always drove me crazy when computers and software were being everywhere touted as being great for education because they are "interactive." What could be more interactive than an actual person.
Institute Manager is one of the best Institute Management Solutionsavailable for Educational Institutions. Contact us to get more information on our world class education software today! We provide complete Institute Management solutions for schools and colleges.
ReplyDeleteI needs to spend some time learning more or understanding more. Thanks for excellent information I was looking for this information.
ReplyDeleteThin Clients & Zero Client
Thanks for sharing...
ReplyDeleteRegards
Remark
software companies in delhi providing end to end software solutions. As a software development company based out of India,
ReplyDeleteThe collage paper editing services are sometimes necessary for students and i wish them all the best.
ReplyDeleteNice article i was reached for a good tips.thanks for sharing your blog.
ReplyDeleteeducational software companies
What a wonderful post you wrote! This will help many beginners like me! It's very clear, precise and descriptive really! Thank you very much, author. Keep writing for us like this :)
ReplyDeleteDedicatedHosting4u.com