Does educational software make a difference?

Does educational software make a difference?  A summary of: Campuzano, L., Dynarski, M., Agodini, R., and Rall, K. (2009). Effectiveness of Reading and Mathematics Software Products: Findings From Two Student Cohorts—Executive Summary (NCEE 2009-4042). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

S. Krashen, May, 2014
Hat-tip: Susan Ohanian

Campuzano et. al. investigated whether software for reading and math had any effect of achievement, as measured by standardized tests. The reading software was used in grades 1 and 4, and the math software was used in grade 6 (pre-algebra) and for algebra (mostly grade 9).

The study lasted one year, and was replicated the second year.  The first year, 16 softward products were tested in 33 districts, 132 schools, and with 428 teachers in either classes that used the softeware or comparison classes that did not.. The second year included 10 products, 23 districts, 77 schools, 176 teachers, and 3,280 students. A variety of well-known standardized tests were used. For reading, the SAT-9 was used in grade 1 and the SAT-10 in grade 4 for reading and grade 6 for math.  The study used the Educational Testing Services’  End-of-Course Algebra Assessment for grade 9 (algebra I). Other tests were also used to confirm that the groups had similar levels of competence in reading and math beore the treatment began.

With the exception of scores on the ETS algebra test, scores were converted to normal curve equivalent (NCE) units. Algebra I scores for the ETS test are reported as percent correct.

The results

NO IMPACT OF USING SOFTWARE

There was no significant difference between test scores of students who used the software and those who did not at the end of the first year.

First year results

READING	software	comparison
grade 1	50.2	49.5
grade 4	42.1	41.7
MATH
grade 6	52.2	50.8
grade 9	37.3	38.1

Results presented in normal curve equivalents, except for grade 9, in percentiles.

NO CONSISTENT EFFECT OF EXPERIENCE WITH SOFTWARE

Some teachers (n = 115) taught both years with the software.  For reading, these teachers' scores were not significantly higher the second year. For math, the 6^th grade scores got significantly worse, and the 9^th grade scores got significantly better.

Difference between first and second year scores for teachers who used software for both years

READING	year 1	year 2
grade 1	0.86	-1.28
grade 4	2.65	4.67
MATH
grade 6	-0.44	-1.24 *
grade 9	-0.34	2.58 *

* statistically significant difference

LITTE DIFFERENCE AMONG THE PROGRAMS

A third analysis was the impact of ten of the software programs used in either the first or second year. Only one difference out of ten was statistically significant and in this case software group's test score was only two NCE's (about three percentiles) higher than the comparison group.

Impact of reading programs

grade 1
Destination Reading	1.91
Headsprout Early Reading	0.29
PLATO	0.5
Waterford	0.42
grade 4
Academy of Reading	-0.16
Leap Track	1.97 *

·      statistically significant difference

·       

Impact of math programs
grade 6
PLATO/Achieve now	-0.58
Larson Pre-Algebra	2.37
grade 9
Larson Algebra I	-0.1
Cognitive Tutor	-1.28

CONCLUSION

Out of 18 comparisons, 15 showed no difference, two showed a significant but small effect in favor of the software, and one showed a small negative effect.  We would expect one comparison out of 20 to be significant because of chance. 

The results are consistent with the conclusion that these educational software programs did not enhance learning. The results also indicate that experience with the programs did not improve their effectiveness.  Nine of the ten programs investigated had no positive effect. The only one that was significantly effective did only slightly better than the comparison group.

Poll of the uninformed

Published in the San Francisco Chronicle, April 28, 2014

"Most in state approve of policy shifts in education," April 24:

Yes, 69 percent of Californians polled said they liked the Common Core standards after hearing a brief statement about them. The statement said only that they "are designed to ensure that students graduating from high school have the knowledge and skills they need to enter college programs or the workforce." I'm surprised anybody would object.

Not mentioned is the fact that the standards call for a uniform but untested curriculum and an astonishing amount of testing. Not mentioned is the fact that the Common Core does not address the largest factor influencing school performance: poverty. When researchers control for the effect of poverty, American students score at the top of the world on international tests.

Eighty percent of those interviewed said they knew either a little or nothing at all about the Common Core. The description they heard, devoid of meaningful detail, made them even less informed.

Stephen Krashen

Incentives for Reading Aloud to Children?

Published in the School Library Journal, 2014.

Introduction by the editors of the School Library Journal:
In School Library Journal‘s January 2014 print issue, Suffolk Cooperative Library System’s youth services coordinator Lisa G. Kropp wrote the article “Vivacious Vocabulary” about how in the interest of building students into critical thinkers and strong readers, libraries are embracing the “1,000 Books Before” program that encourages parents to read 1,000 books to their kids before kindergarten.
One SLJ reader, Stephen Krashen, language expert and professor emeritus at University of California, responds to Kropp’s article in an email:


Read alouds are very pleasant for both parents and children. The vast majority of children say they enjoy being read aloud to: Ninety-seven percent in the study Walker and Kuerbitz (1979), 95 percent in Mason and Blanton (1971), and many parents agree. Eighty-nine percent of mothers interviewed in Newson and Newson (cited in Wells, 1985) said their children liked to be read to.

However, giving certificates, getting one’s picture in the local newspaper, and other incentives could send the message that reading and hearing stories is not pleasant and that nobody would do it without a bribe. This could reduce the amount of reading aloud when the rewards are withdrawn (for research, see Kohn, 1999).

But, I agree with the article’s author Lisa Kropp in that we need to encourage reading aloud. The first step is to make sure that families have books. The second is to make sure that parents, and other caretakers, understand the benefits of read alouds.

Stephen Krashen
 

Kohn, A. (1999). Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A's, and Other Bribes. Boston: Houghton Mifflin.
Mason, G., and W. Blanton. (1971). Story content for beginning reading instruction. Elementary English, 48, 793-796.
Walker, G., and I. Kuerbitz. (1979). Reading to preschoolers as an aid to successful beginning reading. Reading Improvement 16: 149-154.
Wells, G. (1985). Language Development in the Pre-School Years. Cambridge: Cambridge University Press.

Original article: http://www.slj.com/2014/01/standards/early-learning/vivacious-vocabulary-turning-little-ones-into-strong-readers-first-steps/

Address Poverty to boost education

Published in USA Today, May 1, 2014

Address poverty to boost education: Your Say

Original title: Expert Opinion on the Common Core

USA TODAY's editorial does not mention the real opposition to the "Common Core" standards from professional educators and scholars ("'Common Core' demonized as Obamacore: Our view").
Common Core does not deal with the reason for unspectacular performance by American students. The United States has the second-highest level of child poverty among 35 economically advanced countries (now over 23%, compared with Finland's 5.3%). 
[IMPORTANT SENTENCE OMITTED BY USA TODAY EDITORS HERE: When researchers control for the effect of poverty, American scores on international tests are at the top of the world.]
Poverty means poor nutrition, inadequate health care and lack of access to books, among other things. All of these profoundly impact school performance. Instead of protecting children from the effects of poverty, the Common Core provides "tough" standards and more testing.

Stephen Krashen, professor emeritus; University of Southern California; Los Angeles
http://www.usatoday.com/story/opinion/2014/04/30/common-core-education-your-say/8540559/

Original article:
http://www.usatoday.com/story/opinion/2014/04/23/common-core-education-standards-test-scores-editorials-debates/8071745/


Sources (not included in published letter)
Levels of poverty: UNICEF Innocenti Research Centre 2012, ‘Measuring Child Poverty: New league tables of child poverty in the world’s rich countries’, Innocenti Report Card 10, UNICEF Innocenti Research Centre, Florence.

Control for poverty:
Payne, K. and Biddle, B. 1999. Poor school funding, child poverty, and mathematics achievement. Educational Researcher 28 (6): 4-13; Bracey, G. 2009. The Bracey Report on the Condition of Public Education. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/Bracey-Report. Berliner, D. 2011. The Context for Interpreting PISA Results in the USA: Negativism, Chauvinism, Misunderstanding, and the Potential to Distort the Educational Systems of Nations. In Pereyra, M., Kottoff, H-G., & Cowan, R. (Eds.). PISA under examination: Changing knowledge, changing tests, and changing schools. Amsterdam: Sense Publishers. Tienken, C. 2010. Common core state standards: I wonder? Kappa Delta Phi Record 47 (1): 14-17. Carnoy, M and Rothstein, R. 2013, What Do International Tests Really Show Us about U.S. Student Performance. Washington DC: Economic Policy Institute. 2012. http://www.epi.org/).

“Poverty means poor nutrition, inadequate health care, and lack of access to books”:
Berliner, D. 2009. Poverty and Potential:  Out-of-School Factors and School Success.  Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/poverty-and-potential;   Krashen, S. 1997. Bridging inequity with books. Educational Leadership  55(4): 18-22.

More testing:
Krashen, S. 2012. How much testing? http://dianeravitch.net/2012/07/25/stephen-­‐ krashen-­‐how-­‐much-­‐testing/
 and: http://www.washingtonpost.com/blogs/answer-sheet/

Increasing testing does not mean greater achievement:
Nichols, S., Glass, G., and Berliner, D. 2006. High-stakes testing and student achievement: Does accountability increase student learning? Education Policy Archives 14(1).  http://epaa.asu.edu/epaa/v14n1/. OECD. Tienken, C., 2011. Common core standards: An example of data-less decision-making. Journal of Scholarship and Practice. American Association of School Administrators [AASA], 7(4): 3-18. http://www.aasa.org/jsp.aspx.

MOST BIASED POLL EVER

The AP reported that according to a recent poll, 7 out of 10 Californians support the common core.  Those who took the poll first read a description of the common core:

“The Common Core State Standards are a single set of K–12 English language arts and math standards that most states, including California, have voluntarily adopted. The state leaders who developed the standards say they are designed to ensure that students graduating from high school have the knowledge and skills they need to enter college programs or the workforce. In general, do you favor or oppose these standards?”  (Public Policy Institute of California)

This must be the most biased poll ever done. Only 19% said they were very familiar with the common core. This description, devoid of meaningful detail, made them even less informed.

http://www.edweek.org/ew/articles/2014/04/24/694460_ap.html?cmp=ENL-EU-NEWS2

For an alternative, please see: Multiple Choice Quiz on US Education (The Progressive, 1/26/13)  S. Ohanian and S. Krashen http://sdkrashen.com/articles.php?cat=4

The brave new technology

Sent to the Wall Street Journal, April 22, 2014

I do not share the enthusiasm about new digital technology in education ("Founding principles in the digital age," April 22).

There is no evidence that the brave new online tests that promise "real time testing" will be any better than those we have now. In fact, we have not even seen them in action: According to Fair Test (fairtest.org), current common core tests are of the old multiple choice variety.

Nor is there evidence that digital tools that present lectures online and reduce interaction with teachers ("flipped classrooms") will improve achievement. This technology is being pushed on classrooms without proper research.

Instead of careful experimentation, the plan is to use nearly the entire student population as experimental subjects.

Jumping in without proper preparation wastes our students' time and will cost taxpayers much more money in the long run.

Stephen Krashen
Professor Emeritus
University of Southern California

original article: http://online.wsj.com/news/articles/SB10001424052702304279904579515444132790598

Lack of evidence for flipped classrooms: http://www.ascd.org/publications/educational-leadership/mar13/vol70/num06/Evidence-on-Flipped-Classrooms-Is-Still-Coming-In.aspx
Fairtest: http://www.fairtest.org/common-core-assessments-factsheet

A weapon of mass distraction

Response to (“We Need to Talk About the Tests,” Op-Ed, April 10), published under the heading 

"A Need to Go Public With Test Data"  

Published in the New York Times, April 16, 2014

Yes, there’s a big problem with the state tests. But there’s a bigger problem: the whole idea of the Common Core standards. Accurately described by Susan Ohanian, a writer and former teacher, as “a radical untried curriculum overhaul” and “nonstop national testing,” the Common Core is an outrageous scheme with no justification and no empirical support.

The problems described by Elizabeth Phillips will eventually be solved, or at least reduced enough to stop complaints from coming, but the Common Core boondoggle will continue, with new and very expensive tests delivered online.

I suspect that these bad tests are a weapon of mass distraction, so that we forget what the real problem is.

STEPHEN KRASHEN
Los Angeles, April 11, 2014

The writer is professor emeritus at Rossier School of Education, University of Southern California.

This letter and other responses: http://www.nytimes.com/2014/04/17/opinion/a-need-to-go-public-with-test-data.html?partner=rssnyt&emc=rss&_r=0

Original version:

Yes, there's a big problem with the tests ("We need to talk about the tests," April 10). but there's a bigger problem: The whole idea of the common core. Accurately described by Susan Ohanian as  “a radical untried curriculum overhaul and ... nonstop national testing," the common core is an outrageous scheme with no justification and no empirical support.

The problems described by Elizabeth Phillips will eventually be solved, or at least reduced enough to stop complaints from coming, but the common core boondoggle will continue, with new and very expensive tests delivered online.

I suspect that the tests are horrible on purpose, to encourage resistance and debate over  details. When changes are made, it will give critics a sense of accomplishment, while they forget what the real problem is. The lousy tests are a weapon of mass distraction.

Stephen Krashen