Intensive Systematic Phonics, Tests of Reading Comprehension, and the Garan Effect

An article in the Guardian announced that a new study confirmed the positive effect of intensive systematic phonics. In my letter  to the Guardian(see below) I said that the study only confirmed what we already know: "Intensive phonics instruction helps children do better on tests in which they are asked to pronounce words out loud, and on tests of spelling. Not mentioned is the consistent finding that intensive phonics instruction makes no significant contribution to performance on tests in which children have to understand what they read."

I like to refer to this as the Garan Effect: The National Reading Panel (NICHD, 2000) concluded that the experimental research supports intensive systematic phonics.  Garan (2002), in an examination of this report, noted that the impact of intensive phonics is strong on tests in which children read lists of words in isolation, but it is miniscule on tests in which children have to understand what they read. Thus, intensive phonics instruction only helps children develop the ability to pronounce words in isolation, an ability that will emerge anyway with more reading. Garan's results agree with the results of many other studies that show that intensive phonics instruction has a positive impact on tests of decoding but not on tests of comprehension (Krashen, 2009).

Two responses of my claims have been posted on twitter. Both deserve a more detailed response than twitter allows.

In one tweet, it was maintained that the unpublished report that was the basis for the Guardian article (Grant,  2014) did in fact contain data on tests in which children had to understand what they read and that intensive phonics-trained students did better: Students who had studied of intensive systamatic phonics appeared to do well on the English SAT Stage 2 test at age 11, with 94% scoring at the 4+ level, (where 4 = "expected" and 6 = is the highest level), compared with 79% for the entire country (England), and 82% for "similar schools." 

Before we rush to reject the Garan Effect, however, it needs to be pointed out that (1) Grant (2014) asserts that the differences are statistically significant, but provides us only with percentages, no means, no standard deviations, no sample size, and no details about the tests of statistical significance.

(2) The data is based on the 2004 English SAT. That test included sections on writing and spelling in addition to reading comprehension, and I have been unable to find scores for the individual components.

(3) True experiments demand a comparison group that differs from the experimental group in only one way.  Did the intensive phonics graduates' educational experience differ in other important ways from comparisons? For example, a number of studies show that performance on reading and writing components is related to the amount of reading students do, especially free voluntary reading (Krashen, 2004).  Did the intensive phonics students have more access to books and more encouragement and time to read (e.g. sustained silent reading)?

The "Grant challenge" deserves examination, but it is not nearly enough to reject the impressive body of evidence supporting the Garan Effect.

A second challenge, also delivered by twitter, is data from Connelly, Johnston, and Thompson (2001), who  showed that intensive phonics-trained six year olds did better on the Comprehension portion of the Neale Analysis of Reading Test than children with much less phonics instruction. On the Neale, however, students read passages aloud, and are then asked comprehension questions.  While reading, their errors are tallied, and only those who make less than a certain number of errors are asked comprehension questions. This is not a situation that encourages a focus on meaning.

Connelly, V.,  Johnston,  R., and Thompson, B. 2001. The effects of phonics instruction on the reading comprehension of beginning readers. Reading and Writing 14: 423-457.

Garan, E. (2001). Beyond the smoke and mirrors: A critique of the National Reading Panel report on phonics. Phi Delta Kappan 82, no. 7 (March), 500-506.

Grant, M. 2014. Longitudinal Study from Reception to Year 2 (2010-2013) and Summary of an earlier Longitudinal Study from Reception to Year 6 (1997-2004). Unpublished paper.

Krashen, S. 2004. The Power of Reading. Second edition. Portsmouth: Heinemann and Westport: Libraries Unlimited


Krashen, S. 2009. Does intensive decoding instruction contribute to reading comprehension? Knowledge Quest 37 (4): 72-74.

Thanks to Debbie Hepplewhite and Maggie Downie for their comments.

My letter to the Guardian, published June 23, 2014

The limits of phonics

The Guardian's enthusiastic report about the efficacy of phonics is an example of "cold fusion" journalistic practice: Presenting research reports to the public before the scientific community has reviewed them. I provide one brief "peer review" here. Neither the study (thanks to the Guardian for providing a link to the preliminary report) nor the Guardian's article point out that the study only confirms what we already know: intensive phonics instruction helps children do better on tests in which they are asked to pronounce words out loud, and on tests of spelling.
Not mentioned is the consistent finding that intensive phonics instruction makes no significant contribution to performance on tests in which children have to understand what they read. Real reading ability is the result of actual reading, especially of books that readers find interesting. Good readers eventually acquire nearly all the rules of phonics and spelling, as a result of reading.
Stephen Krashen  Professor emeritus, University of Southern California
http://www.theguardian.com/education/2014/jun/23/gove-love-move-limits-of-phonics