Don't trust Ed Trust. Substance 27 (6): 3. 2002.
Stephen Krashen
The powerful impact of poverty on literacy development has been well documented. Children of poverty, in addition to the obvious problems they face, have very little access to reading material ; they have fewer books in the home, inferior public libraries, inferior school libraries,and inferior classroom libraries, (e.g. Duke, 2000; Neuman and Celano, 2001). This means, of course, that they have fewer opportunities to read, and therefore make less progress in developing literacy.
The recent report from Educational Trust West (Ali and Jerald, 2001) appears, at first glance, to show that a significant number of children in poverty have overcome this problem. The report claimed to find 3,592 schools in the US that were "high- performing-high poverty" schools. In California alone, there were 355 high- performing-high poverty school. This result was considered sufficient to "dispel the myth" about the relationship between poverty and educational achievement, and was followed by newspaper articles proclaiming that these high-scoring schools can "offer a lesson" (New York Times, December 17, 2001; Los Angeles Daily News, December 16, 2001).
The Ed Trust Report deserves another look. It has serious flaws, and, in fact, shows exactly the opposite of what it says it shows.
Very few schools qualify. The number of schools classified as high-poverty high- scoring represents about 4% of the nation and state school population. Moreover, a closer look shrinks even this number to considerably. In fact, it shrinks it to nearly zero.
It is easy to qualify as high-scoring. A high-performing school was defined as one in which students in ANY grade scored in the upper third of the schools in its own state in EITHER math or reading. Thus, a good performance by one grade level (in some schools only one classroom) on one test can qualify a school as "high performing."
Consider the case of California. Of the 355 "high-scoring" schools in California, only 134 were high-scoring in reading. There are 8761 schools in California. This means that about 1.5% qualify as "high-flying schools." Of these 134, 83 managed to qualify because of children in only one grade level! This could be due to the performance of a few students in one classroom, perhaps even those from higher- income families (see below). We are now down to 51 schools, about half of one percent.
The powerful impact of poverty on literacy development has been well documented. Children of poverty, in addition to the obvious problems they face, have very little access to reading material ; they have fewer books in the home, inferior public libraries, inferior school libraries,and inferior classroom libraries, (e.g. Duke, 2000; Neuman and Celano, 2001). This means, of course, that they have fewer opportunities to read, and therefore make less progress in developing literacy.
The recent report from Educational Trust West (Ali and Jerald, 2001) appears, at first glance, to show that a significant number of children in poverty have overcome this problem. The report claimed to find 3,592 schools in the US that were "high- performing-high poverty" schools. In California alone, there were 355 high- performing-high poverty school. This result was considered sufficient to "dispel the myth" about the relationship between poverty and educational achievement, and was followed by newspaper articles proclaiming that these high-scoring schools can "offer a lesson" (New York Times, December 17, 2001; Los Angeles Daily News, December 16, 2001).
The Ed Trust Report deserves another look. It has serious flaws, and, in fact, shows exactly the opposite of what it says it shows.
Very few schools qualify. The number of schools classified as high-poverty high- scoring represents about 4% of the nation and state school population. Moreover, a closer look shrinks even this number to considerably. In fact, it shrinks it to nearly zero.
It is easy to qualify as high-scoring. A high-performing school was defined as one in which students in ANY grade scored in the upper third of the schools in its own state in EITHER math or reading. Thus, a good performance by one grade level (in some schools only one classroom) on one test can qualify a school as "high performing."
Consider the case of California. Of the 355 "high-scoring" schools in California, only 134 were high-scoring in reading. There are 8761 schools in California. This means that about 1.5% qualify as "high-flying schools." Of these 134, 83 managed to qualify because of children in only one grade level! This could be due to the performance of a few students in one classroom, perhaps even those from higher- income families (see below). We are now down to 51 schools, about half of one percent.
Scores can be based on students NOT considered high poverty. Ed Trust may
claim that a grade in a high poverty school reached the upper 1/3, but not all the
children at that grade level were high poverty. Consider the case of fourth graders at
the Language Academy, a (magnet) school in San Diego. Academy fourth graders
scored in the upper 1/3 of the state in reading, averaging 61. But the subset of
economically disadvantaged children (n = 27) scored 42, while the advantaged
children (n = 36) averaged 73. Fourth graders at Language Academy were classified
as high scoring high poverty not because of the scores of its disadvantaged children
but because of the scores of its advantaged children. Ed Trust does not present this
kind of a breakdown of scores.
Ed Trust used a low standard for classification as "high poverty." A high- poverty school was defined as one in which at least 50% of the students were from low-income families. The California average is 46%.
The report has numerous inaccuracies. For California, several schools listed as high-poverty were not, and in many cases grade levels Ed Trust said were high scoring were not. The alternative analysis below presents details, as well as confirming that the number of "high-poverty high-scoring schools" is very very small.
An alternative analysis
If we define truly exceptional schools as those with at least three grade levels scoring in the upper one-third in reading, we are down to 20 schools in California. Let's take a closer look at the 20: In two cases, the schools did not qualify as high- poverty, even according to the very modest standard set by Ed Trust.1 For the other 18, a look at SAT9 scores shows that only four of the schools actually had all three classes in the upper one-third in reading, based on California's standards, and none qualified as a high-scoring school using national standards. Of the four that qualified in California, one was a magnet school. The high-scoring classes in the three other schools had a total of 391 children. In one, the Steinbeck school, high scorers in two grades (3 and 6) scored much lower on the language portion of the SAT9 (36 and 30).2
Ed Trust used a low standard for classification as "high poverty." A high- poverty school was defined as one in which at least 50% of the students were from low-income families. The California average is 46%.
The report has numerous inaccuracies. For California, several schools listed as high-poverty were not, and in many cases grade levels Ed Trust said were high scoring were not. The alternative analysis below presents details, as well as confirming that the number of "high-poverty high-scoring schools" is very very small.
An alternative analysis
If we define truly exceptional schools as those with at least three grade levels scoring in the upper one-third in reading, we are down to 20 schools in California. Let's take a closer look at the 20: In two cases, the schools did not qualify as high- poverty, even according to the very modest standard set by Ed Trust.1 For the other 18, a look at SAT9 scores shows that only four of the schools actually had all three classes in the upper one-third in reading, based on California's standards, and none qualified as a high-scoring school using national standards. Of the four that qualified in California, one was a magnet school. The high-scoring classes in the three other schools had a total of 391 children. In one, the Steinbeck school, high scorers in two grades (3 and 6) scored much lower on the language portion of the SAT9 (36 and 30).2
Poverty has a powerful effect
The Ed Trust report is actually a stunning confirmation of the overwhelming effect
of poverty. Even with a very loose definition of high performance, few schools
perform in the upper one-third and a careful look at one state reveals that even
fewer qualify. California has about five million children in school. Ed Trust claimed
that about 230,000 were in high-poverty high-scoring schools for reading.
According to this analysis, the real figure is less than 400. It is extremely difficult to
"defy the odds." Poverty has a powerful effect on educational attainment.
Notes
Notes
-
The Raoul Wallenberg school reported only 41.5% and Richmond only 36.4% of their
students on free or reduced price lunch. Wheatland Union had 50.5% and Pescadero had
50.8% on free and reduced lunch. These were included as "high-poverty" schools.
-
It is a lot easier to place in the upper 1/3 in California than in the most other states;
California ranks at the bottom in reading among states in the USA. State averages are really
low in grade 2 (30th percentile), 9 (33rd), 10 (33rd) and 11 (37th). The fifth, sixth and eighth
grade CA average is 43. All are under the national average of 50.
Even using this lower standard, only four schools in California had three grade levels that actually scored in the upper 1/3 for reading: Borrego Springs (81 children), Bravo Magnet (about 1000 children), Steinbeck (193 children) and Kernville (117). Three out of four grades nominated by Ed Trust actually qualified at Steinbeck and Kernville. For Kernville, grades 4, 5 and 6 met the standard, but there were few disadvantaged children in grade 5.
Ten of the 18 schools had no grade levels meeting the California standard for the upper 1/3: Costano, Cottonwood, Florence, Happy Camp High, Hayfork High, Kernville, Muir, Surprise Valley, Surprise Valley High, Van Duzen.
In five schools, results were mixed: At Clairemont, grades 9 and 10 qualified, but grade 11 did not. For the Language Academy, grades 3 and 4 qualified, but not the subset of disadvantaged children. Grade 7 did not. For Pescadero, two of the three grades did not qualify. Grade 5 did, but not the subset of disadvantaged children. For Perry: grade 3 qualified but not grades 5 and 6. At Wheatland, grade 9 qualified, but not the subset of disadvantaged children. Grade 10 qualified but grade 11 did not.
(California scores were calculated from mean scores (percentile ranks) provided by the State of California Department of Education website, and converting to NCE's.)
Here are SAT9 scores for those grades in "high-poverty" schools categorized as achieving in the upper 1/3.
1) Borrego Springs; grade 9 = 60; grade 10 = 44; grade 11 = 46; only nine disadvantaged
students were tested in grade 9
2) Bravo Magnet: grade 9 = 43 (493); grade 10 = 43 (389); grade 11 = 36 (400); for disadvantaged students only, grade 9 = 44 (434); grade 10= 43 (314); grade 11 = 34 (320).
3) Clairemont; grade 9 = 43 (493); grade 43 (389) ;grade 11 = 36 (400). For disadvantaged students, grade 9 = 44 (434); grade 10 = 43 (314); grade 11 = 34 (320).
4) Costano; grade 3 = 60 (37); grade 4 = 41 (72); grade 7 = 39 (49); for disadvantaged children only, grade 3 = 63 (40); grade 4 = 39 (41); grade 7 = 30 (34). Note that this school reported more disadvantaged children tested than total children tested for grade 3.
5) Cottonwood; grade 3 = 42 (28); grade 5 = 33 (24); grade 7 = 41 (19); for disadvantaged children only, grade 3 = 23 (19); grade 5 = 43 (15); grade 7, no score reported, 10 tested.
6) Florence; grade 3 = 39 (26); grade 4 = 30 (23), grade 5 = 31 (23), grade 6 = 39 (13); for disadvantaged only, grade 3 = 33 (16); grade 4 = 79 (21); grade 5 = 33 (13); grade 6 = 61 (12). Note that the grade 4 and grade 6 scores are mathematically impossible.
7) Happy Camp High; grade 9 = 37 (24); grade 10 = 34 (33); grade 11 = 33 (19). All students were disadvantaged.
8) Hayfork High; grade 9 = 23 (43 students tested); grade 10 = 41 (36 tested); grade 11 = 43 (33 tested); for disadvantaged students only; grade 9 = 30 (23); grade 10 = 22 (23); grade 11 = 24 (32)
9) Kernville; grade 3 = 37 (28 students tested); grade 4 = 70 (31 tested); grade 5 = 73 (23); grade 6 = 63 (30 tested); for disadvantaged students only, grade 3 = 23 (13 tested); grade 4 = 66 (13 tested); grade 5 = no score given, 10 tested; grade 6 = 60 (16)
10) Language Academy (San Diego): grade 3 = 62 (34); grade 4 = 61 (63); grade 7 = 43 (21); for disadvantaged children only: grade 3 = 34 (44); grade 4 = 42 (27); grade 7 = 42 (13) 11) Muir; grade 4 = 43 (26); grade 8 = 43 (34); grade 11, not reported, only ten students tested. For disadvantaged children only, grade 4 = 37 (130; grade 8 = 41 (23); no scores reported for grade 11, only 3 children tested.
12) Pescadero: This school has 50.8% disadvantaged children. grade 3 = 20 (39 students tested); grade 4 = 37 (28 tested); grade 5 = 60 (21 tested); disadvantaged only, grade 3 = no scores given, 9 tested; grade 4 = no scores given, 3 tested; grade 5 = 12 (23 tested).
13) Perry: grade 3 = 68 (31); grade 4 = 39 (43); grade 5 = 43 (43); grade 6 = 43 (32); for disadvantage children only, grade 3 = 74 (33), grade 4 = 60 (33); grade 5 = 43 (29); grade 6 = 30 (31).
14) Steinbeck; grade 3 = 69 (34), grade 4 = 61 (60); grade 5 = 32 (30), grade 6 = 67 (63). All children were disadvantaged.
15) Surprise Valley; grade 4 = 38 (14); grade 7 = 39 (16), grade 8 = 33 (16). No scores reported for disadvantaged children. Only 9 tested in grade 4, 3 tested in grade 7, 10 tested in grade 8.
16) Surprise Valley High School; grade 9 = 37 (13); grade 10 = 30 (14), no scores reported for grade 11, only 10 students tested. No scores reported for disadvantaged students; grade 9 had 7, grade 10 had 6 and grade 11 had 2.
17) Van Duzen; grade 4 = 44 (13); grade 5 = 38 (17); grade 8 = 30 (11); no scores reported for disadvantage children. Ten tested in grades 4,5 and 3 in grade 8.
18) Wheatland Union High School; grade 9 = 44 (179); grade 10 = 40 (133); grade 11 = 42 (147); for disadvantaged children, grade 9 = 33 (61); grade 10 = 40 (32); grade 11 = 36 (36)
Acknowledgment: My thanks to Gerald Coles for numerous insights and suggestions.
2) Bravo Magnet: grade 9 = 43 (493); grade 10 = 43 (389); grade 11 = 36 (400); for disadvantaged students only, grade 9 = 44 (434); grade 10= 43 (314); grade 11 = 34 (320).
3) Clairemont; grade 9 = 43 (493); grade 43 (389) ;grade 11 = 36 (400). For disadvantaged students, grade 9 = 44 (434); grade 10 = 43 (314); grade 11 = 34 (320).
4) Costano; grade 3 = 60 (37); grade 4 = 41 (72); grade 7 = 39 (49); for disadvantaged children only, grade 3 = 63 (40); grade 4 = 39 (41); grade 7 = 30 (34). Note that this school reported more disadvantaged children tested than total children tested for grade 3.
5) Cottonwood; grade 3 = 42 (28); grade 5 = 33 (24); grade 7 = 41 (19); for disadvantaged children only, grade 3 = 23 (19); grade 5 = 43 (15); grade 7, no score reported, 10 tested.
6) Florence; grade 3 = 39 (26); grade 4 = 30 (23), grade 5 = 31 (23), grade 6 = 39 (13); for disadvantaged only, grade 3 = 33 (16); grade 4 = 79 (21); grade 5 = 33 (13); grade 6 = 61 (12). Note that the grade 4 and grade 6 scores are mathematically impossible.
7) Happy Camp High; grade 9 = 37 (24); grade 10 = 34 (33); grade 11 = 33 (19). All students were disadvantaged.
8) Hayfork High; grade 9 = 23 (43 students tested); grade 10 = 41 (36 tested); grade 11 = 43 (33 tested); for disadvantaged students only; grade 9 = 30 (23); grade 10 = 22 (23); grade 11 = 24 (32)
9) Kernville; grade 3 = 37 (28 students tested); grade 4 = 70 (31 tested); grade 5 = 73 (23); grade 6 = 63 (30 tested); for disadvantaged students only, grade 3 = 23 (13 tested); grade 4 = 66 (13 tested); grade 5 = no score given, 10 tested; grade 6 = 60 (16)
10) Language Academy (San Diego): grade 3 = 62 (34); grade 4 = 61 (63); grade 7 = 43 (21); for disadvantaged children only: grade 3 = 34 (44); grade 4 = 42 (27); grade 7 = 42 (13) 11) Muir; grade 4 = 43 (26); grade 8 = 43 (34); grade 11, not reported, only ten students tested. For disadvantaged children only, grade 4 = 37 (130; grade 8 = 41 (23); no scores reported for grade 11, only 3 children tested.
12) Pescadero: This school has 50.8% disadvantaged children. grade 3 = 20 (39 students tested); grade 4 = 37 (28 tested); grade 5 = 60 (21 tested); disadvantaged only, grade 3 = no scores given, 9 tested; grade 4 = no scores given, 3 tested; grade 5 = 12 (23 tested).
13) Perry: grade 3 = 68 (31); grade 4 = 39 (43); grade 5 = 43 (43); grade 6 = 43 (32); for disadvantage children only, grade 3 = 74 (33), grade 4 = 60 (33); grade 5 = 43 (29); grade 6 = 30 (31).
14) Steinbeck; grade 3 = 69 (34), grade 4 = 61 (60); grade 5 = 32 (30), grade 6 = 67 (63). All children were disadvantaged.
15) Surprise Valley; grade 4 = 38 (14); grade 7 = 39 (16), grade 8 = 33 (16). No scores reported for disadvantaged children. Only 9 tested in grade 4, 3 tested in grade 7, 10 tested in grade 8.
16) Surprise Valley High School; grade 9 = 37 (13); grade 10 = 30 (14), no scores reported for grade 11, only 10 students tested. No scores reported for disadvantaged students; grade 9 had 7, grade 10 had 6 and grade 11 had 2.
17) Van Duzen; grade 4 = 44 (13); grade 5 = 38 (17); grade 8 = 30 (11); no scores reported for disadvantage children. Ten tested in grades 4,5 and 3 in grade 8.
18) Wheatland Union High School; grade 9 = 44 (179); grade 10 = 40 (133); grade 11 = 42 (147); for disadvantaged children, grade 9 = 33 (61); grade 10 = 40 (32); grade 11 = 36 (36)
Acknowledgment: My thanks to Gerald Coles for numerous insights and suggestions.
References
Ali, R. and Jerald, C. 2001. Dispelling the Myth in California: Preliminary
Findings from a State and Nationwide Analysis of "High-Flying" Schools. The
Education Trust - West.
Neuman, S. and Celano, D. 2001. Access to print in low-income and middle- income communities. Reading Research Quarterly 36(1): 8-26.
Duke, N. 2000. For the rich it's richer: Print experiences and environments offered to children in very low- and very high-socioeconomic status first-grade classrooms.American Educational Research Journal 37(2): 441-478.
Los Angeles Daily News. 2001. No more excuses. Dec. 16, 2001
New York Times. 2001. School defies the odds and offers a lesson. Dec. 17, 2001
Neuman, S. and Celano, D. 2001. Access to print in low-income and middle- income communities. Reading Research Quarterly 36(1): 8-26.
Duke, N. 2000. For the rich it's richer: Print experiences and environments offered to children in very low- and very high-socioeconomic status first-grade classrooms.American Educational Research Journal 37(2): 441-478.
Los Angeles Daily News. 2001. No more excuses. Dec. 16, 2001
New York Times. 2001. School defies the odds and offers a lesson. Dec. 17, 2001
