The problem is not a lack of technology. The problem is poverty.

Comment on "Education Needs to Change as Fast as Technology," by Zack Sims. Posted at

http://www.forbes.com/sites/techonomy/2014/05/23/education-needs-to-change-as-fast-as-technology/

Mr. Sims hasn't done his homework. He points out that the "US is ranked by the Organization for Economic Cooperation and Development, the U.S. is 31st in math achievement, 24th in science, and 21st in reading." But a number of studies have shown that when we control for the effect of poverty, our students rank near the top of the world.

Mr. Sims also has a great deal of faith in flipped classrooms. But there is no research supporting this faith.  This technology is being pushed on classrooms without proper research.

The problem is not a lack of technology. The problem is poverty. The US child poverty rate is an inexcusable 24%.

Sources:

Control for effect of poverty: Payne, K. and Biddle, B. 1999. Poor school funding, child poverty, and mathematics achievement. Educational Researcher 28 (6): 4-13; Bracey, G. 2009. The Bracey Report on the Condition of Public Education. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/Bracey-Report. Berliner, D. 2011. The Context for Interpreting PISA Results in the USA: Negativism, Chauvinism, Misunderstanding, and the Potential to Distort the Educational Systems of Nations. In Pereyra, M., Kottoff, H-G., & Cowan, R. (Eds.). PISA under examination: Changing knowledge, changing tests, and changing schools. Amsterdam: Sense Publishers. Tienken, C. 2010. Common core state standards: I wonder? Kappa Delta Phi Record 47 (1): 14-17. Carnoy, M and Rothstein, R. 2013, What Do International Tests Really Show Us about U.S. Student Performance. Washington DC: Economic Policy Institute. 2012. http://www.epi.org/).

Lack of evidence for flipped classrooms: http://www.ascd.org/publications/educational-leadership/mar13/vol70/num06/Evidence-on-Flipped-Classrooms-Is-Still-Coming-In.aspx

Fairtest: http://www.fairtest.org/common-core-assessments-factsheet

Free Advertising for Duolingo

Sent to the Japan Times, May 23, 2014
Contrary to the impression given by The Japan Times's free commercial for Duolingo ("Duolingo chief shakes up language learning," May 23), there is no clear evidence that Duolinguo is effective in teaching foreign languages. The only research study done assessing the impact of Duolingo, which was funded by Duolingo, did not produce strong results: Forty-four percent of the subjects, mostly college graduates and many with advanced degrees, did not finish the 22-hour course and 25% of the subjects completed eight hours or less, with one subject doing only two hours.
In contrast, there are hundreds of published papers in respected scientific journals demonstrating what really does work in teaching foreign languages, and unlike Duolingo, these approaches do not require a computer.
For example, studies show that methods based on story telling and easy reading result in profound gains for beginners in English and other languages. For intermediate students, self-selected pleasure reading has been shown to be more effective than traditional methods for developing vocabulary, grammar, spelling, and writing style. Also, reading can be so pleasant that students are often eager to do it on their own.
Many of these studies have been done with those acquiring English as a second language in Japan, and they have been published in scholarly journals all over the world.  I hope that the Japan Times, its readers, and Duolingo CEO Luis von Ahn (who has no background in language education research), will study the work of some of the prominent scholars in Japan who have done quality research in this area, including Profs. Beniko Mason, David Beglar, and Atsuko Takase. 
Stephen Krashen

Some Sources:
Mason, B.,Vanata, M., Jander, K., Borsch, R., and Krashen, S. 2009. The effects and efficiency of hearing stories on vocabulary acquisition by students of German as a second foreign language in Japan. The Indonesian Journal of English Language Teaching, 5(1), 1-14.
Mason, B. 2011. Impressive gains on the TOEIC after one year of comprehensible input, with no output or grammar study. International Journal of Foreign Language Teaching 7(1). (ijflt.com)
Krashen, S. 2014. Does Duolingo "trump" university-level language learning? International Journal of Foreign Language Teaching 9(1):13-15.
Vesselinov, R. and Grego, J. 2012. Duolingo effectiveness study. Final Report. static.duolingo.com/s3/DuolingoReport_Final.pdf

Original article: Duolingo chief shakes up language learninghttp://www.japantimes.co.jp/news/2014/05/23/national/duolingo-chief-shakes-up-language-learning/#.U3-O8y9efhM

In education, satire is obsolete (Alfie Kohn)

Quite a few people who reacted to my post (see below) thought it was real. It wasn't. It was an attempt at satire. Part of its failure was my fault, for not making it clear that it was satire. And part was the situation we are now living in, a situation so bizarre that this kind of announcement is believable. The reaction confirms Alfie Kohn's statement: In education, satire is now obsolete.

The post was inspired by the news that 69% of Californians said they liked the Common Core standards after hearing a brief statement about them. The statement said only that they "are designed to ensure that students graduating from high school have the knowledge and skills they need to enter college programs or the workforce." I'm surprised anybody would object.

I owe more than a hat-tip to Andy Borowitz, who writes amazing political satire. Follow him on borowitzreport.com.

OVERWHELMING SUPPORT FOR THE COMMON CORE!!!

There is now no doubt: Americans overwhelmingly support the common core. In a poll organized by the Pearson Publishing Company, 96% strongly agreed with the statement, "Schools should teach important things." By a wide margin, those surveyed also agreed that "teachers should help students learn stuff."

Education Secretary Arne Duncan announed that "This poll is a real-game changer. Despite the complaints of nay-sayers, the public has finally got the message about the common core."

In a separate poll carried out among staff members of the US Department of Education, 92% demonstrated their familiarity with learning theory, responding that they recognized the name "Piaget." Of those who did, however, 82% associated the name with a watch company.

(Hat-tip: Andy Borowitz)

OVERWHELMING SUPPORT FOR THE COMMON CORE!!!

There is now no doubt: Americans overwhelmingly support the common core. In a poll organized by the Pearson Publishing Company, 96% strongly agreed with the statement, "Schools should teach important things." By a wide margin, those surveyed also agreed that "teachers should help students learn stuff."

Education Secretary Arne Duncan announced that "This poll is a real-game changer. Despite the complaints of nay-sayers, the public has finally got the message about the common core."

In a separate poll carried out among staff members of the US Department of Education, 92% demonstrated their familiarity with learning theory, responding that they recognized the name "Piaget." Of those who did, however, 82% associated the name with a watch company. 

Hat-tip: Andy Borowitz

The common core: Don't forget what the real problem is.

Sent to the Seattle Times, May 15, 2014

I think the common core math standards are lousy on purpose, made lousy so that parents will complain ("2+2=What? Parents rail against Common Core math," May 15).
The common core architects will then make some adjustments, enough to stop the most vigorous complaints. Many complainers will then be satisfied and the public will be impressed with how open-minded the common core directors are.
But the common core itself will remain unchallenged. The common core was designed by non-educators, has no research supporting it, and will be enforced by more testing than we have ever seen on this planet.  It will also cost billions, and the plan to test all students online ensures massive and increasing technology expenditures forever.
Parents are right to protest the math standards, but should not forget what the real problem is.

Stephen Krashen

original article http://seattletimes.com/html/nationworld/2023611545_apxcommoncorecomplicatedmath.html?syndication=rss

Does Shanghai have the world's best public schools?

Does Shanghai have the world's best public schools?
Sent to Newsweek

"Lessons from the world's best public school" (May 9) considers Shanghai to be the "world's best education system" (p. 30), because Shanghai's scores on international tests such as the PISA are at the top of the world. As Newsweek notes, however, Shanghai is "one of China's richest cities" (p. 31) and the children of migrant workers are excluded from their public schools. 

In contrast, American students' PISA scores are unspectacular. But  children are not excluded from school in the US,  and American schools must deal with a very high rate of child poverty, 24%, the second highest among 34 advanced economic countries.

Any comparison of educational programs must consider the effect of poverty.  Every study ever done has shown that poverty has a devastating effect on school performance.  Poverty means, among other things, food deprivation, lack of health care, and lack of access to books.

When researchers control for the effect of poverty, American scores are among the best in the world. When we examine middle class American students in well-funded schools, their scores are close to Shanghai's and ahead of all other countries tested. 

The current move in the US to become more like China in education, with more competition and more rigorous examinations, will do nothing to improve student achievement in the US. The problem is poverty. 


Stephen Krashen

Original article: http://www.newsweek.com/2014/05/09/shanghai-high-confidential-249224.html

Sources:

Level of poverty:
UNICEF Innocenti Research Centre (2012), ‘Measuring Child Poverty: New league tables of child poverty in the world’s rich countries’, Innocenti Report Card 10, UNICEF Innocenti Research Centre, Florence.


Control for poverty:
Payne, K. and Biddle, B. 1999. Poor school funding, child poverty, and mathematics achievement. Educational Researcher 28 (6): 4-13; Bracey, G. 2009. The Bracey Report on the Condition of Public Education. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/Bracey-Report. Berliner, D. 2011. The Context for Interpreting PISA Results in the USA: Negativism, Chauvinism, Misunderstanding, and the Potential to Distort the Educational Systems of Nations. In Pereyra, M., Kottoff, H-G., & Cowan, R. (Eds.). PISA under examination: Changing knowledge, changing tests, and changing schools. Amsterdam: Sense Publishers. Tienken, C. 2010. Common core state standards: I wonder? Kappa Delta Phi Record 47 (1): 14-17. Carnoy, M and Rothstein, R. 2013, What Do International Tests Really Show Us about U.S. Student Performance. Washington DC: Economic Policy Institute. 2012. http://www.epi.org/). Krashen, S.  2010. How poverty affected U.S. PISA scores. http://voices.washingtonpost.com/answer-sheet/research/how-poverty-affected-us-pisa-s.html

“Poverty means poor nutrition, inadequate health care, and lack of access to books”:
Berliner, D. 2009. Poverty and Potential:  Out-of-School Factors and School Success.  Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/poverty-and-potential;   Krashen, S. 1997. Bridging inequity with books. Educational Leadership  55(4): 18-22.

Increasing testing does not mean greater achievement:
Nichols, S., Glass, G., and Berliner, D. 2006. High-stakes testing and student achievement: Does accountability increase student learning? Education Policy Archives 14(1).  http://epaa.asu.edu/epaa/v14n1/. OECD. Tienken, C., 2011. Common core standards: An example of data-less decision-making. Journal of Scholarship and Practice. American Association of School Administrators [AASA], 7(4): 3-18. http://www.aasa.org/jsp.aspx.

Response to Southern Poverty Law Center Statement on the Common Core

Response to: Public Schools in the Crosshairs: Far-Right Propaganda and the Common Core State Standards (Southern Poverty Law Center).

http://www.splcenter.org/get-informed/publications/Public-Schools-in-the-Crosshairs-Far-Right-Propaganda-and-the-Common-Core-State-Standards

 I agree that the debate about the common core must be rooted in the facts, and that the "propoganda machine on the right" has "polluted the debate" with outrageous accusations. There are, however, serious and legitimate arguments against the common core.

The stated reason for the common core is the supposedly poor performance of American students. But  when researchers control for the effect of poverty, American students' international test scores are at the top of the world.  Our  overall scores are unspectacular (but not terrible) because we have so much child poverty, 24%, the second highest among all economically advanced countries. 

Poverty means poor diet, inadequate health care, and little or no access to books. All of these have devastating effects on school performance.  The best teaching has little effect when children are hungry, ill and have nothing to read.

The common core not only ignores the real problem; it does nothing to protect children from the effects of poverty. It only offers us a an extremely expensive plan with no basis in the research: There is no research supporting "tough" standards or nonstop testing. Also, studies show that increasing testing does not improve school achievement.

The common core is a bad solution that is aimed at the wrong problem. 

Stephen Krashen

SOURCES:

Levels of poverty:

UNICEF Innocenti Research Centre 2012, ‘Measuring Child Poverty: New league tables of child poverty in the world’s rich countries’, Innocenti Report Card 10, UNICEF Innocenti Research Centre, Florence.

Control for poverty:

Payne, K. and Biddle, B. 1999. Poor school funding, child poverty, and mathematics achievement. Educational Researcher 28 (6): 4-13; Bracey, G. 2009. The Bracey Report on the Condition of Public Education. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/Bracey-Report. Berliner, D. 2011. The Context for Interpreting PISA Results in the USA: Negativism, Chauvinism, Misunderstanding, and the Potential to Distort the Educational Systems of Nations. In Pereyra, M., Kottoff, H-G., & Cowan, R. (Eds.). PISA under examination: Changing knowledge, changing tests, and changing schools. Amsterdam: Sense Publishers. Tienken, C. 2010. Common core state standards: I wonder? Kappa Delta Phi Record 47 (1): 14-17. Carnoy, M and Rothstein, R. 2013, What Do International Tests Really Show Us about U.S. Student Performance. Washington DC: Economic Policy Institute. 2012. http://www.epi.org/).

“Poverty means poor nutrition, inadequate health care, and lack of access to books”:

Berliner, D. 2009. Poverty and Potential:  Out-of-School Factors and School Success.  Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/poverty-and-potential;   Krashen, S. 1997. Bridging inequity with books. Educational Leadership  55(4): 18-22.

Increasing testing does not mean greater achievement:

Nichols, S., Glass, G., and Berliner, D. 2006. High-stakes testing and student achievement: Does accountability increase student learning? Education Policy Archives 14(1).  http://epaa.asu.edu/epaa/v14n1/. OECD. Tienken, C., 2011. Common core standards: An example of data-less decision-making. Journal of Scholarship and Practice. American Association of School Administrators [AASA], 7(4): 3-18. http://www.aasa.org/jsp.aspx.

Reading for points vs. reading for pleasure

Sent to the Christian Science Monitor, May 9, 2014

The book popularity data from "What are kids reading? Books like 'Hunger Games,' but classics, too," (May 8) comes from reading done in preparation for tests, not reading for pleasure.

The data comes from reading done as part of the Accelerated Reader (AR) program. AR sells tests that students take after reading a book. Students are awarded points depending on how many questions they get right, and can exchange the points for prizes. Thus, AR readers are reading in order to score points on AR tests, and try to remember details that might be on the test, but that are often not relevant to the story (e.g. "What is the name of the talking parrot?" in Dr. Dolittle).

Young readers might be using different criteria in reading for points as contrasted with reading for pleasure.

Stephen Krashen

original article: http://www.csmonitor.com/Books/2014/0508/What-are-kids-reading-Books-like-Hunger-Games-but-classics-too

12th grade NAEP scores: Does the common core deserve the credit?

Sent to the Hartford Courant, May 7, 2014

It is a premature to give the "new academic standards" of the common core credit for Connecticut's good performance on the 12th grade NAEP reading test ("State Students Excel On 'Nation's Report Card'," May 7).

First, the improvement was not uniform: Grade 8 scores increased less than grade 12 scores, and grade 4 scores stayed the same as they were in 2009. The gap between high poverty (eligible for free and reduced lunch) and low poverty students remained nearly the same as it was in 2009.

Second, to show that the common core is the factor, we would have to compare states that are equivalent in all ways except implementation of the common core. This has not been done.

Thus far, no research has shown that tough standards and increased testing increase NAEP scores. Research has shown, however, that low levels of poverty and access to school and public libraries are associated with higher NAEP scores.

Stephen Krashen

Original article: http://www.courant.com/news/education/hc-naep-scores-0508-20140507,0,3768766.story?track=rss

Sources:
More testing does not increase achievement: Nichols, Sharon L., Gene V. Glass, and David C. Berliner. 2006. “High-Stakes Testing and Student Achievement: Does Accountability Pressure Increase Student Learning?” Education Policy Archives 14 (1). <http://epaa.asu.edu/ojs/article/view/72/198> (accessed October 14, 2013).

Impact of libraries on NAEP scores: McQuillan, J. (1998). The literacy crisis: False claims and real solutions. Portsmouth, NH: Heinemann Publishing Company.,
Krashen, S., Lee, S.Y. and McQuillan, J. 2012. Is the library important? Multivariate studies at the national and international level. Journal of Language and Literacy Education, 8(1): 26-36.

Lack of access to books in high poverty neighborhoods. Neuman, S., and D. Celano. 2001. Access to print in low-income and middle-income communities. Reading Research Quarterly 36(1): 8-26.

Amount of time for testing "just right"?

The headline for "Survey: More Educators Think 'Just the Right Amount' of Time Is Spent on Testing" is deceptive.
Yes, a slightly smaller percentage of teachers think there is too much time taken for testing, and a slightly larger percentage think there is "just enough"time taken for  testing compared to two years ago, but 52% of teachers still think there is too much time preparing for and taking tests and 53% think students spend too much time in test taking and preparation.
Note that the question asked was about both "preparing for and taking assessments." In a sidebar, the original report  notes that in 2013, 70 percent of teachers and 55 percent of district administrators said that "the focus on state accountability tests takes too much time away from learning" (p. 9).  No data from two years ago was reported. Ed Week did not report this figure.
Only a tiny percentage thought that not enough time was dedicated to test preparation and administration.  I am sure that staff members of the US Department of Education are in agreement with this small group: As the standards expand to more subject matter areas, we can, of course, expect more testing.  (I have documented this in S. Krashen, 2012, “How Much Testing?” Diane Ravitch’s Blog (July 25). <http://dianeravitch.net/?s=how+much+testing> (accessed October 13, 2013).
Another important result: Students said they value teacher-made tests far more than they value standardized tests: 54% thought that teacher-made tests helped them understand what they learned, but only 21% felt that way about "state accountability tests."
Also not reported in the Ed Week article: 92% of girls and 87% of boys said that they thought that the teacher "cares about my learning" and 92% of girls and 85% of boys said that they "learn a lot in school."

http://blogs.edweek.org/edweek/curriculum/2014/05/survey_finds_more_educators_th.html?cmp=ENL-EU-NEWS2
Northwest Evaluation Association survey: http://www.nwea.org/makeassessmentmatter

Those brave new tests

Ed Week cheerfully reports field-tests for Smarter Balanced and PARCC tests are going jus' fine:"Early Reports Suggest Few Field-Testing Snags"
[http://www.edweek.org/ew/articles/2014/05/06/30fieldtest_ep.h33.html?cmp=ENL-EU-NEWS1]
I posted this comment:
This "field-testing" seems to be mostly about technical aspects of implementation. I get the impression that analysis of the test items will be done later.  But there is no indication that the big questions will even be asked: Is all this testing, the largest investment in testing ever made, worth it? Will it improve student achievement?

"Pressing needs" in American Education

S. Krashen

Diane Ravitch recently observed that there has been a big hurry to implement the Common Core. In 2009, she urged the authors of the Common Core to field test it before implementation, advice they did not follow. Again in 2010, at the White House, she urged field testing, but officials "quickly dismissed the idea. They were in a hurry. They wanted Common Core to be rolled out as quickly as possible, without checking out how it works in real classrooms with real teachers and real children." (http://dianeravitch.net/2014/05/02/my-reply-to-alexander-nazaryan-of-newsweek/)

At about the same time, the US Department of Education, in their National Education Technology plan, was in a big hurry to introduce new technology into the schools. They argued that this must be done immediately, because of the "the pressing need to transform American education ...",  even if this means doing it imperfectly: Repairs can be done later: "... we do not have the luxury of time: We must act now and commit to fine-tuning and midcourse corrections as we go."  In other words, there will be no attempt to see there is any evidence that new technology could, in fact, "transform American education."  (Transforming Education: Learning Powered by Technology. US Department of Education, Office of Educational Technology.  http://www.ed.gov/technology/netp-2010 Quotes here are from the Excutive Summary.)

I think there are two "pressing needs" in education. One is to protect America's children from the effects of poverty. The other is to stop the admittedly unvalidated Common Core, delivered through  admittedly incompletely vetted technology.

Four years of math?

Published in the Washington Post, May 6, 2014, with the title: Maryland's new math requirement doesn't add up.
Maryland will now require all high school students to take four years of math ("Maryland to require math for all years of high school; universities also adjust rules," May 2).
The fourth year, moreover, must be "non-trivial," such as algebra 2, trigonometry, pre-calculus, calculus, statistics and college algebra. This is in order to prevent students  from "getting rusty" and not being prepared for college.

But not all students go to college. And of those who do, few select majors that require this much math and few jobs require this much math.  Michael Handel of Northeastern University in his "Profile of US Jobs" reported that only 22% of all workers use math beyond fractions, decimals and percentages, and of those who do, most only only simple algebra.

Requiring four years of math makes about as much sense as requiring four years of Latin.

FINAL SENTENCE NOT INCLUDED IN THE PUBLISHED VERSION (I should point out that I love math. I took AP calculus in high school, and advanced calculus and differential equations in college. But I took these courses as electives.)

Stephen Krashen
Professor Emeritus
University of Southern California

Original article: http://www.washingtonpost.com/local/education/heading-off-the-senior-slump-maryland-schools-officials-to-require-fourth-year-of-math/2014/05/02/7458986e-c9b4-11e3-a75e-463587891b57_story.html
This letter: http://tinyurl.com/lb6c3sh

Source:
Handel, M. 2010. What do people do at work? Available at www.northeastern.edu/socant/wp-content/.../STAMP_OECD2a_edit2.doc‎  See also: http://www.theatlantic.com/business/archive/2013/04/heres-how-little-math-americans-actually-use-at-work/275260/

The common core: A bad solution that aims at the wrong problem

 Sent to Newsweek, May 1, 2014.

Alexander Nazaryan ("Sorry, Louis C.K., but You’re Wrong About Common Core," May 1) says our schools need the increased "rigor" of the common core because they are so bad: "China, South Korea and Germany are leaving us in the chalk dust, most Americans can barely find America on the map ...".

Not so.  When researchers control for the effect of poverty, American students' international test scores are at the top of the world.  Our  overall scores are unspectacular (but not terrible) because we have so much child poverty, 24%, the second highest among all economically advanced countries.  

Poverty means poor diet, inadequate health care, and little or no access to books. All of these have devastating effects on school performance.  The best teaching has little effect when children are hungry, ill and have nothing to read.

The common core not only ignores the real problem, but also offers us a plan with no basis in the research: There is no research supporting "tough" standards, no research that justifies the bad math homework Louis C.K.'s children had to deal with. Also, studies show that increasing testing does not improve school achievement.

The common core is a bad solution that is aimed at the wrong problem. 

Stephen Krashen

Article appears at:

http://www.newsweek.com/sorry-louis-ck-youre-wrong-about-common-core-249313

Sources:

Levels of poverty: UNICEF Innocenti Research Centre 2012, ‘Measuring Child Poverty: New league tables of child poverty in the world’s rich countries’, Innocenti Report Card 10, UNICEF Innocenti Research Centre, Florence.

Control for poverty:

Payne, K. and Biddle, B. 1999. Poor school funding, child poverty, and mathematics achievement. Educational Researcher 28 (6): 4-13; Bracey, G. 2009. The Bracey Report on the Condition of Public Education. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/Bracey-Report. Berliner, D. 2011. The Context for Interpreting PISA Results in the USA: Negativism, Chauvinism, Misunderstanding, and the Potential to Distort the Educational Systems of Nations. In Pereyra, M., Kottoff, H-G., & Cowan, R. (Eds.). PISA under examination: Changing knowledge, changing tests, and changing schools. Amsterdam: Sense Publishers. Tienken, C. 2010. Common core state standards: I wonder? Kappa Delta Phi Record 47 (1): 14-17. Carnoy, M and Rothstein, R. 2013, What Do International Tests Really Show Us about U.S. Student Performance. Washington DC: Economic Policy Institute. 2012. http://www.epi.org/).

“Poverty means poor nutrition, inadequate health care, and lack of access to books”:

Berliner, D. 2009. Poverty and Potential:  Out-of-School Factors and School Success.  Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/poverty-and-potential;   Krashen, S. 1997. Bridging inequity with books. Educational Leadership  55(4): 18-22.

Increasing testing does not mean greater achievement:

Nichols, S., Glass, G., and Berliner, D. 2006. High-stakes testing and student achievement: Does accountability increase student learning? Education Policy Archives 14(1).  http://epaa.asu.edu/epaa/v14n1/. OECD. Tienken, C., 2011. Common core standards: An example of data-less decision-making. Journal of Scholarship and Practice. American Association of School Administrators [AASA], 7(4): 3-18. http://www.aasa.org/jsp.aspx.