This annotated bibliography is reformatted and customized by the Center for Positive Practices. Some of the authors featured on this page include Judith Cramer, Karen Walker, Daniel F. McCaffrey, Robert Mislevy, Christopher McCarthy, Wayne E. Wright, Chang Pu, Daniel Koretz, Daniel M. Koretz, and Anne McGill-Franzen.
(2005). Alignment, High Stakes, and the Inflation of Test Scores, Yearbook of the National Society for the Study of Education. No one can dispute that tests should measure important content, and for many (but not all) purposes, tests should be aligned with curricular goals. Thus in many cases, alignment is clearly better than the alternative, and nothing that follows here argues otherwise. Unfortunately, however, this does not imply that alignment is sufficient protection against score inflation. Inflation does not require that a test assess unimportant material, and focusing the test on important material–through alignment–is not necessarily sufficient to prevent inflation. In this article, the author aims to explain the relationship between alignment and score inflation. The first sections clarify what is meant by inappropriate test preparation and provide a concrete, hypothetical example that illustrates a process by which scores become inflated. These are followed by a more complete discussion of the mechanisms of score inflation and their link to teachers' responses to high-stakes testing. A final section discusses some implications. [More] Descriptors: Test Use, High Stakes Tests, Accountability, Scoring
(2005). QAR: Enhancing Comprehension and Test Taking across Grades and Content Areas, Reading Teacher. Teachers today face increasing demands to ensure their students achieve high levels of literacy. They may feel overwhelmed by the challenges of teaching reading comprehension strategies that foster the integration, interpretation, critique, and evaluation of text ideas. The challenges are compounded because students of diverse backgrounds often enter classrooms reading far below grade level. In this article, the authors describe how Question-Answer Relationships (QARs) can provide a framework for comprehension instruction with the potential of closing the literacy achievement gap. QAR can serve as a starting point for addressing four problems of practice that stand in the way of moving all students to high levels of literacy: (1) The need for a shared language to make visible the invisible processes underlying reading and listening comprehension; (2) The need for a framework for organizing questioning activities and comprehension instruction within and across grades and school subjects; (3) The need for accessible and straightforward whole-school reform for literacy instruction for higher level thinking; and (4) The need to prepare students for high-stakes testing without undermining a focus on higher level thinking with text. [More] Descriptors: Test Wiseness, Elementary Secondary Education, Instructional Program Divisions, Content Area Reading
(2005). Teachers' Summative Practices and Assessment for Learning — Tensions and Synergies, Curriculum Journal. This article concerns the use of assessment for learning (formative assessment) and assessment of learning (summative assessment), and how one can affect the other in either positive or negative ways. It makes a case for greater use of teachers' judgements in summative assessment, the reasons for this being found in the research that is reviewed in the first sections of the article. This research, concerning the impact of summative assessment, particularly high-stakes testing and examinations, on students' motivation for learning and on teachers and the curriculum, reveals some seriously detrimental effects. Suggestions for changes that would reduce the negative effects include making greater use of teachers' summative assessment. However, this raises other issues, about the reliability and validity of teachers' assessment. Research on ways of improving the dependability of teachers' summative assessment suggests actions that would equally support more effective use of assessment to help learning. The later sections of the article address the issues and opportunities relating to the possibility of assessment that serves both formative and summative purposes, with examples of what this means in practice, leading to the conclusion that the distinction between formative and summative purposes of assessment should be maintained, while assessment systems should be planned and implemented to enable evidence of students' ongoing learning to be used for both purposes. [More] Descriptors: Motivation, High Stakes Tests, Formative Evaluation, Student Evaluation
(2005). Sowing the Seeds: Moving Curriculum and School Culture towards Education for Sustainable Development, Current Issues in Comparative Education. 2005 marks the beginning of the UN Decade for Education for Sustainable Development (ESD), but awareness of this event is practically nonexistent in the New York City public high school where the author teaches English. A concerted effort to teach for–or even about–sustainability in mathematics, science, business and humanities classes is more remote still. The reasons are as various as they are predictable: "poverty, ignorance, disease and violence"–the real "axis of evil"–and high-stakes testing thrown in for good measure. In this article, the author discusses two key issues specific to education that may help teachers deal more effectively with the obstacles keeping ESD from being better known and universally embraced in the schools: 1) curriculum; and 2) school culture. The important points to retain are that ESD requires a fully integrated approach to curriculum development; that small schools are by virtue of their scale better suited to do ESD; and that ideally the physical infrastructure should reflect the values and technologies which make ESD possible. [More] Descriptors: School Culture, Sustainable Development, Humanities, High Schools
(2005). Academic Achievement of English Language Learners in Post Proposition 203 Arizona, Education Policy Research Unit. This report reveals the problems with claims made by Arizona state public education officials that English Language Learners (ELLs) are thriving under English-only instruction. The No Child Left Behind Act of 2001 (NCLB) and the state's accountability system, Arizona LEARNS, require all students, including ELLs, to participate in statewide high-stakes testing. Test scores are the main measure of student achievement under these systems, and labels based on those scores are given to each school (i.e. Highly Performing, Underperforming, etc.). The state education administration's interpretation and strict enforcement of Proposition 203 has ensured that nearly all ELL students in grades K-3 are instructed through the English-only Sheltered English Immersion (SEI) model. They claim that SEI has led to better test scores and increased achievement among ELLs, using as evidence improved test scores and the decrease in the number of schools labeled as "Underperforming." However, analyses of test data for students in grades two through five and changes in the state accountability system revealed the contrary; they exposed serious achievement gaps between ELLs and their counterparts, and proved that positive looking improvements in school accountability labels mask test-score decline in a large number of elementary schools. [More] Descriptors: State Legislation, Educational Legislation, Language of Instruction, Immersion Programs
(2005). Raising Standards in American Schools: The Case of "No Child Left behind", Journal of Education Policy. In January 2002, President George W. Bush signed into law what is arguably the most important piece of US educational legislation for the past 35 years. For the first time, Public Law 107?110 links high stakes testing with strict accountability measures designed to ensure that, at least in schools that receive government funding, no child is left behind. The appropriately named No Child Left Behind Act (NCLB) links government funding to strict improvement policies for America's public schools. Much of what is undertaken in NCLB is praiseworthy, the Act is essentially equitable for it ensures that schools pay due regard to the progress of those sections of the school population who have traditionally done less well in school, in particular, students from economically disadvantaged homes, as well as those from ethnic minority backgrounds and those who have limited proficiency to speak English. However, this seemingly salutatory aspect of the Act is also the one that has raised the most objections. This paper describes the key features of this important piece of legislation before outlining why it is that a seemingly equitable Act has produced so much consternation in US education circles. Through an exploration of school level data for the state of New Jersey, the paper considers the extent to which these concerns have been justified during the early days of No Child Left Behind . [More] Descriptors: Federal Legislation, Economically Disadvantaged, English, Accountability
(2005). Overweight and Obesity. Research Brief, Education Partnerships, Inc.. In this world of receiving immediate gratification, being over scheduled, and having access to a myriad of technology, poor nutrition and lack of daily physical activity are two of the results. "Obesity is a silent epidemic," former U.S. Surgeon General David Satcher stated in 2002 (Healthy schools summit weighs in on obesity). Due to the demands of high stakes testing, many schools throughout the United States have placed an increased emphasis on reading and math skills, to the detriment of other academic areas such as health, nutrition, and physical education. One survey of high school students' eating habits reported that in 2003, 67% exceeded dietary guidelines for fat intake, 72% exceeded saturated fat intake, while only 22% ate the U.S.D.A.'s recommendation of five fruits and vegetables daily (Nutrition). A study from the Center for Disease Control reported that one in every three children born in 2000 will develop diabetes (Nutrition services). By the 2006-2007 school year, according to the Reauthorized Child Nutrition Act, each school must have a wellness plan in action. This paper offers tips on what schools can do to combat the increasing number of students who are overweight or obese. [More] Descriptors: Health Promotion, Physical Education, Physical Activities, Academic Achievement
(2005). Career and Tech Prep. Research Brief, Education Partnerships, Inc.. As was the case when career education programs were begun during the end of the 19th century, business still want students trained to be successful in the world of work. Initially career type programs were established to provide a venue for students who were most likely not college bound, yet would leave high school with viable skills to meet the demands of a specific trade. Throughout its history, this population of students has been referred to as "at risk" and more recently, as the "neglected majority." Historically, these programs included a high number of specific skills type classes, with less emphasis on formal academics, which were believed to be needed only by those who would be going on to college. Due to the advent of the No Child Left Behind Act and its emphasis on high stakes testing, funding for specialty programs, such as Vocational Education, have been drastically reduced or eliminated. There has been an on-going debate about the purpose of vocational education. Is it to prepare students for a specific job or to prepare them for the world of work with a strong background in academics? Currently, students in a majority of these programs are required to take high level academic courses, often more demanding than those who are in the more traditional academic programs. [More] Descriptors: Tech Prep, Vocational Education, Courses, Career Education
(2005). Intuitive Test Theory, Phi Delta Kappan. Many of us have an intuitive understanding of physics that works surprisingly well to guide everyday action, but we would not attempt to send a rocket to the moon with it. Unfortunately, the authors argue, our policy makers are not as cautious when it comes to basing our school accountability system on intuitive test theory. Intuitive physics works well enough for playing catch with your dog or for building a birdhouse. But it doesn't work for constructing a bridge or shooting a rocket to the moon. One aspect of becoming an expert in physics is learning more sophisticated ways of thinking, but another is knowing when you need to use them, and yet another is recognizing when they fail. (Science is also about telling stories, but they are stories that submit to reality checks.) To Americans who go to school or hold jobs in the 21st century, taking tests is an experience nearly as familiar as pushing boxes or watching things fall. So we need to tell stories about tests — their purposes, their construction, our performances on them — and we need concepts to do so. While intuitive test theory is sufficient for classroom testing and for the quizzes in Seventeen magazine, it gets you into trouble when you want to evaluate performance on simulation-based activities, run a high- stakes testing program, or measure change in populations using an achievement survey like NAEP. Descriptors: Testing Programs, Astronomy, Accountability, Physics
(2006). Contamination of Current Accountability Systems, Phi Delta Kappan. As public employees, educators should expect to be held accountable for their use of public funds. Nonetheless, the various state governments and now the U.S. Department of Education have implemented high-stakes achievement testing as the nearly singular approach to accountability. While these accountability efforts vary in a number of ways, virtually all are similarly flawed. The authors feel that a number of issues related to current accountability schemes need to be addressed. In this article, they argue that, in all the accountability systems, four sources of contamination–summer reading loss, retention in grade, test preparation, and testing accommodations–serve to undermine the reliability of estimates of student reading achievement, a common measure of school effectiveness. Unless accountability policies are substantially revised and current practices modified, estimates of school effectiveness will remain unreliable, and the public and policy makers will continue to be misled. [More] Descriptors: Accountability, Reading Achievement, High Stakes Tests, Achievement Gains
(2005). Women, Webquests, and Controversial Issues in the Social Studies, Social Education. America has been called "the argument culture," but you would not know it from many social studies classrooms. Despite a longstanding tradition in social studies of teaching controversial issues, all too few of today's classrooms accommodate this kind of intellectual activity. Perhaps it is the pressure of high stakes testing, or the emphasis on teaching history–often done in a transmission, or didactic mode–or a reluctance to bring up polarizing topics in the currently politicized school climate. Whatever the reason, young people today may graduate from high school having had little chance to debate or otherwise discuss contemporary controversial issues. This is a missed opportunity on more than one account. Research has shown such academic endeavors to be highly motivating for adolescents in making judgments and forming their own opinions about adult topics. This article considers WebQuests, included in the Women of the World course since 2002. WebQuests are well suited to teaching controversial issues, particularly contemporary controversial issues, for which data and other timely resources are likely to be published on the internet. WebQuests can bring together important elements of some much touted approaches to teaching and learning, including critical thinking, cooperative learning, authentic assessment, and technology integration. Descriptors: Controversial Issues (Course Content), Teaching Methods, Learning Activities, Critical Thinking
(2005). Using IRT DIF Methods to Evaluate the Validity of Score Gains. CSE Technical Report 660, National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Given current high-stakes uses of tests, one of the most pressing and difficult problems confronting the field of measurement is to develop better methods for distinguishing between meaningful gains in performance and score inflation. This study explores the potential usefulness of adapting differential item functioning (DIF) techniques for this purpose. It distinguishes between reactive and nonreactive changes in DIF over time and relates these to the framework for validating scores under high-stakes conditions offered by Koretz, McCaffrey, and Hamilton (2001). It contrasts score-anchored and item-anchored approaches to DIF in terms of their potential for this purpose. It explored changes in the distribution of DIF in the NAEP eighth-grade mathematics assessment between 1990 and 2000 in five low-gain and five high-gain states, in each case treating all other participating states as the reference group. It used the score-anchored method of DIF analysis implemented in BILOG-MG (Bock & Zimowski, 2003b), which allows only item difficulties to vary across groups. This exploration indicated that the approach has potential but confronts several substantial difficulties. Further exploration using data from high-stakes testing programs is recommended. [More] Descriptors: Item Response Theory, High Stakes Tests, Achievement Gains, Scores
(2006). Understanding Teacher Stress in an Age of Accountability. Research on Stress and Coping in Education, IAP – Information Age Publishing, Inc.. School districts today face increasing calls for accountability during a time when budgets are stretched and students' needs have become increasingly complex. The teacher's responsibility is to educate younger people, but now more than ever, teachers face demands on a variety of fronts. In addition to teaching academic content, schools are responsible for students' performance on state-wide tests. They are also asked to play an increasingly larger role in children's well-being, including their nutritional needs and social and emotional welfare. Teachers have shown themselves to be more than capable of taking up such challenges, but what price is paid for the increasing demands individuals are placing on schools? Understanding Teacher Stress in an Age of Accountability is about the nature of teachers stress and the resources they can employ to cope with it. Accountability is a two-way street and the authors in this volume suggest remedies for reducing teacher stress and in all likelihood increasing student learning–greater administrative support, more and better instructional materials, specialized resources targeted at demanding children, parental support, and professional recognition. Readers will discover that lack of funding, low pay, concerns about academic performance and student misbehavior, and increased public and governmental scrutiny are not exclusive to the United States. In this volume, the third in a series on Research on Stress and Coping in Education, authors from Australia, Turkey, Malaysia, and the Netherlands sound the same alarms, post the same warnings, and draw similarly disturbing conclusions. Contents of this book are: (1) Forward (Mimi Wolverton); (2) The Stress of Accountability: Teachers as Policy Brokers in a Poverty School (P. Taylor Webb); (3) Weighing the Risks with the Rewards: Implementing Student-centered Pedagogy within High Stakes Testing (Michael M. Grant and Janette R. Hill); (4) Teacher Stress and High Stakes Testing: How Using One Measure of Academic Success Leads to Multiple Teacher Stressors (Sandra Mathison and Melissa Freeman); (5) Shared Needs: Teachers Helping Students with Learning Disabilities to Cope More Effectively (Nola Firth, Erica Frydenberg and Daryl Greaves); (6) Multicultural Competencies and Teacher Stress: Implications for Teacher Preparation, Practice, and Retention (Sylvia C. Nassar-McMillan, Meagan Karvonen, and Cheryl Young); (7) Teacher Stress and Classroom Structural Characteristics in Preschool Settings (Richard Lambert, Jeni Kusherman, Megan O'Donnell, and Christopher McCarthy); (8) Stress in the Student-Teacher Relationship in Dutch Schools: A Replication Study of Greene, Abidin, & Kmetz's Index of Teaching Stress (ITS) (H.A. Everaert & J.C. van der Wolf); (9) Sources of Teacher Demotivation (Zeynep Kltepe); (10) Teachers' Job Stress and Human Resource Development: The Malyasian Experience (Reynaldo Segumpan and Fazli Bahari); (11) Relationship of Teachers' Preventive Coping Resources to Burnout Symptoms (Christopher McCarthy, Debra Kissen, Lauren Yadley, Teri Wood, and Richard Lambert); (12) The Relationship between Burnout and Stress Among Special Educators (Stacey L. Edmonson); and (13) Helping Teachers Balance Demands and Resources in an Era of Accountability (Christopher McCarthy and Richard Lambert). [More] Descriptors: Professional Recognition, Teacher Burnout, Learning Disabilities, Testing
(2005). Academic Achievement of English Language Learners in Post Proposition 203 Arizona. Executive Summary, Language Policy Research Unit. This report reveals the problems with claims made by Arizona state public education officials that English Language Learners (ELLs) are thriving under English-only instruction. The No Child Left Behind Act of 2001 (NCLB) and the state's accountability system, Arizona LEARNS, require all students, including ELLs, to participate in statewide high-stakes testing. Test scores are the main measure of student achievement under these systems, and labels based on those scores are given to each school (i.e. Highly Performing, Underperforming, etc.). The state education administration's interpretation and strict enforcement of Proposition 203 has ensured that nearly all ELL students in grades K-3 are instructed through the English-only Sheltered English Immersion (SEI) model. They claim that SEI has led to better test scores and increased achievement among ELLs, using as evidence improved test scores and the decrease in the number of schools labeled as "Underperforming." However, analyses of test data for students in grades two through five and changes in the state accountability system revealed the contrary; they exposed serious achievement gaps between ELLs and their counterparts, and proved that positive looking improvements in school accountability labels mask test-score decline in a large number of elementary schools. [For full report, see ED508517.] [More] Descriptors: State Legislation, Educational Legislation, Language of Instruction, Immersion Programs
(2003). Testing and Student Engagement with Literature in Urban Classrooms: A Multi-Layered Perspective, Research in the Teaching of English. Though high-stakes testing currently dominates educational policy, few studies examine the consequences of such testing for the teaching and learning of literature in secondary English classrooms. This study takes a multi-layered approach to specify how a high-stakes exam positioned students as readers of literary texts. Drawing on both critical discourse analysis and socio-cultural literacy studies, the study traces the discursive processes through which the exam entered into tasks and talk about a novel in two classrooms to marginalize and contain students' substantive engagement with the issues of race and racism central to the novel. The study points to the need for close analyses of testing texts and of how they influence the ways in which teachers and students read and respond to literature. [More] Descriptors: Educational Testing, High Stakes Tests, Discourse Analysis, Educational Policy