Tag Archives: Testing; test scores

PARCC States Successfully Wrap Up Field Test

Leave a reply

PARCC States Successfully Wrap Up Field Test 

Washington, D.C. — More than one million students completed field tests last Friday, after nine weeks during which students participated in a trial run of the new tests developed by member states in the Partnership for Assessment of Readiness for College and Careers.  The tests are being developed in English language arts/ literacy and mathematics for grades 3-8 and high school.

“The completion of the PARCC field testing marks a key milestone in our development of an innovative assessment system aligned to the state-developed Common Core standards,” said Mitchell Chester, the Massachusetts education commissioner, who also chairs the PARCC governing board. “Overall, students had a very positive experience with the items and the new, online format. We’ll now use the feedback from students and educators to refine the test for next year’s operational testing.”

The goal of the field test was to “try out the nearly 10,000 test items developed by teachers, college and university faculty and others from the PARCC states,” according to Laura Slover, chief executive officer for PARCC, Inc., the non-profit managing the project for the states. She noted that at least 30 sets of eyes had seen each test item before it made it to the field test. Even now, she said, testing experts will review the results of the tests to make sure each item is valid. Some may be discarded or require revision based on the field testing.

The field test was also an opportunity for students and schools to test out their own technology and the test platform, learn about the test administration procedures, and provide feedback.

Student survey results revealed they generally like the online format of the test, and many found the test questions more engaging than their previous standardized tests. Test administrators also were positive, though they noted that the instructions for administrators were too long and could be clearer, something Slover said staff and state members would be working to address. Slover said additional detail from the surveys will be released in coming weeks. “We are compiling a list of lessons learned that we will share with states and the public this summer,” Slover said.

The tests, which will be given starting next school year, are part of a larger assessment system, which includes other voluntary components: interim assessments to measure whether students are on track, diagnostic assessments that allow teachers to check in on student performance throughout the year and adjust instruction to meet student needs, formative tasks, speaking and listening assessments, and professional learning resources for teachers.

Education chiefs from the participating states say the field test has been vitally important to developing the assessment system, and useful for schools as they prepare for next year.

“As we close the PARCC field tests in the District of Columbia, we look forward to going into the new school year knowing that our teachers and students have had a positive initial experience with our upcoming next-generation assessment,” said District of Columbia State Superintendent Jesús Aguirre. “The field test of this assessment, aligned with our common-core state standards, has provided great insight that will inform the work of creating an absolutely high-quality assessment that will support our efforts of ensuring that all of our students are ready for college, career, and life.  As a member of the PARCC consortium, DC has contributed greatly to the development of this assessment and we are thrilled to see it successfully complete this phase of its development.”

“A survey of Louisiana students validates our state’s strategy to use the online PARCC assessments,” said Louisiana State Superintendent John White.  “With nearly 80 percent of the students surveyed indicating they use a computer or tablet nearly every day, it only makes sense that we test them the same way.”

“We are moving ahead on an assessment system of unprecedented quality and breadth,” said New Mexico Secretary of Education Hanna Skandera. “States are going to be sharing an assessment next year, allowing us to compare results and learn from each other. Making sure our students are learning these critical skills is at the core of all our efforts and with the field test complete we’ve moved forward in a way no single state has done before. ” 

The preliminary breakdown of the roughly 790,000 computer-based tests completed by state is: Arizona, 69,236; Arkansas, 37,194; Colorado, 25,248; District of Columbia, 6,561; Illinois, 111,380; Louisiana, 39,938; Maryland, 73,220; Massachusetts, 69,980; Mississippi, 50,687; New Jersey, 100,616; New Mexico, 11,997; New York, 15,905; Ohio, 112,527; Rhode Island, 16,375; Tennessee, 44,768. In addition, just over 10,000 students in Hillsborough County, Florida, completed field tests and roughly 170,000 students took the performance-based portion of the test on paper and an estimated 100,000 or more took the end-of-year portion of the test on paper. Final state by state totals will not be available for several weeks, as the paper-based tests must be returned and processed.

The field tests will not yield scores for students or schools – rather they are used to determine if the questions function as intended. In addition, test developers and the state experts who oversee the work will review research studies this fall, which include a comparison of paper- vs. computer-based assessments; differences in student performance and behavior between tablets, desktops and laptops; whether the text-to-speech accommodation works as intended; and several other studies.

Work will begin soon on preparing for the spring 2015 test and on developing additional test items for future years, as the PARCC states plan to release 40 percent of test items each year.

Learn more about PARCC

College Applicants Sweat The SATs – perhaps They Shouldn’t

Leave a reply

An NPR Morning Edition article on a large study puts SAT and ACT standardized test scores in a real-world, practical context. Read the full article here. Here are the highlights:

  • SAT/ACT testing is not exactly a fair way to show skills.
  • what happens when you admit tens of thousands of students without looking at their SAT scores – the answer is – if they have good high school grades, they’re almost certainly going to be fine.
  • There was virtually no difference in grades and graduation rates between test “submitters” and “nonsubmitters.” Just 0.05 percent of a GPA point separated the students who submitted their scores to admissions offices and those who did not.
  • And college graduation rates for “nonsubmitters” were just 0.6 percent lower than those students who submitted their test scores.
  • High school grades matter — a lot.
  • Kids who had low or modest test scores, but good high school grades, did better in college than those with good scores but modest grades.
  • The study covered 123,000 students at 33 institutions over eight years; the conclusion: test-optional admissions improves diversity [and] does not undermine academic quality.
  • SAT/ACT testing may be discouraging students who have great potential for success [from applying to] a particular school,
  • The private test preparation market for the SAT and the ACT is a $2 billion-a-year industry in the U.S. Critics of the tests have long said the exams better reflect a family’s income and a student’s speed at test-taking than aptitude, competency or intelligence.

I Got Schooled – Practice #5 – More Time in School

Leave a reply

Here is the final I Got Schooled  practice #5 – More Time in School

M. Night Shyamalan (The Sixth Sense) has written a book (I Got Schooled)  describing how – and how not –  to close the education gap in the U.S. It should be very supportive in the current conversation and climate regarding what’s wrong with – and how to fix – New Mexico education.

For five years through his MNS Foundation, Shyamalan studied what is succeeding in closing the education gap — that depended only on practices inside the classroom itself and that were scalable.

He discovered closing the achievement gap depended on five practices and couldn’t be figured out by examining just any single practice by itself.

These five practices must be implemented together to have any substantive effect:

•            Effective teachers – dropping poor; hiring good; why it’s important; how to do it

•            Leadership – how it’s important; what it looks like; how to do it

•            Feedback – critical: frequency, consistency, teacher/principal usability

•            Smaller (high) schools –part of the “system” that turbocharges the other practices

•            More time in school – summers matter – children of low income and of color fall behind a month every summer; by the time they reach third grade they are so far behind it’s virtually impossible to catch up

Covered: successful schools, programs, clinical studies, and data and statistics, including: Knowledge is Power Program (KIPP), Uncommon Schools, Achievement First/Endeavor, FirstLine schools, North Star Academy, Arthur Ashe, Los Angeles Green Dot Public Schools, and more.

The study also found four popular, expensive practices contribute little to closing the education gap:

•            Small classroom sizes

•            Master’s programs and Ph.D.’s for the teachers

•            Paying teachers like doctors

•            Funding the schools at $20,000 per pupil

MORE TIME IN SCHOOL

  • More time in school is needed to overcome the deficiencies inherent in low-income, inner-city family environments
  • Upper-income families use an average of 2,153 words every hour; middle-income families use 1,251; welfare families use 616
  • Average words per year for upper-income are 11.2 million; middle-income families 6.5 million; welfare families 3.2 million words
  • By age four that is a gap of some 30 million words
  • The number of words a four-year-old can understand and speak relates directly to the number of words they have heard
  • By the time poorer kids reach third grade, they are already so far behind it is virtually impossible to catch up
  • Early intervention (preschool) with significant hours of exposure closes the gap
  • Consistency and good quality here are critical
  • Our natural tendencies are to blame failures of others on something flawed inside them, and while attributing our successes to merit and our failures on chance
  • American families that tend to obsess about pre-K education are the ones that need it least
  • The children who would most benefit from preschools are the least likely to be enrolled in them
  • Even then, generally, preschool effects disappear nearly completely by the third year after the program
  • Lower-income and African-American kids stay more or less even with upper-income, white suburban classmates – so long as school was in session
  • They fell behind a month or more every summer!
  • Summer matters far more than any has thought
  • Student test scores change powerfully by what happens between June and September
  • Lower-income student families cannot compensate for middle- and upper-income families’ ability to continue to enrich their children’s experiences over the summer
  • Current standard in most places is 180 days of school a year, each one between six and seven hours long
  • Before the Civil War, Philadelphia’s schools were in session more than 250 days a year; New York’s were open all year except for a two-week break in August
  • While more classroom hours is a serious part of closing the gap, five hundred additional low-quality hours taught by a teaching staff full of below-average instructors who are neither observed regularly by their principals nor given the quantitative and qualitative feedback they need is a waste
  • Keep kids in school longer during the year and you won’t need superheroes to close the gap. Most teachers can do the job just fine
  • And this practice is scalable

I Got Schooled – Practice #3 – Feedback

Leave a reply

So, here is Practice #3 – Feedback that is timely, consistent and teacher/principal-usable

M. Night Shyamalan (The Sixth Sense) has written a book (I Got Schooled) describing how, and how not, to close the education gap in the U.S. It should be very supportive in the current conversation and climate regarding what’s wrong with, and how to fix, New Mexico education.

For five years through his MNS Foundation, Shyamalan studied what is succeeding in closing the education gap — that depended only on practices inside the classroom itself and that were scalable.

He discovered closing the achievement gap depended on five practices and couldn’t be figured out by examining just any single practice by itself.

These five practices must be implemented together to have any substantive effect:

•            Effective teachers – dropping poor; hiring good; why it’s important; how to do it

•            Leadership – how it’s important; what it looks like; how to do it

•            Feedback – critical: frequency, consistency, teacher/principal usability

•            Smaller (high) schools –part of the “system” that turbocharges the other practices

•            More time in school – summers matter – children of low income and of color fall behind a month every summer; by the time they reach third grade they are so far behind it’s virtually impossible to catch up

Covered: successful schools, programs, clinical studies, and data and statistics, including: Knowledge is Power Program (KIPP), Uncommon Schools, Achievement First/Endeavor, FirstLine schools, North Star Academy, Arthur Ashe, Los Angeles Green Dot Public Schools, and more.

The study also found four popular, expensive practices contribute little to closing the education gap:

•            Small classroom sizes

•            Master’s programs and Ph.D.’s for the teachers

•            Paying teachers like doctors

•            Funding the schools at $20,000 per pupil

FEEDBACK

  • Some way is needed to improve the performance of teachers that are retained
  • To increase the value of existing teachers, some way to measure that performance will be needed
  • Value Added Method (VAM – in its ideal form) compares students’ actual progress to predicted progress, and assumes that differences are due to the instruction they received
  • While it is true, “kids can’t be reduced to a test score,” subjective grading is even worse; a flexible yardstick may not have created the achievement gap, but definitely perpetuated it
  • Measuring progress in literacy or math requires more than just observing reading or math level at age seven. It needs to include:
    • Family income
    • Parents’ educational background
    • Geographic location
    • Inner city versus suburbs
    • Past performance by other kids in same school
    • Past performance by the individual student
    • That is to say, some form of VAM
  • VAM works generally quite well so long as
    • good data are available, and
    • when it’s used to measure GROWTH rather than ACHIEVEMENT
    • important: these two measures are easily and frequently confused
  • The most effective schools are focused on GROWTH; based on their scores in the first grade, did our eighth-graders do better than predicted
  • An even more valuable way to use VAM data is for improving teachers and improving instruction; make the practice of teaching data-driven
  • Figure out the reasons for failure and success
  • A successful organization is one that goes through cycles of planning, doing, studying, and acting, over and over again, i.e., continuous Improvement
  • End-of-year tests tell a school whether it is doing its job, but not how to do it better
  • Value-added data adds value only when someone actually uses it
  • To work, data-based instruction can’t be just voluntary
  • Teachers and principals opting out of the feed-back loop system destroy the possibility of positive results
  • Data is collected everywhere, but no one has taught teachers and principals how to analyze, interpret, and use the data that are being collecting
  • Teachers at high-achieving schools receive some kind of feedback from classroom visits and student achievement between 16 and 13 times a semester
  • High-achieving schools make 4 interim assessments (tests) every semester in math and language arts
  • High-achieving schools have more strategies for using data to customize instruction for different students, i.e., different data-driven ways to create individualized lesson plans
  • The frequency of this feedback is critical
  • The effect size for employing effective feedback is twice as large as the effect size for reducing class size
  • The effect size for employing effective feedback is even larger for the estimated effect-size from replacing the lowest-performing 8 percent of teachers with average ones
  • Feedback and consistency are also important for empowering students
  • Feedback must be meticulous, frequent, and mandatory
  • Feedback must be produced in a form that is usable by teachers and principals
  • Keep realistic the number of teachers that every principal is responsible for; maybe schools need to be smaller
  • And this practice is scalable

The Heckman Equation – economic development and education

Leave a reply

This article in Businessweek January 20-26, 2013 is a wider public distribution of the concept, problem, and solution early child education speaks to that Governor Martinez spoke of in her State of the State address today. Heckman’s Equation brings hard, observable data into the realms of making public policy.

James Heckman, is a PhD economist at the University of Chicago and views education for 3- and 4-year olds as fix to a current serious market failure.  In terms of economic development, early child education is a hard-nosed investment that pays off in lower social welfare costs, lower teen pregnancy rates, decreased crime rates, and increased tax revenue, as opposed to  spending on prisons, health and adolescent special education remediation.

Through the Heckman Equation, he found that an initial investment in free instruction of $17,758 per child per year in 2006 dollars, yielded between $60 and $300  return on investment (ROI) in state and federal welfare monies, increased tax revenues, and, most significantly, savings in police and court costs. As an economist, he finds this is  a 7% to 10% annual ROI.

So, not only did this turn out to not be a cost, it more than paid for itself in annual ROI in improved human capital to business and society, he also found it consistently beat historical ROI on equity (the stock markets) of around 5.8%.

The data further show that the earlier a child gets help, the better the results continue to be through each stage of education. Literally, the greatest ROI or ‘bang for the buck’ exists in early intervention – early child education, and this effective ROI reduces throughout adolescence.

A big problem arises when kids ought to be in early education programs which most young parents can’t afford. Heckman says, “The accident of birth is a huge, huge imperfection in the (economic) market.”

The good news is, and data show, this is overcome-able through targeted early child education and it appears to be scalable.

As an aside, in a 2011 Heckman lecture, he points out that cognitive testing stressed by NCLB and Race to the Top, only speaks to one dimension of a multidimensional system that includes, at least: integrity, socio-emotional skills, character, collaboration, grit. Yet data strongly indicates these untested characteristics may account for the largest share of human development, effectiveness in life, and contribution to the economy and society. These are the “soft” skills that are so widely reported today to be both in short supply and increasing demand.

Beyond interesting, the data also show these “soft” skills even affect scores on cognitive tests!

I Got Schooled – Practice #2 – Right Balance of Leadership

Leave a reply

So, here is I Got Schooled practice #2 – Right Balance of Leadership.

Through his MNS Foundation, Shyamalan spent five years studying what is succeeding in closing the education gap that depended only on factors inside the classroom itself and that were scalable. He discovered closing the achievement gap can’t be figured out by examining just any single practice by itself; five things must be implemented together to have any substantive effect.

Shyamalan uncovers five factors present in schools he found were significantly closing the education gap for inner city children and children of color. He provides just tons of example schools, clinical studies, data and statistics, detailing each of these five factors.

These posts are highlights for each of these five factors for folks who might feel they are too busy to read the whole book … and strongly whet your appetite for doing so. The five factors are:

  • Effective teachers – dropping poor, hiring good
  • Right balance of leadership
  • Feedback that is timely and consistent
  • Smaller schools
  • More time in school

A large number of successful schools and programs are covered in the book, including: Knowledge is Power Program (KIPP), Uncommon Schools, Achievement First/Endeavor, FirstLine schools, North Star Academy, Arthur Ashe, Los Angeles Green Dot Public Schools, and more.

Factor #2 – LEADERSHIP, the right balance of

  • School leadership is second only to classroom teaching as an influence on learning
  • “Superman principals” are not scalable
  • There is not a single documented case of a school successfully turning around its achievement trajectory in the absence of strong leadership
  • First, leaders vary in large and consistent ways – the most successful teams work under the most successful bosses (McKinsey & Company study)
  • Second, the boss’s most important job isn’t communicating strategy or even hiring and firing – it is teaching skills that persist
  • Motivating a team of subordinates isn’t nearly as important as improving their skills
  • The difference between a good principal and a mediocre one is nearly as big as that between a great teacher and a poor teacher
  • A principal effects hundreds of students, instead of a few dozen
  • Principals are profoundly important
  • The most effective principals are also the ones engaging in the most “negative teacher selection,” i.e., firing the lemons
  • The most effective principals also improved the instructional success of existing teachers
  • Least effective principals seldom leave the system – they just move to different schools, further widening the achievement gap
  • Existing, thorough application processes do not insure a consistent level of success
  • Principals should be primarily responsible for the instruction of teachers
  • But the typical urban principal spends less that 20 percent of their workday on instruction
  • Principals in the top systems around the world spend 80 percent of the school day on improving instruction in the classroom (McKinsey & Company study)
  • Schools need an Operations Manager to handle food service, discipline, custodians, paperwork, to free principals to help teachers help students learn
  • Such an administrative management model is scalable
  • Brett Peiser founder of Uncommon Schools network, on leaders: “Great leaders know culture eats strategy for lunch.”
  • Culture is everything distinctive to a place; it’s every way you know you are in an uncommon school; it’s rigorous in involving and aligning everything and everyone
  • Culture is beyond advertising brochures, it must have enough force and buy-in to overwhelm its students’ environments outside the school
  • A culture of high expectations seems almost important enough to earn a place as one of the keys to closing the achievement gap
  • Someone has to make certain that all teachers understand and support a uniform school culture in their bones
  • Again, “Culture eats strategy for lunch.”
  • Consistency in students’ experience from class to class and grade to grade;  and quality control of this consistency is really important
  • Both unquestioned authority and humanity are equally important, i.e., caring and respect for students
  • There are techniques and behaviors that produce desirable results consistently, like Lemov’s Taxonomy, Teach Like A Champion, and they should be known and used
  • See also The Wooden Way, the legendary UCLA basketball coach’s system
  • And these practices are scalable

Hoping you are finding these interesting and perhaps useful.

Up next:  Factor #3 – Feedback

Tom

I Got Schooled – Practice #1 – Effective teachers

Leave a reply

I Got Schooled Practice #1 – Effective teachers – hiring good and dropping poor

Through his MNS Foundation, Shyamalan spent five years studying what is succeeding in closing the education gap that depended only on factors inside the classroom itself and that were scalable. He discovered closing the achievement gap can’t be figured out by examining just any single practice by itself; five things must be implemented together to have any substantive effect.

Shyamalan uncovers five factors present in schools he found were significantly closing the education gap for inner city children and children of color. He provides just tons of example schools, clinical studies, data and statistics, detailing each of these five factors.

These posts are highlights for each of these five factors for folks who might feel they are too busy to read the whole book … and strongly whet your appetite for doing so. The five factors are:

  • Effective teachers – dropping poor, hiring good
  • Right balance of leadership
  • Feedback that is timely and consistent
  • Smaller schools
  • More time in school

EFFECTIVE TEACHERS – dropping poor, hiring good

  • Are the engine that make schools run
  • Not all teachers perform equally (as do neither all salesmen, lawyers, accountants, basketball players, school principals, …)
  • Get the least effective teachers out of classrooms
  • You can discover who underperformers are
  • National studies find just 2% of teachers are rated unsatisfactory
  • Teachers in the middle, performance-wise, may be the most important factor for scaling success to 132,656 K-12 schools nationwide in 2010
  • One year with a great teacher is worth $20,000 in lifetime earnings for every student in that class
  • The effects of poor teachers in student learning outweighs the benefits provided by the good ones
  • Someone who barely graduated from Nowhere State is just as likely to be a great teacher as someone with a Summa Cum Laude from Harvard
  • Until you see them teach, you are picking blind; seeing them teach takes 2 to 3 years
  • After 2-3 years, tenure locks both good and bad teachers in place
  • Release procedures can take  300 days and cost $250,000, and so becomes too much trouble to take on
  • By replacing just 5 to 8% of the least effective teachers with average teachers, student overall achievement scores would rise to those of Canada or Finland
  • There is no evidence we can improve teachers’ performance by giving them bonuses or sending them to graduate school
  • Reliable tools must be used to determine teacher competence and effectiveness
  • Actual-versus-expected student progress can be a valid rating system (i.e., Value Added Method – VAM)
  • But … “Perfect is the enemy of the good.”  Voltaire
  • Hence the teacher evaluation problem: “Until someone comes up with a perfect tool for judging teachers, one that judges no teacher unfairly, there’s no point in using anything.”
  • If no one can be fired without a “fair” evaluation, and a fair evaluation needs to be error-free, then no one can be fired.
  • Multiple tests, as designed by the Measures of Effective Teaching project (MET),  are providing greater reliability and validity to teacher evaluation (see the MET January 2013 final report)
  • Good news: an inner city student having a great teacher for four years in a row will close the gap separating her from her suburban counterpart, without doing anything else
  • Bad news: three great teachers won’t make up for one poor one
  • The ratio of great teachers to poor teachers is nowhere near three-to-one
  • For schools on the wrong side of the achievement gap, these “roadblock” teachers effectively trap students
  • Eliminating roadblock teachers is a necessary first step toward closing America’s achievement gap, but it’s not sufficient in and of itself. “You can’t fire your way to excellence.”
  • Hiring good teachers and releasing bad ones is a scalable practice

Next up: The right balance of leadership