Carnegie, ETS Team Up to Develop Competency-Based Assessments
Amid a tough market for testing, two leading players in education are collaborating on a project to measure academic skills differently.
Two major players in K–12 education launched a joint effort last month to develop new assessments that could help shift schools’ focus away from traditional “seat time” requirements and toward more accurate measures of mastery over academic content.
The new tests, to be created by the Educational Testing Service and the Carnegie Foundation for the Advancement of Teaching, are meant to usher in competency-based forms of schooling that would allow students to proceed through academic material at their own pace. Leaders of both organizations hope they will also capture a broader array of non-cognitive qualities, like teamwork and relatability, that are highly prized in the modern workforce but undetectable through conventional academic metrics like grade point average or school attendance.
The adoption of more personalized instruction and assessment has faced a key obstacle in the form of the Carnegie Unit, the namesake foundation’s strict definition of annual credit hours that students must accrue to demonstrate their grasp of material. (The calculation essentially breaks down to one hour of seat time per day, per subject, for 24 weeks.) Though largely unknown outside the education world, many high schools and universities have based their academic requirements on the Carnegie Unit for over a century.
But Timothy Knowles, the foundation’s president, said that while the Carnegie Unit had served a useful purpose at one point, new discoveries in neuroscience and cognitive psychology have proven that pupils learn different subjects at highly variable rates. What’s more, he added, the capacity now exists to test for valuable qualities that were previously invisible to admissions officers and employers.
“We’re in a position to do something that we hadn’t before,” Knowles said. “Unlike 20 years ago, we can actually reliably measure the skills that we know are predictive of success in postsecondary education and work.”
Competency-based learning and assessment has long been theorized as a preferable alternative to existing educational models, which critics describe as too standardized to deliver instruction to individual students with vastly divergent levels of academic preparation. Instead, they allege, the status quo came to reflect the production processes of 20th-century industry, with students replacing widgets as the product. In a recent interview, Knowles himself telegraphed his desire to phase out the Carnegie Unit, calling time a “crude” metric to determine educational attainment.
With a range of philanthropic and education-focused advocates backing the movement, virtually every state has promoted some version of competency-based policies. Those efforts hit a high-water mark in Maine, where high school graduation requirements were refigured over the last decade to emphasize proficiency on subject material. But disputes over the definition of proficiency and teachers’ differing grading standards led many to question the new approach, with legislators later backing away from the competency-based model.
Similarly rocky transitions were seen in Vermont and New Hampshire, which attempted similar shifts. The central puzzle facing critics of the current model (i.e., calendar-centered requirements and standardized assessment) is what will come to supplant it.
Scott Marion, president of the National Center for the Improvement of Educational Assessment, said that the challenge in executing the hoped-for switch to competency-based learning lay in designing realistic measures of achievement to replace existing tests. To deliver on advocates’ promises, he observed, such measures would need to be both tailored to individual students and academically credible.
“Competency-based assessment is not for the faint of heart,” Marion said. “It’s being done quite poorly in a lot of places. So if ETS and Carnegie can bring a little more rigor to it, it might be good.”
With interest in competency-based approaches growing, more players have leapt into the field, with the best-known among them a national consortium that developed a “mastery transcript.” The project has gained adherence among high schools over the last year.
The ETS-Carnegie proposal is also emerging at a time when traditional high school admissions exams, such as the SAT and ACT, have lost significant market share. Both the aftereffects of the pandemic and concerns about inequitable outcomes from standardized testing have led thousands of colleges and universities to go test-optional in the last few school years. With those leading indicators of secondary achievement potentially passing from the scene, demand is expected to rise for measures that could take their place.
ETS, which administers the widely used GRE, PRAXIS, and TOEIC tests, has itself announced multiple rounds of layoffs during and after the pandemic.
Perhaps the biggest question hanging over the newly announced partnership is the proposed measurement of not just cognitive and behavioral skills — including everything from comprehension of math content to teamwork and leadership — but so-called “affective” skills as well. As described by ETS head Amit Sevak at the educational technology conference ASU-GSV, such skills could include something like emotional intelligence, or the ability to successfully convey sincerity and empathy to others. Just how those kinds of competencies can be conveyed to students, let alone measured by third parties, is debatable even to backers of competency-based instruction.
Michael Horn, a cofounder of Harvard’s Clayton Christensen Institute for Disruptive Education, said he would be watching the development of such measures carefully.
“This part, from my reading of the literature on assessment, is both unproven and underdeveloped. So the how is going to be very important,” Horn said. “I’m going to be very curious to see what the investments look like as they go forward, and I hope they don’t overpromise.”
While no concrete timeline has been released for the conception of the new suite of assessments, Carnegie and ETS are reportedly aiming to conduct a multi-state pilot that could begin as early as next year. In an interview, Sevak said he envisioned students being able to access a digital “transcript” detailing their ongoing growth in areas like collaboration and creativity. Real-time data could build their awareness of their comparative strengths and weaknesses, he added.
“That more holistic approach is in contrast to much of the assessments in K–12 and higher education, which are really cognitive-driven and tied to logic and reason,” Sevak said. We’re looking at a more holistic approach that is more tied to the future of work.”
