On August 29, 2011, the SMARTER Balanced Assessment Consortium (SBAC) released its draft of ?Content Specifications with Content Mapping for the Summative assessment of the Common Core State Standards for Mathematics? (henceforth, Draft).? Earlier this month, we posted our feedback to SBAC on their ELA content specifications. (See here.) Below is the feedback from W. Stephen Wilson, professor of mathematics and education at Johns Hopkins University and lead math analyst for our 2010 report, The State of State Standards?and the Common Core?in 2010. Wilson has participated in numerous projects on standards, curricula, and textbooks. He received his Ph.D. in mathematics from M.I.T. and has published over sixty mathematics research papers in the field of algebraic topology.
Review of the SBAC Math Content Draft
Overview [pullquote]The conceptualization of mathematical understanding on which SBAC will base its assessments is deeply flawed.? The consortium focuses on the Mathematical Practices of the Common Core State Standards for Mathematics (CCSS-M) at the expense of content, and they outline plans to assess communication skills that have nothing to do with mathematical understanding. [/pullquote]
The conceptualization of mathematical understanding on which SBAC will base its assessments is deeply flawed.? The consortium focuses on the Mathematical Practices of the Common Core State Standards for Mathematics (CCSS-M) at the expense of content, and they outline plans to assess communication skills that have nothing to do with mathematical understanding.? In addition, they will be unable to provide student-level data for critical procedural skills, instead providing data only at the classroom or school level. And, unclear on the concept of a summative assessment, the content assessed on end-of-year assessments will generally be drawn from standards from previous years. In the end, the Draft supplies little guidance for curriculum developers or for the assessment of mathematical content knowledge.
Analysis
The very hierarchy of the organization of the bulk of the Draft demonstrates the consortium's emphasis on the Mathematical Practices rather than content.? The five levels in the hierarchy are, from top to bottom, Claims, Rationale, Evidence, Targets, and Priorities.? The Claims lay out, very broadly, what the consortium plans to assess and the Rationale level deals entirely with Mathematical Practices. ?Actual content finally shows up at the fourth level in Targets, and the all-important setting of content Priorities is done as an afterthought in an appendix. We will address each level in turn.
From the Draft: ?The contents of this document describe the extent of the Consortium?s current development to specify critically important claims about student learning that are derived from the Common Core State Standards. When finalized, these claims will serve as the basis for the Consortium?s system of summative and interim assessments and its formative assessment support for teachers.?
The ?claims? referred to are:
Claim #1 - Students can explain and apply mathematical concepts and carry out mathematical procedures with precision and fluency.Claim #2 - Students can frame and solve a range of complex problems in pure and applied mathematics.
Claim #3 - Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others.
Claim #4 - Students can analyze complex, real-world scenarios and can use mathematical models to interpret and solve problems.
With difficulty, we will ignore the odd, guidance-challenged, approach to assessments through ?claims.? Instead, we focus on Claims as the top level in the structure of the Draft and immediately see discomforting directions the assessments are expected to take.? Essential ingredients in both Claim # 1 and Claim # 3 are communication skills.? Explicitly (from page 10), ?a major requirement for a successful performance in mathematics as outlined in the CCSSM is a high level of verbal and written communication skills.?? Communication skills are NOT critical to mathematical understanding and insisting on them in order to judge mathematical understanding is misguided at best.? These top-level claims already demonstrate a failure to properly interpret mathematical understanding.
The next level, Rationale, is a rationale for each Claim.? These rationales consist almost entirely of quotes from the Mathematical Practices of the CCSS-M, each Claim getting a page or two.
Unfortunately, while the Mathematical Practices in CCSS-M are probably the best process standards there are, they shouldn't be there.? Mathematical Practices, or what was usually called ?process? standards in most states, do little more than describe how someone pretty good at mathematics seems to approach mathematics problems.? As stand alone standards, they are neither teachable nor testable. Mathematics is about solving problems, and anyone who can solve a complex multi-step problem using mathematics automatically demonstrates their skill with the Mathematical Practices, (whether they can communicate well or not).
SBAC demonstrates the inappropriate use of the Mathematical Practices by making them a central theme of this Draft.? They acknowledge ?The Common Core State Standards for mathematics require that mathematical content and mathematical practices be connected (CCSSM, p. 8).?? However, they get the connection backwards.? Problem solving leads to mathematical practices, not mathematical practices leads to problem solving.
The next level, Evidence, is about ?what sufficient evidence looks like for Claim # __.?? Since Claim # 1 is about procedures, it is disturbing that they have made it explicit that they do not expect to be able to give a sub-score for individual students for Claim # 1, but possibly only for classrooms or schools.? This would miss giving essential feedback on foundational skills to teachers.
Claim # 2 is about non-routine problems, i.e. problems ?the student will not have been taught a closely similar problem.?? ?It is recognized that such tasks will be new to many students.?? If CCSS-M is followed, and students work many complex problems, they should not be new to students when they are tested.? Non-routine problems cease to be non-routine when students have worked enough problems, witness the education students get in Singapore and Shanghai, where students easily perform well on PISA, a collection of so-called non-routine problems.
Worse, with Claim # 2, ?Because of the high strategic demand that substantial non-routine tasks present, the technical demand will be lower ? normally met by content first taught in earlier grades.? (page 33) In other words, the content assessed for this Claim each year will be content that students should have mastered in earlier grades (likewise for Claims # 3 and # 4).? One may well ask what the point of teaching grade level content is, if it is the previous year's content that will be assessed?? This will be extremely problematic for curriculum developers and teachers.
Claim # 3 (and Claim # 4) will have problems that are difficult to score in a consistent, fair manner because they rely heavily on communication skills, and, as such, will be hard to connect to mathematical understanding.
Finally, at the Targets level of the Draft, mathematical content shows up in ?summative assessment targets,? at this stage of production only for grades 4, 8 and 11.
At the end, in Appendix A, the most important aspect connecting the standards to curriculum and assessments is finally addressed; the setting of priorities indicating how much ?time and focus on an assessment? each cluster from CCSS-M should get.? This is attributed to only one of the contributors to the Draft, so it is unclear if it is a minority report or truly embraced by SBAC, at the end, finally, in an appendix.? Even if embraced, it has been smothered by the Mathematical Practices, and it is not clear that it will survive appropriately to the assessments. [pullquote]Even if embraced, the sound priorities presented in the appendix have been smothered by the Mathematical Practices, and it is not clear that it will survive appropriately to the assessments.[/pullquote]
Now that something seriously relevant has been brought into the picture, it is necessary to see how well these priorities are set up.? As it turns out, the setting of priorities is almost identical (but not completely) to the priorities set in PARCC's framework, and, on the whole, are reasonable. Unfortunately, it sits as an afterthought and doesn't reassure that, in the end, the assessment will be focused where it should be.
Ultimately, the actual assessments will tell us all what SBAC thinks is important.? This Draft does not give good guidance for curriculum developers because content is an afterthought. It appears that the assessments will focus on communication skills and Mathematical Practices over content knowledge.? As such, there is little to be optimistic about.
--W. Stephen Wilson