Review of Huff, Steinberg, and Thomas; The Promises and Challenges of Implementing Evidence-Centered Design in Large-Scale Assessment."

Citation:  Huff, K., Thomas, M., Steinberg, L., (2010). The Promises andChallenges of Implementing Evidence-Centered Design in Large-Scale Assessment. Applied Measurement In Education, 23, 310-324.

Summary:
This article examines the benefits and challenges of using evidence centered design
(ECD) to design courses and exams that accurately reflect the students understanding and competencies in Advanced Placement (AP) courses.  The two major challenges
facing the AP program are understanding how and what students learn and being able to compare scores accurately across the years. Evidence centered design is beneficial for the design of the AP program because it provides clarity for alignment, definition for evidence of claims, and detailed  design guidelines that will serve as a basis for comparability. 
The AP program covers a vast network of teachers and schools. To achieve understanding of how and what students learn,  the AP program will need to be sure there is alignment between the teaching, learning, and what is measured on the exam. Using the ECD process, the AP program can define the claims and what evidence will be required to support those claims.  This will help teachers to design their curriculum and instruction to help their students achieve the deep conceptual knowledge. 

  The article shows how the ECD method could be used to improve performance and consistency of  AP courses. The benefits of the program are: 1) Alignment of course and exam. 2) Explication of deep conceptual understanding and complex reasoning.3) Foundation for comparable scores.4) Structure for valid interpretation of scores. While the use of  ECD in this project does not contain any breakthrough information, it does show how complex the use of ECD can be.  The problem with using the ECD approach in a project of this magnitude is it requires a large amount of resources, time,  and infrastructure to be put in place for it to be consistent and successful.  In order to attempt to develop this program, it would require a whole new way of thinking. If this type of assessment could be developed it could impact the value of this type of assessment.  It would give us a greater ability to understand what is being measured. This type of ECD use would be easier to develop while developing the assessment. Evidence-centered assessment design posits explicit evidentiary standards at the beginning of design rather than post hoc (Huff, Thomas, and Steinberg, 2010).  Conclusion: This article was well written and explored a valuable use of ECD.  It showed how ECD could be used in a large scale assessments to bring consistency and clarity to the evaluation process of the AP program.  Unfortunately, it also pointed out that undertaking a large scale program like this can be very complicated, time consuming, and expensive.  It shows how ECD could be implemented into the educational assessments but points out that it would be more efficient to develop ECD when developing the assessment instead of trying to retrofit an existing large scale assessment.  I do believe that if the resources were available that the AP program could be improved by the use of ECD.  It would be interesting if they could at least do on AP test with the ECD process to evaluate if the investment is worthwhile.