We are pleased to announce that the SCILLSS project has been
recognized by the American Education Research Association (AERA) as first place
winner of the AERA 2020 Division H’s Outstanding
Publications Competition in Category 4: Assessment & Accountability!
The award is a testament to the significant contributions that the SCILLSS states
and partners have made in developing a pair of assessment system evaluation
protocols and an accompanying digital workbook. For a complete list of SCILLSS
resources, visit the project website at www.scillsspartners.org.
SCILLSS (Strengthening Claims-based Interpretations and
Uses of Local and Large-scale Science Assessment Scores) is a four-year
project that brings together the Nebraska Department of Education (NDE), the
Montana Office of Public Instruction, the Wyoming Department of Education, and
a team of researchers and educational measurement specialists including edCount,
LLC, SRI International, ACS Ventures, and an elite technical advisory panel of
science and assessment experts. SCILLSS is funded by an Enhanced Assessment
Grant that the US Department of Education awarded to the NDE in April of 2017.
Its purpose is to bridge the gap between state-, district-, school- and
classroom-level interpretation and implementation of recent shifts to
three-dimensional science standards and assessments.
SCILLSS partners, led by Drs. Ellen Forte of edCount and Andrew Wiley of ACS Ventures, created the set of award-winning publications that are designed to (a) provide state and local educators with a grounding in the principles for high quality assessment that are critical to the appropriate selection, development, and use of assessments in educational settings, and (b) offer a framework for making decisions about whether to develop or adopt tests and for evaluating tests currently in use. These publications include:
- A Digital
Workbook on Educational Assessment Design and Evaluation
Through five interactive chapters, this workbook
provides an in-depth exploration of the purposes and uses of educational
assessments and the concept of validity and presents a series of key validity
questions that guide the evaluation of assessment scores for their intended
purposes and uses.
Rigor in State Assessment Systems: A Self-Evaluation Protocol
Ensuring Rigor in Local Assessment
Systems: A Self-Evaluation Protocol
Building upon the key validity questions from the digital workbook, these two
protocols are specifically designed to assist state and local educators in evaluating
the assessments that make up their assessment systems.
Congratulations to everyone involved and stay tuned to the
AERA website (http://aera.net) for additional
information about the award.
Dr. Ellen Forte, CEO and Chief Scientist at edCount, joined Daniel F. Mix and Dr. Kristen Huff, both with Curriculum Associates, in a presentation/breakout session at the NCME Special Conference in Boulder, CO on September 18, 2019. Their presentation/session focused on how data is generated for the many assessment reports from a balanced assessment system and how these assessment data can be used to inform recommendations for individualized, small group, and whole class instruction. The purpose of the NCME Special Conference this year was to foster dialogue between education practitioners, policymakers, and researchers around implementing principled approaches to developing and using classroom assessment for teaching and learning.
The Standards and Testing Agency (STA) of the Education Department for the United Kingdom has invited Dr. Ellen Forte, CEO and Chief Scientist at edCount, to join its Technical Advisory Group (TAG). The STA is responsible for overseeing the national assessments for UK students in the grades that correspond to kindergarten through grade 6 in the United States. As a member of the STA-TAG, Dr. Forte will support the STA in ensuring that its practices meet professional standards for high quality assessment systems as established in The Professional Standards for Educational and Psychological Measurement (AERA, APA, NCME, 2014).
edCount, ACS Ventures, the Nebraska Department of Education, and the Indiana Department of Education presented a NCME conference session to share information about a compilation of resources developed by the federally-funded Strengthening Claims-based Interpretations and Uses of Large-scale Science Assessment Scores (SCILLSS) project. The resources are designed to (a) provide state and local educators with a grounding in the principles for high quality assessments that are critical to the appropriate selection, development, and use of assessments in educational settings, and (b) offer a framework for making decisions about developing or adopting tests, as well as for evaluating tests currently in use.
The presentation focused on a discussion of two tools that are designed to assist educators in developing a deeper understanding of the components of a comprehensive assessment system, where information can be efficiently collected and evaluated to support inferences about student achievement and the appropriate use and interpretation of test scores–Ensuring Rigor in State Assessment Systems: A Self-Evaluation Protocol and Ensuring Rigor in Local Assessment Systems: A Self-Evaluation Protocol. Presenters also introduced and discussed the Digital Workbook on Educational Assessment Design and Evaluation, a companion resource developed by SCILLSS partners that includes a collection of five interactive chapters designed to provide state and local educators with a grounding in the principles of high-quality assessments. The workbook provides an in-depth exploration of the purposes and uses of educational assessments and the concept of validity. It also further elucidates the four key validity questions that guide the evaluation of assessment scores for their intended purposes and uses.
To provide user perspective regarding the process and outcomes of implementing the SCILLSS resources at a state level, the Nebraska Department of Education, lead state and grantee on the SCILLSS project, and the Indiana Department of Education shared how their agencies implemented the protocol and workbook to better understand the current status of their state assessment system and frame next steps to enhance the overall quality them, ensuring the validity and reliability of test scores for their intended uses. A highlight of the session was hearing a school principal from Nebraska share how her teachers benefited from implementing the local protocol to better understand the status of their local assessment system.
The NCME presentation slides, the local and state self-evaluation protocols, and the digital workbook can be found here and more information about the SCILLSS project is available here.
The Assessing Special Education Students (ASES) State Collaborative on Assessment and Student Standards (SCASS ) of CCSSO invited edCount Executive Vice President, Liz Summers, Ph.D., to share the methodology, results, and lessons learned from conducting student interaction studies (e.g., modified cognitive laboratories) with students with the most significant cognitive disabilities taking alternate assessments based on alternate achievement standards while working with the National Center and State Collaborative (NCSC). The ASES SCASS supports states as they enhance their 21st century assessment, accountability, and curriculum and instruction systems to provide full equity for students with disabilities. The student population taking these assessments is unique in their learning characteristics and traditional cognitive laboratory methodologies are inappropriate to gather response process validity evidence. Dr. Summers’ presentation was followed by a group conversation with all participating states discussing application of the lessons learned in current state contexts and evaluation plans.
edCount’s Strengthening Claims-based Interpretations and Uses of Local and Large-scale Science Assessment Scores (SCILLSS) project team developed a Digital Workbook on Educational Assessment Design and Evaluation: Creating and Evaluating Effective Educational Assessments. This digital workbook encompasses five assessment literacy modules, akin to chapters in a book, that together are intended to provide state and local assessment leaders and educators with a grounding in the principles of developing and evaluating high quality assessments. The first chapter provides an overview to the series and to the categories of validity questions that the subsequent four chapters target, which are construct coherence, comparability, fairness and accessibility, and consequences.
The chapters of the digital workbook are designed to be used on their own or as a resource for those completing the SCILLSS self-evaluation protocols at the local or state level. Each chapter is designed to be self-paced and approximately one-hour in duration. Dr. Ellen Forte, CEO and Chief Scientist at edCount, drafted the content with strong input from experts in each validity area and narrates each chapter. In addition to the core presentation for each chapter, supplemental information and resources such as research citations, articles, web links, and a glossary are provided for participants to access.
The Digital Workbook on Educational Assessment Design and Evaluation is comprised of an Adobe Connect digital workbook, PowerPoint presentation, script, and MP3 for each of the five chapters, and can be found here: http://www.scillsspartners.org/scillss-resources/. We encourage assessment leaders to access, share, and use these materials in states and local education agencies to enhance knowledge and understanding of assessment design and evaluation.