NILOA Guest Viewpoints
We’ve invited learning outcomes experts and thought leaders to craft a Viewpoint. We hope that these pieces will spark further conversations and actions that help advance the field. To join the conversation, click the link below the Viewpoint. You can also sign up here to receive monthly newsletters that headline these pieces along with NILOA updates, current news items, and upcoming conferences.
Taking Stock of the Assessment Movement –
Liberal Education, Winter, 2017
Peter Ewell, Pat Hutchings, Jillian Kinzie, George Kuh & Paul Lingenfelter
NILOA Senior Scholars
Thanks to AAC&U for featuring the assessment movement in higher education in the Winter, 2017 issue of Liberal Education. The focus is timely, not only because it’s useful after 30 years of work to step back and take stock, but because the need is greater than ever that an even larger proportion of the nation’s citizens engage in high-quality postsecondary study. Sound assessment of student learning is required to assure that colleges and universities have evidence to guide their continuous improvement efforts.
The National Institute for Learning Outcomes Assessment (NILOA) was established in 2008 to “discover and disseminate ways that academic programs and institutions can productively use assessment data internally to inform and strengthen undergraduate education….” Although NILOA is a logical outgrowth of the assessment movement that began decades earlier, its work is a direct response to the 2006 report, A Test of Leadership: Charting the Future of American Higher Education, released by US Secretary of Education Margaret Spellings’ Commission on the Future of Higher Education. The Commission called for changing higher education “from a system primarily based on reputation to one based on performance.”
Many in the higher education community objected to particular recommendations and the “tone” of the Commission report, which seemed to reflect little confidence in academic leadership and undue confidence in the effectiveness of consumer information and hard-edged accountability measures. At the same time, few disputed the growing importance of higher education and the need to increase the quality and extent of educational attainment in the United States. NILOA was founded to advance that purpose by improving assessment practices and by disseminating our growing knowledge about what works to promote student learning in undergraduate study.
Last spring NILOA issued a policy statement that summarizes what we have learned about how to approach the assessment of learning and using its results to obtain higher achievement. Indeed, using evidence of student learning for improvement is perhaps the strongest argument against excessive compliance requirements or crude accountability mechanisms. http://www.learningoutcomesassessment.org/NILOA_statement.html
In this context, the first three essays in the Winter, 2017 issue of Liberal Education are instructive for reviewing the challenges facing higher education, the achievements of the assessment movement, and the work that remains.
The opening essay by W. Russell Neuman, “Charting the Future of US Higher Education: A Look at the Spellings Report Ten Years Later,” focused on two facets of the “performance” of higher education which have competed for attention in the last decade: preparing more students for successful careers, and improving the quality of student learning outcomes and the number of students who succeed.
Career success obviously depends on what students know and can do, but it is not a simple matter to measure either of these outcomes. Neuman chronicles the academic community’s resistance to efforts to hold higher education accountable for performance based on crude outcomes like graduation rates and the pay and employment status of graduates. He also describes the limited successes of the community’s efforts to measure critical thinking and other learning outcomes through standardized measures like the Collegiate Learning Assessment (CLA), as well as a variety of evidence-gathering approaches advanced by the accreditation community.
Neuman argues that such efforts to focus on performance have gained some purchase at the institutional level, but they “seldom make themselves felt in classrooms and faculty offices.” In the next ten years, he suggests that technological innovations in teaching will naturally lead to a greater faculty focus on the intended educational outcomes of instruction and the relative effectiveness of different instructional approaches. He also suggests that technology is providing useful opportunities to enhance both assessment and instruction because it enables detailed real-time analysis of the student’s interactions with computer based instructional materials.
The second essay in the issue, “Toward an Improvement Paradigm for Academic Quality” by Douglas D. Roscoe, argues that an “improvement paradigm” should replace the “assessment paradigm" to assure quality in higher education. Roscoe argues that assessment as currently practiced, mostly in response to institutional accreditation, is costly and increasingly ritualistic, which more often generates faculty resistance than it inspires improvement. In his words, “The reality is that the improvement piece of the assessment paradigm often takes a back seat to the collection of data.”
This is a serious concern. Over-emphasizing accountability and compliance naturally leads to a demand for precision of measurement that is impossible to achieve. Indeed, the most “reliable” and “valid” assessments, if they take the form of standardized tests, usually provide poor representations of the complex abilities that graduates should take away from a program of undergraduate study.
Roscoe’s proposed solution to this problem, however, is to focus on improving instruction directly through faculty conversation and the research literature without reference to a shared framework for assessing learning outcomes or, indeed, by collecting any new evidence at all. While research does provide helpful guidance for improvement, efforts to improve that do not start by defining the objectives of instruction are akin to sailing without a compass. We agree that “improvement” must be the desired end. But efforts to improve without shared instructional goals and associated measurements will not lead to systematic improvement.
Consistent with this view, Terrell Rhodes’ essay, “The VALUE of Learning: Meaningful Assessment on the Rise,” chronicles the achievements of the assessment movement through the work of AAC&U. Rhodes is convinced that the academic community has made significant progress in articulating core learning objectives, developing meaningful, authentic assessments of learning, and improving instruction and attainment through use of the resulting information. As a case in point, he describes how AAC&U’s Valid Assessment of Undergraduate Education (VALUE) initiative, and its associated rubrics that address numerous collegiate proficiencies, is based on a broad consensus among educators and employers on desired learning outcomes, and emphasizes the importance of basing assessment on actual student work.
The utility of this approach has been widely demonstrated, perhaps most visibly by the Multi-State Collaborative to Advance Learning Outcomes Assessment (MSC) project undertaken jointly by AAC&U and the State Higher Education Executive Officers (SHEEO). Now in its third year, the MSC involves thirteen states using VALUE rubrics to rate tens of thousands of student artifacts in communications, quantitative reasoning, and critical thinking. NILOA has supported the work of the MSC in providing states with assignment design help in the form of state-wide charrettes. Meanwhile, NILOA’s own work using the Degree Qualifications Profile (DQP) is adding to this knowledge base by assisting about 800 institutions in harnessing the DQP and its associated Assignment Library, which contains a growing number of model assignments designed explicitly to yield scorable student artifacts for various DQP proficiencies. Finally, NILOA continues to assist institutional good practice in assessment through its periodic surveys of institutional assessment practices (the third of which will be out in the field in April 2017), its numerous publications on assessment, and its many web-based resources. Further, the NILOA 2015 book, Using Evidence of Student Learning to Improve Higher Education, focused upon moving the conversation on assessment away from a compliance/accountability exercise and towards more meaningful use, populated with various examples of how such work is unfolding in institutions throughout the US.
In sum, we are grateful to AAC&U for helping the field “take stock” of the assessment movement in the most recent issue of Liberal Education. We look forward to continued partnering with AAC&U and other organizations and individuals committed to furthering this important work in the years to come. As we conclude in our policy statement, “it is no longer beyond the capacity of a college or university to articulate expectations for learning, to document student progress toward these expectations, and to use the resulting evidence to improve student success...doing this job, and doing it well, is within our grasp…failing to do so shortchanges our students and the many others who have a major stake in the quality of higher education.”