National Institute for Learning Outcomes Assessment |

National Institute for Learning Outcomes Assessment

NILOA Guest Viewpoints

We’ve invited learning outcomes experts and thought leaders to craft a Viewpoint. We hope that these pieces will spark further conversations and actions that help advance the field. To join the conversation, click the link below the Viewpoint. You can also sign up here to receive monthly newsletters that headline these pieces along with NILOA updates, current news items, and upcoming conferences.

 

Eight Years On: Early—and Continuing—Lessons from the Tuning Project
Daniel J. McInerney
Utah State University

 

It has been eight years since the first teams of educators took part in the introduction of the Tuning process to U.S. higher education. In April 2009, under a generous grant from the Lumina Foundation, groups from the state systems of Indiana, Minnesota, and Utah gathered in Chicago to learn of the initiative first introduced in the European Union in 2000 as a university response to the Bologna Project.1 Presentations about the work, guided by leading experts such as Clifford Adelman, Robert Wagenaar, Volker Gehmlich, and Tim Birtwistle, reviewed the background, operation, and informing purposes of a project designed to clarify points of convergence and comparability in curricula and degrees.

Two sets of concerns quickly arose, however, both of which still have a tendency to throw a monkey wrench into the progress of some key academic initiatives. First, in the Chicago meeting, participants who were cutting their teeth on all things related to assessment had difficulty keeping up with discussions of “outcomes,” “rubrics,” “diploma supplements,” “qualifications frameworks,” and “generic and specific competences.” The very vocabulary of European academic reform—along with the institutional structures and administrative frameworks of E.U. higher education—remained foreign to the minds of many in the audience.

A second concern arose outside of the Chicago sessions. With the publication of the first news articles about the meeting, many educators began to express doubts, skepticism, and fears about the project. In particular, commentators worried that: Tuning’s discipline-based focus could hamper efforts to build and expand interdisciplinary studies and skill sets; a competence-based focus would turn attention away from serendipitous educational journeys of discovery, creativity, and innovation; the pedagogically-based scholarship of teaching and learning might impose a new and constraining level of “expertise” in higher education discussions; outcomes-based objectives could result in complicated or meaningless “measures” of attainment; and a convergence-based drive towards disciplinary consensus and harmony might endanger academic freedom. 2

In other words, the Chicago conference revealed two sets of difficulties that required prompt attention: finding the right words to describe the project and anticipating the objections that colleagues would likely raise. Over eight years, “Tuners” have carefully tackled—and continually revisited—these initial problems, forming thoughtful solutions to early obstacles that regularly re-surface with audiences new to Tuning.

Developing an American idiom for Tuning’s reforms

It was clear to those at the opening conference that advocates would have to go beyond European descriptions of Tuning if the project was going to resonate with audiences across the States.3 Fortunately, guest speakers in Chicago helped by turning their attention to a “speed dating” form of delivery intended to present the work in the clear and transparent terms that the project was meant to embody. Adelman in particular drew from themes in his earlier publications on Bologna and Tuning, bluntly reminding participants that the initiatives represented “the most far-reaching and ambitious reform of higher education ever undertaken.”4 He alerted educators in the States of the need to “listen up” to a set of projects that were “the most important academic change since the development of the community college system.”5

As colleagues began to take Tuning back to campuses, they quickly figured ways to pitch the process to American audiences. An initial guidebook to Tuning, published by the Institute for Evidence-Based Change in 2010, was one of the earliest publications to suggest an informing question still used to spark discussions about the project: when students complete a major or degree, what should they know, understand, and be able to do? 6A second “conversation starter” came from early conference papers and presentations, asking those new to Tuning what a major or degree represented – not in terms of required courses, Carnegie credit hours, or grade point averages but in terms of the learning our programs expect students to develop.7 When a large disciplinary society, the American Historical Association, began its own Tuning work in 2012, the project director, Anne Hyde, posed a third way of beginning faculty discussions. “Imagine,” she wrote, “a first meeting of the academic year where no one talked about budgets, assessment, course assignments, or parking. What if we all started the year discussing what disciplinary ideals link us as historians and how we might best introduce those to our students?”8 Boiling down the work of Tuning to an appealing catch phrase, Holiday Hart McKiernan and Tim Birtwistle suggested that the project addressed ways of “making the implicit explicit” in higher education.9 Drilling down even further into the essence of Tuning, Paul Gaston focused on a single word, “intentionality,” as the key term conveying “the thread that connects Tuning” with related projects in the Degree Qualifications Profile, Essential Learning Outcomes, and assessment.10

As one who is often on the road explaining (and encouraging) faculty work in Tuning, I still draw from all five of the suggestions colleagues have developed, not merely to introduce the goals of the initiative but to translate its purposes into the practical, day-to-day terms that an instructor confronts in a course, an office hour, a department meeting, or a committee. Specialized, jargon-laden, acronym-filled descriptions of a reform project tend to go in one ear and out the other with many faculty colleagues. Anyone promoting programs such as Tuning should begin not by concentrating on the inner logic and component parts of a project but, rather, on what’s in it for another educator. It is often best to start discussions with the conditions, frustrations, and puzzles of a classroom—and the expertise, hopes, and commitment of an instructor. As colleagues in the organization Public Agenda have suggested, “begin where people are, not where you want them to be.”11

The Functions of Academic Conflict

News of the first U.S. meetings on Tuning may have evoked a stream of protests, warnings, and even conspiratorial theories from educators concerning the straitjacket of standardization under which academe would soon operate in both teaching and testing. It’s worth noting, however, that even the first stories about the Chicago meeting also carried strong support for the project. Commentators rejected fears that the energizing diversity of American higher education would be lost in the program, arguing that “Tuned” programs would not be forced “to surrender their identity and become homogeneous, characterless blobs.” Others challenged the notion that Tuning held U.S. institutions up to inappropriate expectations from abroad, reminding readers that “when countries are willing to learn from each other, one gets adaptations within contexts and not carbon copies.” Still others brushed off worries of Tuning as an authoritarian system that would leave educators powerless, pointing out that Tuning “TRUSTS the faculty—to center student learning, to reconsider ways to give students practice in order to develop competencies, and to be informed of the efficacy of their work through authentic, deliberate assessment.12

If there is a model for handling questions, doubts, and suspicions about Tuning and related projects, I am proud to point to the work of the premier disciplinary organization in my own field of study, the American Historical Association. With a grant from Lumina in 2012, the AHA launched the first Tuning effort in the world led by a scholarly society. And with 14,000 members known to be rather skeptical, cantankerous, and independently-minded, it came as no surprise to hear from many card-carrying colleagues who felt both affronted and alarmed by the new project. Some were worried about the “instrumentalist” approach to education they saw in the Tuning project. Some feared “corporate” influences reshaping scholarly study. Others expressed concern that colleges (like high schools) would soon be forced to concentrate on “teaching to the test.” The AHA’s executive director, James Grossman, quickly embraced a simple and straightforward approach to Tuning opponents: give critics a clear, public, and recurring voice. The AHA purposely invited skeptics into the project, published their concerns in its monthly newsletter, and opened the floor to their comments during the organization’s annual conferences.13

The approach taken by Grossman and the AHA suggested an academic variant of a model outlined in 1956 by sociologist Lewis Coser in his classic work, The Functions of Social Conflict.14 Rather than viewing debate and controversy as forces of dysfunction and disorder in a collective group, the disciplinary society viewed its vigorous internal discussions on Tuning as healthy signs of individual engagement and organizational openness. Rather than stifling, marginalizing, or ignoring opposition, the AHA acknowledged criticism, provided a forum for debate, and responded to the concerns some members expressed.

The approach was one I found appropriate four years earlier in my own work—on a much smaller scale. In 2008-2009, a period of frightening budget cutbacks due to the Great Recession, my job as interim department head included guiding fellow historians into our state’s Tuning work. One can easily guess the response I received from colleagues who were (properly) more focused on the question of who would still have a job the following year. I anticipated that there would be considerable resistance and thought the best way to proceed was to give people plenty of room to vent. They did, of course. Most of the discussion pointed to past failures—of both previous assessment projects and of students unprepared for our majors’ required senior thesis capstone. It did not take long, however, for complaints about gaps in our curriculum to turn into a different type of discussion: whose courses actually taught students the skills they would need to succeed in a large research project? One by one, colleagues stepped up and offered to build skills-based units into their course assignments. Our department had stumbled unknowingly on the strategy of “backward design,” starting with the final requirement of a program and tracing back the steps students needed to succeed in that task. The discussion showed me that intelligent, thoughtful faculty members probably have many complaints to voice; but eventually, they also start to come up with answers to the problems they observe.

I was also struck, however, by colleagues’ comments on past projects of academic review. The university had dragged us through several programs of course evaluation and curriculum discussion. The work typically followed a top-down design from administrative offices with a one-size-fits-all approach that disregarded disciplinary distinctions. Past discussions had all remained “in house” with academics talking to academics. Faculty’s regular evaluation of actual student work in actual classes remained suspect. And none of us could recall seeing any concrete results of the activity. In other words, faculty resistance was not necessarily grounded in stubborn resistance or curmudgeonly complaints. The criticisms were also grounded in traumatic memories of assessments past. Who would want to slog through those old procedures again?

The department discussion has stayed with me, and in visits to other campuses I regularly remind audiences that Tuning has changed the rules of the academic review game. The project is faculty led, discipline specific, and outward looking, calling on educators to engage with a wide range of “stakeholders” inside and outside our institutional walls. And meaningful assessment of student learning, now more than ever, is faculty centered, drawn from rigorously designed assignments carefully “tuned” to the stated goals of a course.

Perhaps most importantly, Tuning provides educators with a solid grounding in conversations we will all continue to face in assessment, accreditation, and accountability. By digging deeply into the informing goals and activities within our own disciplinary fields, educators become more reflective participants in broader discussions about courses, curricula, programs, and degrees. The best starting point for all of this work is for colleagues to dig deeply into the areas of study where they are the most familiar: the disciplinary fields to which they have devoted their time, attention, and passion.

                                     

Footnotes

1For a general overview of the Bologna Project, see: “The Bologna Process and the European Higher Education Area,” Education and Training; European Commission,

2See, for example, the debates among readers that quickly followed the publication of Scott Jaschick’s article, “‘Tuning’ College Degrees,” Inside Higher Ed, April 8, 2009. For additional reports on the first Tuning meeting, see: Aisha Labi, “Europe's Higher-Education Restructuring Holds Lessons for U.S., Report Says,” Chronicle of Higher Education, April 08, 2009; Stan Katz, “‘Tuning’ American Higher Education,” Chronicle of Higher Education, April 10, 2009; Tamar Lewin, “Colleges in 3 States to Set Basics for Degrees,”New York Times, April 8, 2009.

3Consider, for example, the fairly complicated definition of the Tuning initiative presented on the first major E.U. website for the project. The opening page stated that Tuning began as “a project to link the political objectives of the Bologna Process and at a later stage the Lisbon Strategy to the higher educational sector. Over time Tuning has developed into a Process, an approach to (re-)designing, develop, implement, evaluate and enhance quality first, second and third cycle degree programmes. . . . The name Tuning is chosen for the Process to reflect the idea that universities do not and should not look for uniformity in their degree programmes or any sort of unified, prescriptive or definitive European curricula but simply look for points of reference, convergence and common understanding.” Tuning Educational Structures in Europe.

4Clifford Adelman, The Bologna Club: What U.S. Higher Education Can Learn from a Decade of European Reconstruction (Washington, DC: Institute for Higher Education Policy, 2008). In addition to Adelman’s pathbreaking work, one of the most influential studies of the Bologna project and its relevance to U.S. higher education may be found in Paul L. Gaston, The Challenge of Bologna: What United States Higher Education Has to Learn from Europe, and Why It Matters That We Learn It (Sterling, Virginia: Stylus Publishing, 2010).

5 Author’s notes, “Tuning USA Conference,” Lumina Foundation, Chicago, April 2009.

6David W. Marshall, Michelle Kalina, Will Dane, Tuning Educational Structures: A Guide to the Process, Version 1.0 (Encinitas, CA: Institute for Evidence-Based Change), 2010), 1.

7Marianne Wokeck, Norm Jones, Dan McInerney, “Tuning the Disciplines, the Humanities: The Tuning Process in Two States,” Association of American Colleges & Universities 2010 Annual Meeting, January 22, 2010.

8Anne Hyde, “Tuning and Teaching History as an Ethical Way of Being in the World,” AHA Today: A Blog of the American Historical Association, July 17, 2014,

9Holiday Hart McKiernan, Tim Birtwistle, “Making the Implicit Explicit: Demonstrating the Value Added of Higher Education by a Qualifications Framework,” The Journal of College and University Law 36:2 (2010): 512-60. When the article appeared, McKiernan served as senior vice-president and general counsel for the Lumina Foundation; Birtwistle, a UK Bologna expert, was visiting fellow at the Oxford Centre for Higher Education Policy Studies.

10Paul Gaston, “Essential Learning Outcomes, Degree Qualification Profiles, and Assessment: Connecting the Dots,” What is an Educated Person Conference, Utah System of Higher Education (October 26, 2012). In an essay entitled “The Tools of Intentional Colleges and Universities: The DQP, ELOs, and Tuning” (NILOA Newsletter [April 2015]), Gaston asks readers to imagine a scholar who submits a manuscript to an academic journal for publication–but refuses to clarify the structure, context, and objectives of the piece, insisting that “readers will eventually appreciate the importance of his article, even though it may take them many years to do so.” The expectation that we display intentionality in research, Gaston reminds us, carries over to our work in teaching.

11Alison Kadlec, Will Friedman, Changing the Conversation About Productivity: Strategies for Engaging Faculty and Institutional Leaders (New York: Public Agenda, 2010), 10.

12“Comments” (from Christopher Sharrock, Cliff Adelman, and Kathryn MacKay), “‘Tuning’ College Degrees.”

13Lumina announced the Tuning grant to the AHA in February 2012. By April, the first debates about the project were published in the history organization’s monthly publication. Executive Director Grossman was at the center of the conversations: James Grossman, “Tuning in the History Major,” Perspectives on History (April 2012); Christopher Doyle, “On Historians, Tuning, and Markets,” Perspectives on History (September 2012); James Grossman, “Response to On Historians, Tuning, and Markets,” Perspectives on History (September 2012).

14Lewis A. Coser, The Functions of Social Conflict (Glencoe, IL: Free Press, 1956).

Check out our past Viewpoints:

Real-time Student Assessment: Prioritizing Enrolled Students’ Equitable Progress toward Achieving a High-Quality Degree
Peggy Maki

Academic and Student Affairs Sides of the House: Can We Have an Open Concept Learning Design?
Darby Roberts

Just Assessment. Nothing More. Nothing Less.
Wayne Jacobson

Design for a Transparent and Engaging Assessment Website
Frederick Burrack and Chris Urban

Improvement Matters
Peter Felten

Working Together to Define and Measure Learning in the Disciplines
Amanda Cook, Richard Arum, and Josipa Roksa

The Simplicity of Cycles
Mary Catharine Lennon

Helping Faculty Use Assessment Data to Provide More Equitable Learning Experiences
Mary-Ann Winkelmes

Ignorance is Not Bliss: Implementation Fidelity and Learning Improvement
Sara J. Finney and Kristen L. Smith

Student Learning Outcomes Alignment through Academic and Student Affairs Partnerships
Susan Platt and Sharlene Sayegh

The Transformation of Higher Education in America: Understanding the Changing Landscape
Michael Bassis

Learning-Oriented Assessment in Practice
David Carless

Moving Beyond Anarchy to Build a New Field
Hamish Coats

The Tools of Intentional Colleges and Universities: The DQP, ELOs, and Tuning
Paul L. Gaston, Trustees Professor, Kent State University

Addressing Assessment Fatigue by Keeping the Focus on Learning
George Kuh and Pat Hutchings, NILOA

Evidence of Student Learning: What Counts and What Matters for Improvement
Pat Hutchings, Jillian Kinzie, and George D. Kuh, NILOA

Using Evidence to Make a Difference
Stan Ikenberry and George Kuh, NILOA

Assessment - More than Numbers
Sheri Barrett

Challenges and Opportunities in Assessing the Capstone Experience in Australia
Nicolette Lee

Making Assessment Count
Maggie Bailey

Some Thoughts on Assessing Intercultural Competence
Darla K. Deardorff

Catalyst for Learning: ePortfolio-Based Outcomes Assessment
Laura M. Gambino and Bret Eynon

The Interstate Passport: A New Framework for Transfer
Peter Quigley, Patricia Shea, and Robert Turner

College Ratings: What Lessons Can We Learn from Other Sectors?
Nicholas Hillman

Guidelines to Consider in Being Strategic about Assessment
Larry A. Braskamp and Mark E. Engberg

An "Uncommon" View of the Common Core
Paul L. Gaston

Involving Undergraduates in Assessment: Documenting Student Engagement in Flipped Classrooms
Adriana Signorini & Robert Oschner

The Surprisingly Useful Practice of Meta-Assessment
Keston H. Fulcher & Megan Rodgers Good

Student Invovlement in Assessment: A 3-Way Win
Josie Welsh

Internships: Fertile Ground for Cultivating Integrative Learning
Alan W. Grose

What if the VSA Morphed into the VST?
George Kuh

Where is Culture in Higher Education Assessment and Evaluation?
Nora Gannon-Slater, Stafford Hood, and Thomas Schwandt

Embedded Assessment and Evidence-Based Curriculum Mapping: The Promise of Learning Analytics
Jane M. Souza

The DQP and the Creation of the Transformative Education Program at St. Augustine University
St. Augustine University

Why Student Learning Outcomes Assessment is Key to the Future of MOOCs

Wallace Boston & Jennifer Stephens

Measuring Success in Internationalization: What are Students Learning?
Madeleine F. Green

Demonstrating How Career Services Contribute to Student Learning
Julia Panke Makela & Gail S. Rooney

The Culture Change Imperative for Learning Assessment
Richard H. Hersh & Richard P. Keeling

Comments on the Commentaries about "Seven Red Herrings"
Roger Benjamin

Ethics and Assessment: When the Test is Life Itself
Edward L. Queen

Discussing the Data, Making Meaning of the Results
Anne Goodsell Love

Faculty Concerns About Student Learning Outcomes Assessment
Janet Fontenot

What to Consider When Selecting an Assessment Management System
R. Stephen RiCharde

AAHE Principles of Good Practice: Aging Nicely A Letter from Pat Hutchings, Peter Ewell, and Trudy Banta

The State of Assessment of Learning Outcomes Eduardo M. Ochoa

What is Satisfactory Performance? Measuring Students and Measuring Programs with Rubrics
Patricia DeWitt

Being Confident about Results from Rubrics Thomas P. Judd, Charles Secolsky & Clayton Allen

What Assessment Personnel Need to Know About IRBs
Curtis R. Naser

How Assessment and Institutional Research Staff Can Help Faculty with Student Learning Outcomes Assessment
Laura Blasi

Why Assess Student Learning? What the Measuring Stick Series Revealed
Gloria F. Shenoy

Putting Myself to the Test
Ama Nyamekye

From Uniformity to Personalization: How to Get the Most Out of Assessment
Peter Stokes

Transparency Drives Learning at Rio Salado College
Vernon Smith

Navigating a Perfect Storm
Robert Connor

It is Time to Make our Academic Standards Clear
Paul E. Lingenfelter

In Search for Standard of Quality
Michael Bassis

Avoiding a Tragedy of the Commons in Postsecondary Education
Roger Benjamin