National Institute for Learning Outcomes Assessment |

National Institute for Learning Outcomes Assessment

NILOA Guest Viewpoints

We’ve invited learning outcomes experts and thought leaders to craft a Viewpoint. We hope that these pieces will spark further conversations and actions that help advance the field. To join the conversation, click the link below the Viewpoint. You can also sign up here to receive monthly newsletters that headline these pieces along with NILOA updates, current news items, and upcoming conferences.

 

Lessons from the Field: “Tracer” Study Shows Connections Between Faculty Development and Student Learning

Carol Rutz & Ellen Iverson
Carleton College

 

Introduction

How do programs for faculty development in pedagogy influence teaching and student learning in higher education?  Given the importance of student learning outcomes at all levels, from the institution to the course, can an institution gauge the degree to which improved teaching leads to improved outcomes?  Moreover, how are such programs supported in the institutional budget?

These questions informed a study funded by the Spencer Foundation and conducted by faculty and staff at Washington State University (WSU) in Pullman, WA, and Carleton College in Northfield, MN.  Other than shared geographic latitude, the two institutions would seem to have little in common:  One is a large, public, land-grant university with multiple colleges and degrees, and the other is a small, private, liberal arts college that grants a baccalaureate degree.  However, both institutions have devoted resources to faculty programs designed to improve teaching that align with learning outcomes.  The study traced (hence its nickname, “Tracer”) the effects of faculty development on teaching and found evidence that student learning is improved as a result.  Evidence obtained from both campuses is detailed in Condon, et al., (2016).

Spanning more than three years and using multiple methods, including faculty interviews; analysis of syllabi and assignments; classroom observations; and analysis of students’ work, the study sought to situate the effects of faculty development on student learning within the broader context of the campus culture. Both institutions point to long-standing portfolio assessments of student writing as a site for research.  The study emphasized that faculty development programs informed by assessment helped motivate the pedagogical changes that promote improved learning. 

For example, Carleton’s strategy for faculty development stresses the articulation of course learning goals; scaffolding of large assignments with smaller, staged components; providing feedback on required drafts; and using rubrics and other response techniques that connect with learning goals.  A typical workshop on say, using data in writing assignments, would touch on all of the foregoing as well as specific ways to incorporate quantitative information—written and visual—into assignments for courses ranging from social sciences to natural sciences to humanities.

Study Findings

The study shines a light on lessons about faculty development which may be useful in other institutional contexts such as: identify clear goals for changes in instruction that align to institutional goals; align institutional goals with student competencies that faculty value; promote excellent teaching through explicit support opportunities; and be mindful of opportunities where routine faculty work can embed faculty learning about teaching. Following are major findings of the Tracer study.

  • Retrospective assessment of course documents and student work—obtained, at both campuses, through writing portfolios and faculty generosity—showed growth in assignment clarity and sophistication that supported improvements in students’ learning as demonstrated in their written work. Such evidence is heartening, especially to those who argue for consistent funding for faculty development programming.

  • Improvements are identified at the institutional level as well as in individual classrooms. Short-term assessments of individual classrooms can miss the effects of continued improvements. The Tracer study centered on overarching competencies valued by the institution’s faculty and detected in student writing across disciplines. The faculty development goals aligned to the campus initiative, which aligned to specific student learning competencies. To test the connections, researchers applied an assessment instrument to the data that reflected the institutional alignment of the initiative, the faculty development which supported it, the changed faculty practices, and the demonstrated student learning. Changes at the course level reflect institutional goals of improving teaching and learning.

  • The study also uncovered the snowball effect of formal faculty development at Carleton. Carleton offers a support structure that includes formal faculty development, grants for curricular changes, brown bag series, and outside speakers, all of which promote experimentation with real resources. When faculty perceive that work and experimentation on teaching is valued and rewarded by the institution, good practices spread. Both campuses revealed the spread of teaching practices to those who were not involved in the formal faculty development events.

  • Cultivating teaching as a learning process for faculty can support institutional goals. The study found that the ability of faculty to bring their new knowledge into practice relied on the institutional context. At both sites, service work, such as committee service, offered opportunities for the faculty learning process to take hold. Faculty from WSU and Carleton perceived writing portfolio rating sessions as powerful opportunities for learning about teaching. Other routine sites of faculty professional life including hiring practices, faculty orientations, performance evaluations, and curricular planning. Any and all of such activities can support learning about teaching.

Final Thoughts

Every campus has a culture of teaching and learning. The Tracer findings revealed the generative nature of teaching and learning at both institutions. It was easy to identify the high participators of faculty development at both sites: These faculty thrive on improving their teaching; are likely to lead campus initiatives, hold lunchtime discussions, bring in outside experts, and catalyze positive changes in teaching and learning. While these faculty may be well known within a small liberal arts college, they are visible in other ways at larger institutions. For example, at WSU, a group of faculty recognized as members of the President’s Teaching Academy designed the Six Learning Goals of the Baccalaureate.

Ultimately, by making visible the substantial faculty learning that is taking place, a generative culture of teaching and learning can support innovation in teaching and excellence in student learning.  Institutions do well to invest in activities that speak to faculty learning as well as to student learning outcomes. In sum, professional development opportunities are shown to increase student learning, not only for those that participate directly, but for the campus as a whole.  Investments in professional learning of all types create a productive culture of teaching and learning, increasing the capacity of faculty to learn about effective education and improve their practice.  Campuses seeking to create strong educational environments for their students benefit from such investments. 

Reference

Condon, W., Iverson, E. R., Manduca, C. A., Rutz, C., & Willett, G. (2016). Faculty development and student learning: Assessing the connections. Indiana University Press.

 

Check out our past Viewpoints:

Lessons from the Field: “Tracer” Study Shows Connections Between Faculty Development and Student Learning
Carol Rutz & Ellen Iverson

Can Teaching Metacognitive Learning Strategies Counteract Student Stress, Anxiety, and Depression?
Saundra McGuire

Promoting an Improvement Culture
Claudia J. Stanny

Rubrics: Lessons from the Field
Laura Massa & Margaret Kasimatis

In Appreciation of Clifford Adelman
Peter T. Ewell

Why Are We Assessing?
Higher Education Assessment Practitioners

The EEQ CERT: A New Way to Assure and Communicate Program Quality, Relevance, & Value
Melanie Booth

Bringing Student Voices to the Table: Collaborating with our Most Important Stakeholders
Ann E. Damiano

Responses to “The Misguided Drive to Measure ‘Learning Outcomes’”
Natasha Jankowski

Collaborating for Individual and Institutional Success: Libraries as Strategic Campus Partners
Jennifer Duncan, Kacy Lundstrom & Becky Thoms

Rethinking the Role of Work in Higher Education
David W. Marshall

Demand interoperability to dramatically improve the educational ecosystem
Jeff Grann

The Comprehensive Student Record at Dillard University
Demetrius Johnson

NILOA Remembers Assessment Pioneer Sister Joel Read of Alverno College
Peter Ewell, Pat Hutchings, & Russ Edgerton

The Neuroscience of Learning and Development: How can Evidence Legitimize Self-Reflection?
Marilee Bresciani Ludvik

Taking Stock of the Assessment Movement – Liberal Education, Winter, 2017
Peter Ewell, Pat Hutchings, Jillian Kinzie, George Kuh & Paul Lingenfelter

Eight Years On: Early—and Continuing—Lessons from the Tuning Project
Daniel J. McInerney

Real-time Student Assessment: Prioritizing Enrolled Students’ Equitable Progress toward Achieving a High-Quality Degree
Peggy Maki

Academic and Student Affairs Sides of the House: Can We Have an Open Concept Learning Design?
Darby Roberts

Just Assessment. Nothing More. Nothing Less.
Wayne Jacobson

Design for a Transparent and Engaging Assessment Website
Frederick Burrack and Chris Urban

Improvement Matters
Peter Felten

Working Together to Define and Measure Learning in the Disciplines
Amanda Cook, Richard Arum, and Josipa Roksa

The Simplicity of Cycles
Mary Catharine Lennon

Helping Faculty Use Assessment Data to Provide More Equitable Learning Experiences
Mary-Ann Winkelmes

Ignorance is Not Bliss: Implementation Fidelity and Learning Improvement
Sara J. Finney and Kristen L. Smith

Student Learning Outcomes Alignment through Academic and Student Affairs Partnerships
Susan Platt and Sharlene Sayegh

The Transformation of Higher Education in America: Understanding the Changing Landscape
Michael Bassis

Learning-Oriented Assessment in Practice
David Carless

Moving Beyond Anarchy to Build a New Field
Hamish Coats

The Tools of Intentional Colleges and Universities: The DQP, ELOs, and Tuning
Paul L. Gaston, Trustees Professor, Kent State University

Addressing Assessment Fatigue by Keeping the Focus on Learning
George Kuh and Pat Hutchings, NILOA

Evidence of Student Learning: What Counts and What Matters for Improvement
Pat Hutchings, Jillian Kinzie, and George D. Kuh, NILOA

Using Evidence to Make a Difference
Stan Ikenberry and George Kuh, NILOA

Assessment - More than Numbers
Sheri Barrett

Challenges and Opportunities in Assessing the Capstone Experience in Australia
Nicolette Lee

Making Assessment Count
Maggie Bailey

Some Thoughts on Assessing Intercultural Competence
Darla K. Deardorff

Catalyst for Learning: ePortfolio-Based Outcomes Assessment
Laura M. Gambino and Bret Eynon

The Interstate Passport: A New Framework for Transfer
Peter Quigley, Patricia Shea, and Robert Turner

College Ratings: What Lessons Can We Learn from Other Sectors?
Nicholas Hillman

Guidelines to Consider in Being Strategic about Assessment
Larry A. Braskamp and Mark E. Engberg

An "Uncommon" View of the Common Core
Paul L. Gaston

Involving Undergraduates in Assessment: Documenting Student Engagement in Flipped Classrooms
Adriana Signorini & Robert Oschner

The Surprisingly Useful Practice of Meta-Assessment
Keston H. Fulcher & Megan Rodgers Good

Student Invovlement in Assessment: A 3-Way Win
Josie Welsh

Internships: Fertile Ground for Cultivating Integrative Learning
Alan W. Grose

What if the VSA Morphed into the VST?
George Kuh

Where is Culture in Higher Education Assessment and Evaluation?
Nora Gannon-Slater, Stafford Hood, and Thomas Schwandt

Embedded Assessment and Evidence-Based Curriculum Mapping: The Promise of Learning Analytics
Jane M. Souza

The DQP and the Creation of the Transformative Education Program at St. Augustine University
St. Augustine University

Why Student Learning Outcomes Assessment is Key to the Future of MOOCs

Wallace Boston & Jennifer Stephens

Measuring Success in Internationalization: What are Students Learning?
Madeleine F. Green

Demonstrating How Career Services Contribute to Student Learning
Julia Panke Makela & Gail S. Rooney

The Culture Change Imperative for Learning Assessment
Richard H. Hersh & Richard P. Keeling

Comments on the Commentaries about "Seven Red Herrings"
Roger Benjamin

Ethics and Assessment: When the Test is Life Itself
Edward L. Queen

Discussing the Data, Making Meaning of the Results
Anne Goodsell Love

Faculty Concerns About Student Learning Outcomes Assessment
Janet Fontenot

What to Consider When Selecting an Assessment Management System
R. Stephen RiCharde

AAHE Principles of Good Practice: Aging Nicely A Letter from Pat Hutchings, Peter Ewell, and Trudy Banta

The State of Assessment of Learning Outcomes Eduardo M. Ochoa

What is Satisfactory Performance? Measuring Students and Measuring Programs with Rubrics
Patricia DeWitt

Being Confident about Results from Rubrics Thomas P. Judd, Charles Secolsky & Clayton Allen

What Assessment Personnel Need to Know About IRBs
Curtis R. Naser

How Assessment and Institutional Research Staff Can Help Faculty with Student Learning Outcomes Assessment
Laura Blasi

Why Assess Student Learning? What the Measuring Stick Series Revealed
Gloria F. Shenoy

Putting Myself to the Test
Ama Nyamekye

From Uniformity to Personalization: How to Get the Most Out of Assessment
Peter Stokes

Transparency Drives Learning at Rio Salado College
Vernon Smith

Navigating a Perfect Storm
Robert Connor

It is Time to Make our Academic Standards Clear
Paul E. Lingenfelter

In Search for Standard of Quality
Michael Bassis

Avoiding a Tragedy of the Commons in Postsecondary Education
Roger Benjamin