David Shupe, designer of learning assessment management system eLumen, St. Paul, MN,
and Peter Daly, research director, Carrington College, Sacramento, CA

An Interview with The Civic Caucus

8301 Creekside Circle #920, Bloomington, MN 55437

May 4, 2012

Notes of the Discussion

Present (All by phone): Verne Johnson (chair), Dave Broden, Janis Clay, Paul Gilje (coordinator), Curt Johnson, Sallie Kemper, Tim McDonald

A. Introduction of guests - Peter Daly is the director for research and assessment at Carrington College California, in Sacramento. Carrington College CA is a for-profit career and technical school. A native Australian, Daly has been in the United States since 2004.

David Shupe is chief innovation officer for eLumen Collaborative, St. Paul, MN, an independent academic R&D firm. Before co-founding eLumen Collaborative in 2003 Shupe was System Director for Academic Accountability for the Minnesota State Colleges and Universities (MnSCU) system, Vice President of Academic Affairs and Student Development at Inver Hills Community College, and a Program Director at the University of Minnesota. He began his career as an administrator at Control Data Corp. Shupe is a graduate of Bethel College, and received his PhD from Syracuse University.

B. Discussion -

THE PROBLEM: Colleges do not know specifically what students have learned during their time in their programs

Colleges know more about a student when they arrive, through admissions, than when they leave, Shupe said.

Upon leaving a college a student is issued a transcript - it states which courses they took and what grade they received but doesn't say what they did well or poorly.

THE GOAL: Schools should be able to show specifically what students know and can do and, for each of these areas, how well

"It seems to us as a vocational college there's something missing in that transcript," Daly said. "It doesn't tell us what they can do and how well they can do it.

THE STRATEGY: Measure students for outcomes, not process

"I think it's best not to get into the technicals but keep it simple," Daly said, as he sought to illustrate what eLumen looks like to Carrington College California. "It's a far-reaching reform, but not a complicated one to understand."

Focus on learning outcomes

A learning outcome is a succinct statement of what a student can do as a result of a period of learning. In a course like Medical Assisting they will need to demonstrate to Carrington teachers very specific skills. That course has four student learning outcomes drawn from the program's set of expected learning outcomes, Daly said. The college will be able to say the students can do those things and the level at which they can do them.

Under this arrangement the college or a potential employer have much more information on outcomes than just courses and grades. "It allows us to have more granular detail."

"We require students to provide demonstration of their competency of learning outcomes," Daly said. "We have hundreds and hundreds of student learning outcomes that are being assessed in our courses at any given time. We have a lot of data."

Through eLumen the instructor can see what percentage of students are performing at which levels. That tells the instructor what the students have not learned as much as what they have learned. They know then where the students are weak and where they are strong. They know which students were able to master a skill, which weren't, and where the gaps are. The gap tells them where to put additional effort. Learner support sources can be targeted to those points.

Moving toward competency-based progression

Shupe commented that information on expected and actual learning outcomes can be used to drive continuous improvement of the college.

By working the eLumen management system into the organizational model the college begins the first of several incremental steps toward the ultimate goal of attending to student progress based on competency.

Create a culture of achievement, defining degrees by outcomes

Students at Carrington are focused on outcomes from the time they arrive, Daly said. Professors can identify the students that aren't doing well and create a special session for them on specifics. The system shows real time progress so teaching responds in real time to what students are and are not achieving, not just after the fact.

Made possible by new technology

"The institutional capacity to know in real time where any student or set of students stands relative to any set of expected learning outcomes is new territory - hasn't existed before," Shupe added.

What's been missing to this point is the ability to gather the data in one place. The technology is here, but has moved slower than it might otherwise because it has been in the realm of administrators - teachers haven't had the authority to do it. More academic institutions are interested in it now because the accrediting bodies are insisting on it.

This arrangement sets up the possibility for a new educational social contract, Shupe argued: Students will become responsible to work for outcomes that the faculty have defined, and the college becomes responsible for paying attention to student abilities and strengths, even ones that are unexpected. In the end, each student will have a digital personal outcomes transcript that complements the traditional transcript.

Daly added that for the student the college experience is more integrated. Courses are not each conducted in isolation. The evaluation of students is integrated. Instructors can see how students are doing in real time. The technology is designed to give real time access.

"Right now, without such technology, a college is like a holding company for departments and courses doing their own thing," Shupe said.

Instructors come to support the change

"Typically faculty members on the front end tend to be skeptical but see it's not as bad as they thought it would be," Shupe said. "This is less work than keeping track of points and grades. You're asking them to evaluate students differently."

Daly added that at Carrington there was some trepidation about accountability. Instructors are getting much more granular information about their teaching now.

"When they see the data," he said, "everything changes. Once we get the data people start to realize just how measureable of a reform that this is."

The resistance is an unknowing fear, and when they become familiar with it - particularly held to the alternative of standardized tests - they will choose the more academic version, Shupe observed.

C. Conclusion - Shupe projected growth for this way of organizing learning, and eLumen in particular.

Two institutions in Minnesota use the program - Concordia University, St. Paul, and Anoka Ramsey Community College - as part of 31 across the country reaching from California to Nova Scotia. Three of their clients are for-profit; 28 are non profit. Shupe says he plans to have 400 schools involved in 3-5 years.

"It used to be that the technology to do this did not exist. Now it does. The question now is which institutions will choose it."

The chair thanked Daly and Shupe for the visit.