When it comes to demonstrating the effectiveness of our learning strategies, is badging the answer?
The current paradigm
In corporate learning, we often talk about ways to evaluate the effectiveness of training. Primarily, we are interested in a) whether or not our participants learned what we aimed for them to learn, and b) whether or not they applied that learning to support or improve their performance on the job. There are so many problems with the methods we have traditionally used to measure these outcomes that we too often can’t find the time or support to effectively measure them. And the expansion of self-directed (informal) learning complicates the issue even further. (I could write several entire posts on the strategies and issues of evaluation, but that’s not the point I’m trying to get to here.)
A paradigm shift for consideration
Since the old paradigm is so problematic, the concept of “badging'” is intriguing as a potential paradigm shift. Our colleagues in the academic community – and especially those folks interested in do-it-yourself-education – are working to formalize a strategy of digital badging as a way of documenting knowledge and skills in ways that can be shared with employers and others interested in knowing what we have learned and what we have to offer. If you haven’t heard about this approach, take a moment to check out these resources:
Mozilla Open Badges – web site describing Mozilla’s efforts to develop a badging platform
‘Badges’ Earned Online Pose Challenge to Traditional College Diplomas – article from the Chronicle of Higher Education
Could Badges for Lifelong Learning Be Our Tipping Point? – HASTAC blog post by Cathy Davidson
So what you think about the possibility that we could use this strategy for documenting the outcomes of workplace learning activities? Imagine, for example, that a learning “transcript” inside an organization would document specific skills that have been demonstrated in the workplace. We could put in place criteria-referenced evaluations to be completed by designated evaluators (e.g. managers, peer experts, learning facilitators) based on observation of the skill in the context of day-to-day work. It wouldn’t matter how the skill was developed, although we could offer both formal and informal learning to support the development of the most critical skills in our organization.
This is another way to think about competency evaluations in organizations. One of the most frustrating things about annual competency reviews is that competencies are defined at such a high level that different evaluators provide completely conflicting ratings. If, however, we broke the skills down to a more nuanced level, we might actually be able to develop a better picture of what skills a particular employee has developed.
An example, and a possible can of worms
Let’s look at an example close to home. Let’s say we want to know the degree of skill an employee has in the area of conducting needs assessment.
> On a learning transcript, we might find a course on needs assessment – but what exactly did the employee learn? And did he or she ever successfully apply the specific concepts and strategies offered in the course?
> On a competency review, the skill of needs assessment is likely evaluated in one line, even though that skill is complex and multi-faceted. How a professional is evaluated may depend on how challenging his or her needs assessment projects were that year and how expert the evaluator is in the subject.
> With a badging system, we may be able to evaluate separately different aspects of needs assessment: for example: assessment planning, job analysis, focus group facilitation, qualitative data analysis, quantitative data analysis, survey construction, performance analysis – the list of specific skills goes on. And there’s a possibility of creating meta-badges that roll up a longer list of skills, as well as a possibility of “leveling up” which would document increasing levels of skill. With a badging system, our learning records could be more nuanced, but the system of documenting learning would be more complex.
Why consider an alternative?
Our systems of tracking learning don’t hold up when we move to a more informal learning strategy. Our competency evaluation systems don’t seem to be nuanced enough to be helpful, and they certainly aren’t consistent across multiple raters. We all agree, I think, that what is really important is whether or not employees “know” something – and more to the point – whether or not they are capable of “doing” something (knowing at what level of proficiency would be a nice bonus). A badging strategy may get us there.
The conversation around using digital badging as a learning record is interesting on many fronts. On a global scale, there are tons of problems to solve – establishing the quality of the badge issuer’s process, preventing gaming the system, measuring discreet skills without overwhelming a “badge backpack” with too many details, promoting badging as evidence of knowledge and skill, and more. And in the end, maybe badging is just the “degree” and “certification” dressed up in new clothing.
The conversation continues. What do those of you in corporate learning and development environments think about this hot topic?