Building a Better CME Program: A Guide to Needs Assessment, Learning Objectives, and the Kirkpatrick-Barr Model

A needs assessment should be the first step in any educational program development process. It enables identification of the educational activities that need to be developed in order to improve physician competence.

It is a systematic process for identifying any gaps between the current and ideal states of physician competence. The process begins with a review of the literature and existing data sources.

We then identify experts in the field who can help us refine our understanding of the gaps and build potential bridges. These experts may be physicians, other healthcare providers, or patients and clients.

Once we have a good understanding of the lacunae in physician competence, we develop educational objectives to address them. We then design educational activities that will allow physicians to achieve the objectives and close the gaps.

These activities may include didactic sessions, small group discussions, case studies, or simulations. We also develop methods for assessing whether or not the educational activities have been successful in improving physician competence.

When it comes to creating measurable learning objectives, many CME planners turn to Bloom’s Taxonomy of Measurable Verbs. This handy tool provides a list of verbs that can be used to create objectives at each level of the taxonomy, from the most basic level of knowledge to the highest level of critical thinking.

One of the great things about using Bloom’s Taxonomy to create learning objectives is that it forces you to think about the level of complexity you want your learners to reach. For example, if you want them to simply remember a list of facts, you would use lower-level verbs such as “know” or “define”. However, if you want them to apply that knowledge to a real-world situation, you would use higher-level verbs such as “analyze” or “evaluate”.

By using this list as a guide, you’ll be sure to craft objectives that are clear, measurable, achievable, and appropriate.

Remember:

Understand:

Apply:

Analyze:

Evaluate:

When planners are improving their training programs, it is important for them to have a framework in place to guide the process. The Kirkpatrick-Barr model is just such a framework, which aids in evaluation of the effectiveness of training. It was developed by Dr. Donald Kirkpatrick and Dr. James Barr in the early 1970s.

The Kirkpatrick-Barr model is based on four levels of evaluation:

The first level of evaluation is reaction. This is simply a measure of how participants reacted to the training.

Did they like it? Did they find it helpful? Were the materials and instructions clear? This level of evaluation can be gathered through surveys and interviews.

The second level of evaluation is learning. This is a measure of what participants learned from the training. It can be evaluated through tests and quizzes.

It is important to note that reaction and learning are not the same. Just because participants liked the training or found it useful does not mean they learned from it.

The third level of evaluation is behavior. This is a measure of whether participants are actually using what they learned from the training in their everyday work.

This can be evaluated through observations and interviews. It is important to ensure that participants are using the knowledge and skills they have learned in their workplaces.

This is a measure of whether the training had a positive impact on the organization as a whole. This can be evaluated through key performance indicators (KPIs) and other measures of success. It is important to ensure that a training program is actually of benefit for the organizations of trainees.

The Kirkpatrick-Barr framework is a valuable tool for evaluating training programs. It can help organizations determine whether a program is effective, and identify areas for improvement.