Analyzes Change (C11), Program Analysis (C12) and Program Improvements (C13) are intended to capture a provider’s self-reflection of their continuing medical education (CME) program and their overall impact on their learners. These criteria follow the plan-do-study-act cycle for continuous improvement.

Analyzes Change – Activity Level

In all accredited activities, CME providers must gather data about the changes that their educational intervention has had on learners’ competence, performance or patient outcomes. This data can come from multiple sources such as audience response, evaluation, hospital data, follow-up surveys or interviews, or onsite reviews where change in practice is demonstrated. Regardless the method of gathering data, it is important that the data captures what has or will change, rather than if a change may occur.

The data that is gathered at each activity then becomes the foundation blocks on which Analyzes Change, Program Analysis and Program Improvements are determined.

Accredited CME providers should periodically review all the data they have gathered from all the activities held in a time period. It is an organizational choice how often they reflect upon their CME programs impact, but it must be done at least once per accreditation cycle.

Analyzes Change Program Level

The first step in the overall program improvement criteria is to study and analyze the data collected from all your activities to see if educational interventions are making an impact in closing the educational gaps and improving changes to practice in your learners. As a program, you may consider these questions:

1. Are we making a change in our learners’ competence? How does the overall program data show this?
2. Are we making a change in our learners’ performance? How does the overall program data show this?
3. Are we making a change in our learners’ patient outcomes? How does the overall program data show this?

Use these questions to help make conclusions as to whether or not you were able to change learner competence, performance or patient outcomes across your overall program of accredited activities. By analyzing learner change across the full range of your CME program periodically, you will have the information you need to respond to the Analyzes Change in your self-study.

Program Analysis

The second step in the overall program improvement criteria is analyzing your overall program data to see if the educational interventions you planned were in-line with your current mission statement. This should be done at that same periodic review as Analyzes Change at the program level. However, instead of thinking about change in learners’ practice, you now think about your CME mission’s expected results in terms of competence, performance and patient outcomes. Did you hit the mark, or perhaps you strayed off your mission? Were you successful in doing what you said you would do? Be honest with yourself. It is okay to say that you were unsuccessful in achieving what you set out to do in terms of expected results of your educational interventions regarding competence, performance, or patient outcomes; however, you must clarify why. You must come to a conclusion as to the degree to which your activities/ educational interventions come to meeting your CME mission.

Program Improvements

The final step is using the analysis and conclusion of change in learners from Analyzes Change, and the conclusion of meeting your mission statement in Program Analysis, and now make plans to improve your CME program. These improvements can be anything from the structure of activities or revising your mission statement to implementing time-saving strategies for CME staff. However, you must include the elements of identification, planning and implementation of changes during the accreditation term. Big or small, CME programs can always find ways that they can continually improve. Make sure to track these improvements. During the pressure of a submitting reaccreditation materials, you may sometimes forget changes made in the beginning of your term, because they have become ingrained over four to six years.

When determining your periodic review timeframe, keep in mind that at least one resulting program improvement should be implemented and reported on at time of reaccreditation material submission. While this cycle of improvement must be done at least once per accreditation term, regular and ongoing continuous self-reflection is encouraged.

As always, reach out to cme@cmadocs.org if you have any questions.