PQI: Perkins Institute

 

 Case Study:  Applying the 5-Step Model

 

 

 

 

 

APPLYING THE PROGRAM IMPROVEMENT PROCESS

 

NORTH STAR COLLEGE

 

 A MnSCU CASE STUDY

Work Session Break-Out 2

 Steps 3 Through 5

 

Thursday, January 23, 2003

7:30 am ? 9:00 am

 

           

1. Review/revisit yesterdays work????????????????10 minutes
2. Step 3: Identify best solutions????????????????...30 minutes
3. Step 4: Develop evaluation strategy??????????????..15 minutes
4. Step 5: consider implementation solutions????????????15 minutes
5. Fill out self-assessment form for steps 3-5????????????10 minutes
6. Prepare report-out?????????????????????...10 minutes

 

 

Recorders should be prepared to provide a 3-5 minute summary of the group?s deliberations and decisions during the report-out session.

 

 

 

 

 

 

 

 

 

 

 

 

Step 3: Select Best Solutions

 

Once you have identified the most critical root causes to address in your improvement effort, the next step is to identify and select the solutions that seem most promising for testing and evaluation. Consult pages 20-23 of the guidebook for assistance with this step. 

 

To focus our work, in the space below write in the single most critical root cause that you believe affects student performance.

 

 

 

3.1  Arraying the Solutions

Based on your own experience and review of the literature, identify between 3 to 5 possible solutions to the root cause you identified.

 

 

 

 

 

 

 

3.2  Narrowing the Choices

Select the most promising of these potential solutions and answer the following questions about your choice. Review pages 21-22 of the Guidebook for a list of selection criteria.

 

a.       How does your solution address the primary root cause you?ve identified?

 

b.   What evidence do you have that your solution might work?

 

c.   What other types of data might you use (e.g., student satisfaction surveys) to make your decision?

Step 4: Pilot Test and Evaluate Solutions

 

Now that you have identified a set of promising solutions on which to base initial improvement efforts, you?ll want to create an evaluation strategy that will allow you to assess how well the improvement strategies and models are working. This section will assist you in selecting practical evaluation designs and analysis tools that you can use to gauge the success of your improvement efforts. Consult pages 24-29 of the guidebook for assistance with this step.

 

 

4.1  Choose a Study Design

Listed below are three study designs you might consider when structuring your evaluation. Please circle the one that you will implement and remark on why you chose this design.

 

·   Option A: Random Assignment with Control Groups?randomly assigning students into treatment and non-treatment groups to ensure comparability among groups.

·   Option B: Comparisons with Similar Populations?compare student outcomes by taking into account the influences of other factors on performance.

·   Option C: Comparing Individuals Against Themselves?comparing pre- and post-tests of student performance or other measures that compare outcomes prior to, and following participation.

 

 

 

 

 

 

 

4.2  Select Pilot Programs

While it is tempting to implement your program improvement strategy college wide, it is advisable to select a subset of programs in which to pilot-test ideas. Which programs will you pilot the strategy? How will you choose these programs?

 

 

4.3  Select Outcome Measures

To help track performance changes, develop two short- and long-term measures to provide some indication of the success of your improvement efforts.

 

List two short-term measures that focus on immediate results that must be achieved to eventually obtain results on the Core Indicators.

 

 

 

 

List two long-term measures to provide direct evidence of your success in improving performance on the Core Indicators.

 

 

 

 

 

 

 

 

 

4.4  Identify Data Sources

After selecting short- and longer-term outcome measures, you will need to identify data sources and collection instruments that will allow assessment. Consult the list on page 27 of the Guidebook and identify some methods of data collection you may wish to consider.

 

 

 

 

4.5  Train Pilot Program Staff

Since the outcomes of your improvement effort will depend upon it being successfully introduced, you will want to spend some time with pilot program staff reviewing improvement purposes.

 

Who will work with pilot program staff to coordinate improvement efforts?

 

 

 

What types of supporting materials will need to be developed?

 

 

 

 

How will implementation be assessed?

 

 

 

Step 5: Implementing Solutions

 

Congratulations! If you?ve come this far it is likely that the solution you selected in Step 1 was successful in increasing student performance in pilot programs. As you prepare to expand the pool of participants, it is a good idea to come up with a monitoring process that will allow you to obtain ongoing feedback on the improvement strategy. This section will help you to develop implementation plans to monitor outcomes across the full range of implementation programs.  Consult pages 31-32 of the guidebook for assistance with this step.

 

 

5.1  Moving from Pilot Testing to Statewide Implementation

Replicating small successes on a larger scale can be complicated by the addition of new factors and administrative challenges that can reduce program effectiveness.

 

Are there any unique characteristics of your pilot programs that will make it difficult for you to expand your efforts to other programs? If so, what are they and how might you plan around them?

 

 

 

 

What strategy will you use to roll out your efforts across new programs?

 

 

 

 

 

 

5.2  Monitoring Ongoing Implementation

Expanding your improvement efforts requires developing a comprehensive strategy for monitoring the implementation and evaluation approach.

 

How will you collect information from all programs to ensure that that your intervention efforts are being adopted effectively?

 

 

 

 

 

APPLYING THE PROGRAM IMPROVEMENT PROCESS

 

NORTH STAR COLLEGE

 

 A MnSCU CASE STUDY

Work Session Break-Out 2

Steps 3 Through 5

Thursday, January 23, 2003

7:30 am ? 9:00 am

 

Summarizing Group Findings

 

STEP 3:  SELECT BEST SOLUTIONS

 

Are your solutions based on a sound theory of root causes (step 2)?  Have the solutions worked under similar or comparable circumstances?  Is the evidence strong and compelling?

 

 

 

 

STEP 4:  PILOT TEST AND EVALUATE SOLUTIONS

 

What study design(s) did you choose to test your solution(s)?  How did you select your pilot sites, short- and long-term outcome measures, and identify your data sources?

 

 

 

STEP 5:  IMPLEMENT SOLUTIONS

 

How did you decide to move from pilot testing to full college implementation?  How will you monitor ongoing implementation and work to sustain your improvement efforts?