Challenge #18: Create a Plan for the Dissemination and/or the Diffusion of the Interventions
Artifiact: EDCI 528 HPT Performance Assessment
Reflection:
My artifact for this challenge is a report that is addressed to stakeholders/the client regarding a performance assessment for Company X. For context, founded in 2015, Company X is a small scale-up organization in Canada in the education technology industry. My performance assessment report describes the performance system within Company X, goals of company X as well as the goals for the intervention initiative, identification of drivers of performance gaps within Company X, and finally, potential barriers and supporters.
Performance Issue
There are two key performance issues that I analysed for Company X. The first, is an instructional issue, which is a result of training inefficiencies causing poorer training retention resulting in an increase in student application errors. As a result of this instructional issue and directly correlated non-instructional issue is a spike in poor customer experience feedback resulting in frustrated and unhappy customers. In my performance assessment, I focus on two goals that are critical to the success of Company X. The first goal is to increase employee training retention and results to 85% from 65%. As its currently standing, senior employees spend weeks delivering solely ILT-based training and then delivering retraining (if needed) but it appears to be ineffective as employees are making continuous repeated mistakes, which are found later through application audit checks. This is a concern for the company for several reasons such as:
Increased number of applications with errors
Creates backlog of applications due to the additional time spent on fixing errors + reduces the number of employees that could be doing the job
Backlog of applications results in longer waiting times for customers
Increase in application errors and longer waiting times results in overall poor customer satisfaction and experience
The second goal is to increase customer satisfaction scores as it currently sits at 65%. As found in a recent customer feedback survey, a large majority of customers were unhappy with the level of errors and mistakes found in their applications. The goal is to increase the customer satisfaction score to 75% or more.
Proposed Interventions and Impact
In my report, I proposed several interventions to address these organizational performance challenges. These interventions initiatives include:
Conduct Root Cause Analysis and Streamline Content. It is possible that training may be overloaded, outdated, or misaligned with what employees need to know to do the job. This intervention in my plan for dissemination and diffusion is appropriate for the needs and culture of Company X because it focuses on addressing inefficiencies by diagnosing whyperformance gaps exist, not just treating symptoms. This supports the culture of the company for various reasons such as:
It supports a continuous improvement mindset
It focuses on building trust so that employees know their time and efforts are respected
It encourages transparency in identifying what's not working
Shift to a blended learning approach with scalable digital resources. Relying solely on Instructor-Led Training (ILT) drains time between both the senior employees/trainers and the trainees as it’s not a scalable approach to support independent performance once the training meetings end. Additionally, this allows Company X to align to different learning styles and opens the opportunity to curate digitized content that is available in various languages to meet the needs of a global audience. Note: As a result, this reduces dependency on senior employees/trainers and improves knowledge retention and reference with ongoing on-demand reinforcement. This intervention in my plan for dissemination and diffusion is appropriate for the needs and culture of Company X because it focuses on reducing reliance on senior staff for every knowledge transfer moment and it enhances accessibility, which is great for hybrid, remote, and global teams, like Company X. This supports the culture of the company for various reasons such as:
It promotes autonomy and ownership of learning
It encourages and promotes digital transformation, which is a new norm
It supports inclusive environments by accommodating different learning styles and/or languages
Integrate Post-Training Performance Support & Feedback Loops. Without feedback or coaching, learners can’t calibrate their performance, and mistakes will continue to persist. Note: Feedback channels convert training into continuous learning opportunities rather than one-off events. This intervention in my plan for dissemination and diffusion is appropriate for the needs and culture of Company X because it reinforces learning through real-time correction and guidance, allows feedback, which promotes skill refinement and behavior change, and creates the possibility of opportunities for coaching and mentorship within the organization. This supports the culture of the company for various reasons such as:
Fosters a feedback-rich environment and continuous growth
Reduces fear of failure, where mistakes become learning moments or “aha” moments
Challenge #19: Evidence of implementing a formative evaluation plan to provide information that can be used to make adjustments and improvements in the design. Artifiact: EDCI 528 HPT Performance Assessment Reflection:
In EDCI 528, I completed a performance assessment for Company X and subsequently built an HPT Solutions and Change Management Plan and an Evaluation Plan for the organization. This assignment was based on the experience and knowledge I had of the organization and its processes while I was employed there. This plan could be tweaked and applied to any other organization; however, it has not been fully implemented yet as no such opportunity has emerged.
Formative Evaluation
I have designed a comprehensive evaluation plan that I could use if I were to implement this in an organization to address an existing HPT problem. The formative evaluation in my plan spans several phases from needs analysis, design and development to pilot testing and post-launch and ongoing feedback collection. My formative evaluation plan is designed to ensure that I can capture the feedback of both the trainees and trainers throughout the training development to ensure I design a training program that meets their needs and is tailored to address the learning and performance gap. While real data collection has not yet occurred, the evaluation tools required, and the plan are in place once a program enters the implementation phase. The questionnaire I developed (refer to appendix of my performance assessment plan) is structured to collect data in key areas such as the clarity, accuracy, and relevancy of the content, trainee engagement and attitudes, and trainee preparedness post completion of the training to ensure it helps with completing on-the-job tasks and learners are motivated to use it. Once implemented, the tool will provide actionable feedback to refine the training program and/or instructional products. Since the formative evaluation plan has been developed but not fully implemented, there are no tangible results to present at this time. However, the evaluation plan is in place to ensure once the feedback has been gathered that it will inform the changes or revisions that need to be made to the training design.
Reflection of Previous Experiences
I have not had previous experience with developing a comprehensive evaluation plan or with developing extensive formative evaluation tools in previous roles. However, as part of my on-the-job projects I have had the opportunity to conduct formative evaluation to collect feedback to make enhancements to the instructional product or training that I am developing. This has occurred in two ways – 1. I am re-designing a program e.g., new hire onboarding, so for this I conducted a formative evaluation focus group discussions where I asked targeted questions to gather feedback and then during development collected feedback and 2. I design and develop an instructional product e.g., eLearning course and gather feedback in two stages/rounds of feedback to inform my design choices and overall learning experience. Through this assignment I gained a better understanding of how to evaluate an HPT problem and write a change management and evaluation plan. The hope is to utilize these tools in the workplace and apply this learning to my future roles.
Challenge #20: Evidence of implementing a summative evaluation plan to evaluate the effectiveness of the instruction and decide whether to continue to use instruction.
Artifiact: EDCI 577 Kirkpatrick Evaluation Plan Reflection:
EDCI 557 was designed to have a partner to complete the project where we had to work together to develop an Evaluation Plan for an organization of our choice. I worked with Cindy Malerba, and we collaborated for the creation of the project. Cindy worked at a hospital, specifically in Breast Cancer Education and Support and we chose to do our project for a program that she was leading at the time. We did not use the real name of the program and in the documents we refer to it as the Breast Cancer Education and Survivor Support Program (BCESSP). In the plan we outline general background information about the program, provide a detailed instructional product analysis, an overview of the audience and instruction context, and a comprehensive evaluation of the instructional product using Kirkpatrick’s four levels of evaluation.
Summative Evaluation Plan
Although the Evaluation Plan is fully built out, it was not implemented, so it was not possible to fully realize whether the evaluation process met expectations in practice. However, the evaluation plan was carefully designed to align with the training objectives and included appropriate tools such as post-training assessments and feedback surveys to measure the effectiveness and impact of the program. However, if implemented, the plan would provide data on whether learners achieved the intended learning objectives and whether the training justified continued investment. It would have also helped with making modifications to the program or discontinue it based on effectiveness. I know my colleague Cindy recently submitted a grant application, and she informed me that she used parts of the Evaluation Plan as part of the grant application, so it is not that the plan is completely untested.
Reflection of Previous Experiences
I have not had previous experience with developing a comprehensive evaluation plan in the past. In previous roles, unknowingly, I have integrated Kirkpatrick Model’s levels 1 and 2 in my summative evaluations, however, this class taught me what the model is and how to truly apply it to a workplace training program. I hope to continue to apply the learnings from this class to future work projects to realize the full effectiveness of the instructional products or training programs that I do design.