Journal of Medical Sciences and Health
DOI: 10.46347/jmsh.v9i2.22.445
Year: 2023, Volume: 9, Issue: 2, Pages: 146-151
Original Article
Reeta Dhar1 , Almas Dalvi2 , Shilpi Sahu3 , Manisha Tambekar1 , Bansi Kotecha4
1Professor, Department of Pathology, MGM Medical College, Navi Mumbai, Maharashtra, India,
2Senior resident, Department of Pathology, MGM Medical College, Navi Mumbai, Maharashtra, India,
3Professor and HOD, Department of Pathology, MGM Medical College, Navi Mumbai, Maharashtra, India,
4 Junior resident, Department of Pathology, MGM Medical College, Navi Mumbai, Maharashtra, India
Address for correspondence:
Almas Dalvi, Senior resident, Department of Pathology, MGM Medical College, Navi Mumbai, Maharashtra, India.
E-mail: [email protected]
Received Date:06 October 2022, Accepted Date:27 March 2023, Published Date:30 August 2023
Introduction: Objective Structured Practical Examination (OSPE) is an effective method for formative and summative assessment of undergraduate students during practical examination. Aim: The aim of the study was to study the effectiveness of OSPE in the subject of Pathology and to evaluate them on the basis of these parameters, provide feedback on performance, evaluate on basis of practical skills and measure minimal competences. Materials and methods: This study was conducted among a total of 120 M.B.B.S. students during their second year of MBBS course. OSPE based practical examination was carried out followed by a close ended questionnaire to evaluate student’s perception regarding the same. Results: OSPE was seen as a useful exercise by 83 % of the students. Feedback from the undergraduate students put forward that 17 % students felt that OSPE was an objective tool in evaluating practical skills. Conclusion: OSPE is a more valid and beneficial learning tool according to the students as well as faculty.
Keywords: Objective structured practical examination, Pathology, Medical education, Teaching tool
Testing, measurement and evaluation play an important role in all educational institutions, including medical undergraduates. Evaluation tools have far-reaching consequences for students in their success or failure, consequently educators have the responsibility for development of testing devices or procedures that fairly evaluate student’s achievements and yields accurate results. Numerous methods for evaluating the students as a whole are available, however only a few methods to assess individual domain of learning such as knowledge, comprehension, psychomotor skills, and ability to communicate are present. 1
Objective Structured Practical Examination (OSPE) is a new pattern of practical examination. In OSPE each component of ethical competence is tested uniformly and objectively for all the students who are taking up a practical examination at a given place. OSPE was derived from OSCE (Objective Structured Clinical Examination) and was revised by Harden and Gleeson which assesses practical competencies by observing the performance of the students directly 2. It can be used for formative and summative evaluation.
Objective: Examiners use a checklist for evaluating the students.
Structured: Every student sees the same problem and performs the same task in the same time frame.
Practical: The tasks are representative of those found in real clinical situations.
An examination.
In 1985, at the International conference at Ottawa medical education scholars from all over the world debated and exchanged their experiences with OSPE 3. OSPE was considered to be an objective, reliable, feasible, effective, and acceptable medical education tool. From 2019 onward, the regulatory body for medical education. The Medical Council of India (MCI) has introduced competency-based medical education (CBME) curriculum. 4
To study the effectiveness of OSPE in the subject of Pathology in II MBBS students and evaluate them on the basis of these parameters.
Provide feedback on performance.
Evaluate on basis of practical skills.
Measure minimal competences.
OSPE module was introduced for 120 students during formative examination (term two - September 2021) graduate II MBBS students for internal assessment. The present study was conducted during the period of September 2021 to August 2022 for a period of one year.
All the students who were willing to participate in the study.
120 Students of II MBBS batch.
The present study was conducted during the period of September 2021 to August 2022 for a period of one year. The 120 students during exams were divided into 2 batches. Batch 1 was subjected to conventional examination and batch 2 to OSPE.
A written informed consent was taken from all students who participated in the study after the institutional ethical committee approval was obtained.
The 120 students were divided into two batches.
Batch 1: 60 students were subjected to Traditional/Conventional method
Batch 2: 60 students were subjected to OSPE
Examination was conducted as three sessions per day with 10 students in each session. During the OSPE, each student was made to rotate through 12 stations, 5 minutes for each station.
Out of 12 stations, 2 of them were performance stations while 8 were response stations and 2 rest stations. Observers with uniform check list and score card were present at the performance stations to grade the student's step wise performance based on the procedure. The exercises kept for these observed stations included blood grouping, peripheral blood smear preparation or staining, haemoglobin estimation or chemicals tests of urine analysis that were aimed at analysing the psychomotor domain of the students. At the unobserved stations, students were provided with objective questions regarding an instrument/microscopic slide/chart with clinical history and laboratory investigations/gross specimen that tested their practical knowledge. Two additional rest stations were provided after the fifth and last working station respectively labelled as ‘blank’ for students to complete the answer sheet (05 minutes each).
In the conventional assessment method, which was conducted for the other 60 students of Batch 1, each student performed a particular practical procedure which was followed by viva-voce and overall judged by the allotted examiners.
Pre-test, post-test and feedback was taken to assess and evaluate the learning improvement and effectiveness.
The questionnaire was validated by third party from within the department (internal) and expert faculty from another medical college (external). Students were provided with a standard questionnaire and asked to carefully read the questions, contemplate and then tick the option they find as the most suitable. They were not permitted to discuss amongst each other during these 20 minutes’ exercise.
Basic statistical analysis of the responses of the students in the feedback questionnaire were presented as percentages and then assessed.
Department of Pathology, Mahatma Gandhi Mission’s Medical College and Hospital, Kamothe, Navi Mumbai.
83% of the students found OSPE quite helpful while 17% found it to be barely helpful. (Figure 1)
83% of students found the that OSPE aids in the process of learning, none of them rated the method of no utility however, 4.3% were indecisive and 7.54% found it to be barely helpful.
Out of all, more than half of the students (54.47%) agreed that adequate time was given for the session while the rest did not approve of the time limit for the exercise. Almost three quarters of the students could complete the stations within the allotted time whereas, about 25% of the students found it difficult to complete it with the time limit. About 94% of the participants were of the opinion that the OSPE stations were well-organized while 6% of them didn’t agree. (Figure 2)
The analysis of instruction for each OSPE station is as shown in the Figure 3. Approximately 34% of the students stated that they were able to perform the procedure well after receiving instructions. 1.8% of the students neither agreed nor disagreed regarding instructions aiding in their performance while 11% disagreed about instructions proving to be helpful. 45.28% could comfortably perform the given exercise while as much as 1% of the participants were of the opinion that they were not comfortable to perform the exercise.
Rating its difficulty level, approximately 43.39% of students found it to be easy, 34% of students found it neither easy nor hard and around 23% students believed it to be very easy. When asked about improvement in the current settings, the most common answer received was that the students should have prior information about OSPE as a method of learning and the next commonly received answer was allotment of more time.
To conclude, the students felt that OSPE was a helpful method and created interest towards the learning of the topic as well as it was helpful in self-assessment, an easy, interesting and quick method for revision of the topics.
89% of the students were of the opinion that the study was useful, 4.84 % found it to be neither helpful nor not helpful while 6.45% found it to be barely helpful.
About 60% of the students agreed that this method was unbiased. About half the total number of students found that the time given was not adequate and couldn’t satisfactorily complete the stations. About 90% of the students believed that questions asked were covered and related to the topic. 90% of the students were of the opinion that OSPE was well-organized. On contrary, 9.67% of the students believed that it wasn’t organized properly. (Table 1)
69% of the students were clear about what is to be done, 11.29% students were indecisive while 6.45% of the students did not understand as to what is to be done. About 53% of the students believed that OSPE was less stressful than other exams, 24.19% students were indecisive and about 5% the students found it stressful.
53% of the participants found that it was easy to perform practical after OSPE exercise, 35.48% participants found it neither easy nor difficult and 6.45% found difficult to perform practical after OSPE. (Figure 4)
When asked about improvement in the current settings, the most common answer received was that the students should have prior information about OSPE as a method of learning and the next commonly received answer was allotment of more time.
|
Questions asked post OSPE session |
Feedback given by students |
|
|
Yes |
No |
|
|
Was Adequate time given? |
48.38% |
51.61% |
|
Were the questions covered during teaching activity? |
90.32% |
9.67% |
|
Was OSPE well-organised? |
90.32% |
9.67% |
Researchers all across the globe have modified OSPE as a teaching and learning tool with a better educational impact 5, 6, 7. In Pathology, several psychomotor skills are taught in procedures such as haemoglobin estimation, urine examination, renal function test, body fluid examination, blood grouping, histopathology and haematology slide interpretation and require to be evaluated. When skill assessment is done by the traditional method that has been in practice since many years, it is found to be subjective in nature. There is an apparent lack of scope for direct observation of the performance of skills during the entire procedure.
Some of the premier medical institutes in India have initiated use of OSPE in assessment of psychomotor skills. OSPE has been considered to be a reliable tool by many studies conducted, which helps in evaluating the practical skills of medical students. It takes into account knowledge of the students as well as their attitude towards education. Besides, it is also known to remove any bias by the examiner for different students. 8, 9
In a study by Kundu et al. it was found that 99 % of students believed that OSPE was helpful. 81% of the students felt that OSPE fits both as a learning and evaluation tool 10. This was in accordance with the results of our study.
Charles et al. suggested that OSPE was more objective, uniform, focused, and a fair assessment for the student, which in turn enhances their confidence, improves academic performance, encourages learner feedback, lessens the stress levels among the young minds and also can decrease the total time required to finish the practical examination similar to the findings of the present study. 11
Nigam R et al. concluded that OSPE was effective as an assessment tool compared to the conventional practical examination which is correlating with the findings in the present study 12. A study by Trivedi et al. also suggested that OSPE was a better assessment tool with the students having a higher chance to score more marks.13
A study conducted by Wani et al. study revealed that OSPE can be executed for the entire haematology practical session so as to make sure that all the students get identical questions with comparable difficulty levels. 14
In the present study, 53 % of the students stated that the level of stress in OSPE was less as compared with the traditional practical examination, which is similar to a study conducted by Wadde et al. 15
According to the students, the structured checklist pattern in OSPE helped them through the different steps and also enabled identification of their weaknesses thereby creating a learning stimulus 16, 17. On scrutinising the answers given by the students in the OSPE session, the most common mistakes were identified. This was used to recognize the weaker areas in a particular topic and more emphasis was given in the subsequent lectures.
In the near future, OSPE may be used as a device for assessing multiple dimensions of a post graduate student, including steps in performing procedures, identification of molecular techniques, biomarkers, interpretation of results and reaching a diagnosis. It could also prove beneficial to judge the skills acquired by the postgraduate student during the course of the study. 18
During the study, the feedback received at the end of each practical session offers reasonable evidence that OSPE is a satisfactory and beneficial teaching as well as assessment tool for defined skills in practical examination. It also eliminates inter-examiner variation and bias. It is a useful method to minimize the variations in subjectivity and enhancing objectivity while performing practical procedures, such as haemoglobin estimation, blood grouping, urine examination, etc. When used as a formative tool, it will help in modifying strategies for a better understanding of the subject by the students. OSPE as an appropriate assessment tool proves to be an integral part of teaching medical undergraduate students and the CBME curriculum.
Ethical approval: Approval from the Institutional Ethics Committee for research was taken before the conduct of the study dated 15/04/2021 (Approval no. N-EC/2021/04/40).
Subscribe now for latest articles and news.
