by Salvador Ros
(leer en español)
The cooperative learning methodology based on the research team-work method that has been implemented in the classroom (the ten stages procedure explained in former articles) has, as a final stage, the evaluation of the methodology itself.
This last stage is crucial due to it allows us, in an analytical way, to value the results, to come to a conclusions and to propose improvements; aspects that are central in order to carry out a field investigation and innovation project which consist of learning from the experience and using it to innovate.
There is a parallelism between the methodology that teachers implement to develop this educational project and the cooperative learning methodology used in the classroom. In fact, we teachers, as a researchers, are using tools that are common to those used in the proposed system for students. A success condition when working is to believe in what you do. Using a method whose effectiveness we trust makes easier to transmit enthusiasm to students because we apply in the classroom the same method.
The question is:
Why do we trust so much in the efficacy of the method?
Months ago, when I started to research in order to develop the teaching methodology, one of the most relevant references I met was the Delors report where the commission concluded with the four pillars of education.
- P1: Learn to know
- P2: Learn to do
- P3: Learn to live together
- P4: Learn to be
Cooperative learning, and specifically the formula used herein, is intended to students acquire the four pillars.
Firstly, learn to know means to have achieve mastery of tools that ease learning or, put another way, to have learned to learn. Research team-work makes individuals learn alternative ways of creating knowledge through interaction with peers.
Secondly, learn to do involves the acquisition of skills, which are reached by carrying out tasks that imply to test their talent, such as the proposed activity of research.
Third, learn to live together suggests to take awareness of other human minds, come to understand and respect different points of view, develop empathy, respect and acquire skills to resolve conflicts. In this regard, one of the things that has most impressed me during the experience is the improvement of the relationships among students. In some of the teams, there were previous situations of antagonism that have been happily solved as a result of the team-work, sometimes mediated by the teacher. Also, the activities have sometimes generated tensions among students, some of them difficult to manage, but today I can say that the atmosphere in the classroom is quite better than three months ago. Being aware of their happiness, their calmness, optimism, trust in the teacher and involvement in the learning contents turns out a really worthy experience for a teacher. This paper aims to offer the gathered data, their analysis and to come to conclusions, and that is what I am going to do. However, I could not help but remark the above experiences because their relevance and the fact that a system for their quantification had not been forecasted. Therefore, the data will not refer to the satisfaction that the students and I have perceived. A conclusion I have come to is the need of exploring ways to quantify the mentioned satisfaction. Though, considering the sentence of sociologist William Bruce Cameron: “not everything that can be counted counts, and not everything that counts can be counted”, some perceptions, by its nature, may be better describe them rather than quantify them.
Finally, learn to be is related to the development of the individual. The knowledge of oneself and reflection about the relationship with others are fundamental. The stage 9 of the procedure consist of a self assessment and the a peer assessment to compare the the results. This stage not only reveals the achieved learning outcomes but also fosters reflection on the own behaviour and the personal contribution to the team-work. Students know that their work will also be quantified by their peers. This situation, in my opinion, encourages students to objectivize themselves.
The assessment is carried out giving a score between 0 and 3 to the LO (Learning Outcomes). Let us review the 10 LO and connect them to the four pillars:
- LO 1: Provide with good information sources. (P1)
- LO 2: Contribute to synthesize information and make it more understandable and organised. Offers useful selection criteria. (P2)
- LO 3: Contribute to the team not disperse and lose track of the target. It focuses on the scheduled task for each stage. (P3)
- LO 4: Provide with solutions for obtaining the best results based on the evaluation criteria of the project or final result. (P2)
- LO 5: Use English and does so in a sympathetic mode towards those who have difficulty for understanding. (P2)
- LO 6: His/her attitude encourages the team members to use the English language. Teaches how to use resources or strategies for language learning. (P3)
- LO 7: Recognize mistakes and acts accordingly. (P4)
- LO 8: Contribute to team cohesion with communication and mediation. (P3)
- LO 9: Propose workable solutions to problems. Offer good ideas when planning how to carry out the tasks. (P2)
- LO 10: Adequately understand the research methodology and proposes appropriate actions when blockages or uncertainties arise. (P1)
It has to be noted that the activity of assessing relates to P4.
The first objective of this activity, which is to be part of the tools for the evaluation of each student, has been fulfilled. In addition to the scores, the most relevant result is the difference between self and peer assessments. This difference leads to overvaluation, if the self-assessment score is higher than that given by teammates, or undervaluation, if the opposite happens. Thus, the strength of this tool is that it provides a measure of objectivity of each student about their personal labor. Furthermore, the results seen together characterize the group.
Data from two class-groups of students are offered below: a group of 16 students of Graphic Design distributed into 4 teams of 4 members each (of which 13 students participated in the survey) and a group of 6 students of Decoration Project Management on two teams of three members.
The percentage of overvaluation of each individual is obtained by the difference in points between the self assessment score and the average of the scores of peer assessment. Then the result is divided by the overall average obtained (for calculating this average, the score of self assessment and all of peer assessments are used) and multiplied by 100. If a negative result is obtained, it indicates undervaluation.
% Overvaluation = (score self-assessment – score peer assessment) / (average score) * 100
When characterizing the individual, as a criterion, three ranges of overvaluation percentages in absolute value have been considered, both positive (if overvalued) or negative (if undervalued):
- Less than or equal to 10% → objective individual: Category 1
- Difference between 10 and 20% → individual moderately subjective: Category 2
- Differences of more than 20% → subjective individual or personal perception of reality somewhat distorted: Category 3
Within each category there may be individuals who overvalue or undervalue themselves. The higher the Category, the greater discrepancy between their opinions and the peers.
Obtained results of the Graphic Design group-class: Population of 13 students, 7 males and 6 females.
The overall percentage of overvaluation is 5’41%, which shows that there is not a strong tendency to overvaluation.
Lets see now the results of objectivity classified by ranges of percentages.
The following percentages have been obtained:
- 46% of individuals into Category 1 (differential between 0-10%), 33% of which overvalue themselves and 67% undervalue.
- 31% of individuals into Category 2 (differential between 10-20%), 25% of which overvalue themselves and 75% undervalue.
- 23% of individuals into Category 3 (differential greater than 20%), 100% of which overvalue themselves.
Objectivity is majority in the group, although there are considerable differences since percentages of categories 2 and 3 are significant.
Lets see now the results of overvaluation of the group.
- A) 46% of individuals overvaluate themselves
- B) 54% of individuals undervaluate themselves
In this sense the group is balanced.
The study considering genders gives the following results:
Results of objectivity:
- A) 57% of males were objective (Category 1), 14% of Category 2 and 29% Category 3.
- B) 33% of the females were objective (Category 1), 50% of Category 2 and 17% Category 3.
No major differences were observed as objectivity regards. There are more males in Category 1 (the most objectives) but also in Category 3 (the least objectives).
Results of overvaluation:
- A) 29% of the males overvalued themselves compared to 71% who undervalued.
- B) 67% of the females overvalued themselves compared to 33% who undervalued.
This indicates a more generalized overvaluation in females (67% versus 29% of males). In this case, the data of my students contrast with the latest PISA report “The ABC of gender inequality”. Regarding that, Borja Robert in his article: The education gap is triggered: gender differences between students have increased in recent years (newspaper La Verdad, Friday March 6, 2015) says: “Although (females) showed equivalent skills to those of their peers, they considered themselves less capable.” In our case, the females do not consider themselves less capable because they value their skills more than males do. It is true that the report refers to individuals 15 years of age and our students are in higher education (about 20 years of age).
Obtained results of the Decoration Project Management group-class: Population 6 students, all of them are females.
Results of objectivity:
The overall percentage of overvaluation was 7.8%. 60% were Category 1 and 40% of Category 2. Therefore, no case of subjectivity that leads to distorted perception of reality appeared.
Results of overvaluation:
In this case we have 60% of females who overvalued themselves. The other 40% showed great balance, since the self-ratings scores coincided exactly with those given by the peers. Therefore, there has been a 0% of undervaluation. Also in contrast to the PISA report.
This procedure can be used to characterize the students of our studies. It should be further studies with a larger population and more teachers involved to reach certain conclusions. The results will serve primarily as a first test to evaluate the quantification methodology.
The cases of overvaluation above 20% must be registered and pay attention to the evolution of these students. 3 individual cases have appeared. I have not observed any problem of adaptation to the group, living together or academic results. Therefore, having fluid communication with them should be enough. It is important to note that it was just a test, besides experimental; no more relevance than it really means should be given.
In general terms, the 10 items (Learning Outcomes) of the rubric are adequate. However, their modification is going to be taken into account based on two considerations:
- The possibility of including, for different subjects, items directly related to the subject matter. (Now, only transferable and general skills are assessed. Specific skills are not).
- Changes based on the results of the survey that was handed to students for the assessment methodology (a future article will focus on the results obtained in this survey) will be proposed.
Finally I conclude that, regarding a qualification by the teacher based on the self and peer assessment scores, the results of objectivity are relevant: the global overvaluation is less than 10% (around 5 and 7%). Moreover, is the teacher free from subjectivity?
UNESCO, (1996). La educación encierra un Tesoro. Madrid: Santillana.
O’Toole, G. (2010, May 26). Not Everything That Counts Can Be Counted. [Web log post]. Retrieved from: http://quoteinvestigator.com/2010/05/26/everything-counts-einstein/
Robert, B. (2015, March 6). La brecha educativa se dispara. La Rioja. Retrieved from: http://www.larioja.com/culturas/201503/06/brecha-educativa-dispara-20150306010608-v.html