Full paper
Attempts to sustain economic growth and diversification within the Middle East have seen governments invested in various educational initiatives. In alignment with this ethos, the United Arab Emirates offers citizens, free education in government-funded higher educational institutes, and bursaries linked to grade point average scores.
It is believed that incentivised learning environments provide a powerful motivation to stimulate academic excellence in summative assessments—linked to grade point average scores. An unfortunate outcome of these environments may, however, be that non-grade point reliant tasks are not prioritised by learners. Thus, identifying pedagogical tools which engage student agency towards such tasks, is of obvious interest to educators working within these settings.
One such tool which may engage student agency towards formative assessments, is the digital game-based learning platform Kahoot. Kahoot is known to have permeated many educational domains due to its claimed ability to transform classrooms into fun, competitive environments, where students are engaged and motivated to learn. There is, however, a sparsity of literature evidencing the effects that contextual or demographic influences may have upon this and other digital game-based learning tools effectiveness. As such, an explanatory case-based study situated in the United Arab Emirates incentivised learning environment, was undertaken to evaluate female students’ perceptions of Kahoot, as a formative assessment tool. To achieve this Mwanza's eight-step model for translating data into activity theory components was utilised to develop semi-structured interview questions (Mwanza, 2002). These questions permitted analyses of the social, the individual, and socio-economic structures influencing user engagement with this technology.
Upon introduction of Kahoot as a formative assessment tool, into the incentivised learning context of the UAE, students described contradictions between classroom and institutional rules, community expectations and the divisions of labour. The transformed classroom environments resultant from resolution of these contradictions produced, greater learner collaboration, superior knowledge retention, reduced test anxiety, and increased levels of student satisfaction.
In contrast with other contexts, this study evidenced that digital game-based learning did not predominantly influence student’s motivation to study outside of the classroom for formative assessments. Instead, conventional paper-based assessments provided the greater impetus to engage in out of class learning. This finding may have resulted from the socio-economic and socio-cultural perceptions of the students within this specific learning context.
This study, therefore, urges researchers and educators to undertake shifts in their consciousness to acknowledge socio-cultural and intersubjective factors, which may impact upon the effectiveness of digital game-based learning as a formative assessment tool. This is in recognition of this study’s findings that such technologies, may be heavily dependent upon both contextual and demographic influences of those utilising them.
Keywords: digital game-based learning; Kahoot; formative assessments; Mwanza’s eight-step model; student motivation; United Arab Emirates
Part of the Special Issue Technology enhanced learning in the MENA region
Over the last few decades, governments within the Middle East have recognised the necessity to invest in educational initiatives to nurture long term economic growth, development and diversification (Kamel, 2014). In response to these needs, national objectives and strategies have been developed to sustain the foundations of life-long learning and improve the quality of educational initiatives being delivered (Kamel, 2014). Indeed, the development of human capital through national strategies, in this manner, Kamel (2014) highlights is the best way to ensure a country’s ‘effective and sustainable socioeconomical development’ (p. 104).
It is, therefore, unsurprising that one of the United Arab Emirates (UAE) strategic visions, is to stimulate the creation of an educated and trained nation (Federal Government UAE, 2010). To help achieve this reality, the UAE provides UAE nationals with free education in government-funded higher educational institutes (Embassy of the UAE, 2021), and bursaries linked to student’s grade point average (GPA) scores (FCHS, 2021a). Indeed, it also offers similar privileges to non-national students who are offered tuition scholarships dependent upon GPA attained (FCHS, 2021b; KU, 2021).
Such incentivising of learning environment via the provision of these external rewards is believed to be a powerful tool, capable of encouraging student motivation in achieving learning outcomes; particularly when test results contribute to incentives offered (Cameron, 2001; Putwain, Langdale, Woods & Nicholson, 2011; Sunder & Kitsantas, 2004). However, it should also be highlighted that GPA scores are conversely used by policymakers to ensure conformity to expected standards. As such, GPA scores are not only used to encourage achievement, but also to dismiss poorly performing students, or withholding permission to graduate until specified cumulative GPA scores are achieved (Quinn & Peters, 2017).
In attempts to help aid the learning process (Fry, Ketteridge, & Marshall, 2015) and encourage GPA attainment, formative assessments are often incorporated into teaching and learning strategies within the UAE. This is in recognition of formative assessments ability to enlighten students as to what they must do to achieve target competencies, measured in summative assessments (Lyon, Oláh, & Wylie, 2019). Successful formative assessment strategies, therefore, provide learners with effective feedback on their progress, to enable appropriate remedial actions to be implemented (Black & Williams, 2009; Fry et al., 2015).
Given the power of formative assessments to positively influence student learning, many supporters of these practices have described the need to encourage student agency towards formative assessment processes (Andrade, Wang, Du, & Akawi, 2009; Lyon et al., 2019). Student agency is believed to be evidenced when students review their progress and reflect upon how it was achieved (Lyon et al., 2019). To undertake such actions, however, requires motivated and engaged learners, as student motivation is key to eliciting meaningful learning (Biggs & Tang, 2011) when undertaking formative assessments.
Although it is recognised that incentivising learning may encourage student agency towards summative assessments, it is unlikely to impact upon student’s desire to engage with non-GPA reliant tasks, such as formative quizzes. Indeed, this is supported by claims that the greatest effects of motivation and student agency are produced when incentives are linked to performance (Cameron, 2001; Putwain et al., 2011; Sundre & Kitsantas, 2004). Therefore, identifying pedagogical tools which may assist educators in motivating students' engagement in non-GPA reliant tasks is of utmost importance to educators within these environments.
One way in which student engagement and motivation can be enhanced within the classroom is through the utilisation of digital game-based learning tools (Iwamoto et al., 2017; Morillas-Bario, Munoz-Organero, & Sanchez-Soriano, 2016). One such digital game-based learning platform is Kahoot (Kahoot, 2018). This student response system combines role-play within a game show format, alongside audio-visual aids to enhance the gaming experience (Wang, 2015). This tool has become immensely popular as it’s claimed to transform classrooms into competitive, fun environments, which stimulate learners motivation and ambition to succeed in answering questions correctly (Bicen & Kocakoyun, 2018; Ismail & Mohammad, 2017; Licorish, Owen, Daniel, & George, 2018).
Recently Kahoot has been recommended for use as an effective formative assessment tool within medical education (Bicen & Kocakoyun, 2018; Iwamoto et al., 2017; Licorish et al., 2018). However, some have cautioned that this tool may not be suited for use in all educational contexts, and may not be received favourably by all student populations (Ismail & Mohammad, 2017). Indeed, a common limitation of research conducted within the field of digital game-based learning and technology-enhanced learning is the failure to recognise that user engagement may be influenced by socio-cultural and intersubjectivity factors as opposed to the technology alone.
Yet despite the lack of empirical evidence to support digital game-based learning in all contexts and environments, increasing availability and affordability of interactive technologies has resulted in the widespread adoption of gaming within learning environments (Ebner & Holzinger, 2007; Papastergiou, 2009). As such, digital game-based learning has today succeeded in permeated all levels of education and has thus been utilised to achieve a wide range of educational purposes (Hainey et al., 2016; Ismail & Mohammad, 2017; Licorish et al., 2018).
In light of Kahoot’s ability to motivate students to engage in formative assessments, this tool was selected for use within an all-female paramedic undergraduate programme within the UAE. This approach it was hoped would provide the impetus to motivate student engagement towards non-GPA reliant tasks. It was, however, recognised that certain students within certain contexts may find this tool more effective than others; as such, this study was undertaken to explore socio-cultural and intersubjective factors which may impact upon the student’s receptiveness of digital game-based learning as a formative assessment tool. Thus, this study asked the following research questions:
How does the introduction of Kahoot, as a game-based tool for formative assessment, impact upon the learning practices of female undergraduate students in an incentivised educational system in the United Arab Emirates?
It has been claimed that formative assessment which utilises electronical tools (e-assessments) can improve and support learners to a greater extent than more traditional paper-based assessments (Bahati, Fors, Hansen, Nouri, & Mukama, 2019; Pachler, Daly, Mor, & Mellar, 2010). This is because they can provide immediate grading of student performance, and therefore expediate feedback mechanisms to rapidly address misconceptions (Shieh & Cefai, 2017). It is, therefore, conceivable to expect that e-assessments would unanimously be welcomed by all students.
However, in a recent study conducted within the UAE, students taking e-assessments for both summative and formative assessments expressed increased levels of test anxiety with this testing modality. These students thus claimed to prefer more traditional paper-based modalities, due to concerns for internet connectivity and/or system failures (Patronis, Ishtaiwa-Dweikat, Al Awad, & Aburezeq, 2019).
Incentivising learning environments have in general, been claimed to trigger feelings of fear (Castro et al., 2018), which may be accentuated if learners are reliant upon financial incentives, or fearful of non-compliance policies. Fear of failure manifesting as test anxiety can present as either cognitive or emotional concerns aligned with physiological symptoms (Schwarzer, 1984). However, there is conflicting opinion as to whether increased test anxiety enhances student performance (Sundre & Kitsantas, 2004) or whether it lowers GPAs attainment (Duty, Christian, Loftus, & Zappi, 2016).
The middle ground in this debate is that the relationship between test anxiety and performance is not linear, and as such whilst some level of anxiety may facilitate performance; excessive anxiety may negatively impact upon performance (Keeley, Zayac, & Correia, 2008). Thus, given the possibility of test anxiety impacting adversely upon students, educators need to identify ways of reducing excessive anxiety in learners—especially in contexts where the implications of failure are so high.
The digital game-based learning platform Kahoot has been identified as an effective formative assessments tool, capable of motivating students to learn, assisting in knowledge acquisition, and stimulating students impetus to study (Bicen & Kocakoyun, 2018; Ismail & Mohammad, 2017). It is postulated that this is achieved by Kahoot’s propensity to influence classrooms dynamics through the development of a competitive, interactive and collaborative classroom that motivates engagement with lecture content thereby fostering deep learning practices (Bicen & Kocakoyun, 2018; Licorish et al., 2018). Indeed, Kahoot has been credited with enabling students to speak out in class, providing a means to seek feedback immediately upon performance, and more recently attributed with permitting engagement of large numbers of students at the same time (Nkhoma, Nkhoma, Thomas, & Tu, 2018; Parra-Santos, Molina-Jorda, Casanova-Pastor, & Maiorano-Lauria, 2018; Wang & Tahir, 2020). These are indeed important attributes, as many researchers advocate that student engagement and motivation are key to eliciting meaningful learning (Biggs & Tang, 2011; Meguid & Kahalil, 2017).
Whilst a recent systematic review of the effects of using Kahoot, also revealed that Kahoot helped alleviate test anxiety in some learners; it also conversely highlighted that in some contexts Kahoot had little to no effect on student performance, classroom dynamics, attitudes or students’ levels of anxiety (Wang & Tahir, 2020). Indeed, Ismail & Mohammed (2017) claimed that female learners expressed less appreciation of this digital game-based learning platform than their male counterparts and theorised that the content being assessed through this platform also impacted upon its utility.
It is, therefore, apparent that this tool may not have the same efficacy within all environmental setting. Indeed, the literature has not yet considered wider socio-cultural influences which may impact upon Kahoot’s utility in any depth, particularly within incentivised learning environments of the UAE. As such, the remainder of this paper shall consider activity theory as a theoretical structure to evaluate these factors within the UAE setting. It shall, therefore; describe the study performed, and the students’ perceptions of Kahoot, to deliver formative assessments within the UAE.
Activity theory posits that human activities are collective and sustained efforts regulated by objects and mediated by artefacts (Bligh & Flood, 2015). Human subjects form activities because they wish to work on objects; and develop or appropriate artefacts, to help them undertake that work. An example might be working on the object of a GPA score using the tools of formative assessment.
The model of activity system we will use in this work is a unit of analysis for activity. In other words, human activity is understood as divided up into different activity systems, with different activity systems interacting across the world of human practice. An activity system, in this model, consists of seven interlocking components: subject, artefacts, rules, community, division of labour, outcomes (see Figure 1).
Table 1 depicts how these components are conceptualised and gives some idea of our initial thoughts on how these might be instantiated in the context of the study. Those initial ideas, of course, would be changed once we started the analysis of our data.
Component | How conceptualised | Initial idea |
---|---|---|
Subject | Those engaged in the activity. | Students |
Object | The motive held by the subject to complete the activity | To sustain GPA |
Outcome | The why of engaging in the activity | Successfully complete the undergraduate program |
Artefact | Used to mediate the subject-object interaction. | Formative assessment exercises |
Rules | Explicit and implicit regulation of the activity. | GPA maintenance and other college policies within the context setting |
Community | The wider social group to which the subjects belong. | Peers, teachers, family, and friends. |
Division of Labour | How the activity is distributed and carried out within the system. | Students work in groups. |
Table 1. Components of the activity system within the context of this study
Upon analysing the interactions of these components with one another, over time; internal problems and tensions known as contradictions are expressed externally, and revealed to those analysing these systems (Engeström, 1987; Murphy & Rodriguez-manzanares, 2013). As those involved in the activity strive to resolve contradictions, activity systems experience transformation, change and innovation (Engeström, 1999, 2001; Morch, Nygard, & Ludvigsen, 2010). As such, contradictions may be viewed as dynamic forces of change (Hardman, 2005), and thus identifying them permits the forces driving transformation within the activity system to be revealed.
Engeström (1987) classifies activity system contradictions as primary, secondary, tertiary and quaternary, and defines these contradiction as follows:
Primary contradictions—occur within elements of the activity, often pertaining to value-system conflicts such as the use to the subject compared to exchange value within some field/market.
Secondary contradictions—those which exist between the components of the system.
Tertiary contradictions—occur when there are tensions between existing forms of the activity with remnants of an old system.
Quaternary contradictions—manifested when an activity system interacts with a neighbouring activity system.
Within the educational setting, contradictions have been used to describe relationships between students, teachers, as well as instruments, rules and collaborations (Barab, Barnett, Yamagata-Lynch, Squire, & Keating, 2002). Indeed Hardman, (2005) utilised activity theory to understand how the introduction of technology into deprived schools mediated shifts in the object of the classroom's activity. Others have utilised activity theory to contribute to the understanding of how a presentation tool may mediate teaching and learning practices within a specific university setting (Bligh & Coyle, 2013), thus highlighting how the artefact (presentation tool) produced disturbances to both classroom and assessment rules and norms.
Similar to these studies, activity theory within this paper is utilised as a lens to focus attention upon contradictions within the digital game-based formative assessment activity system. Hence, the principles and concepts of activity theory shall form the background of this paper's data collection and shall be utilised to analyse and describe the complex relationships between students, educational practices and the community where the study was set. Thus, this study shall seek to identify how these context-specific factors impacted students' perception of the utility of Kahoot as a formative assessment tool compared to conventional paper-based assessments.
This research was conducted within an all-female paramedic undergraduate program within the UAE, which utilised formative assessment strategies within a paramedic-pharmacology module. Digital game-based learning was introduced into this module by the teacher/researcher as a means of motivating and engaging students in formative assessments and in the recognition of the potential test anxiety that learners may experience in response to the increased test burden of non-GPA reliant tasks.
A single qualitative case-based study was implemented in recognition of its ability to explaining, describing, illustrating and enlightening (Yin, 2009) practices and settings. Mwanza's (2002) eight-step model to translate activity theory into its components was utilised as a coding frame to structure this case study upon.
Activity Theory component | Mwanza’s Eight Step Translation Model |
---|---|
Activity | What sort of activity am I interested in? |
Object/Outcome | Why is this activity taking place? |
Subjects | Who is involved in carrying out this activity? |
Artefacts/Tools | By what means are the subjects carrying out this activity? |
Rules and Regulations | Are there any cultural norms, rules and regulations governing the performance of the activity? |
Division of Labour | Who is responsible for what when carrying out this activity and how are the roles organized? |
Community | What is the environment in which the activity is carried out? |
Table 2. Mwanza's (2002) eight-step model
Fourteen female Paramedic students in the second year of their bachelor’s degree (18-21 years) were invited to participate in this study. All fourteen students had undertaken both Kahoot and conventional paper-based formative assessments as part of the module’s formative assessment strategy. A total of ten students agreed to take part in this study.
During a Kahoot assessment, students individually logged onto the Kahoot website and entered a game pin into their mobile devices, which linked them to a previously prepared formative assessments quiz (Figure 2). Students individually answered the questions displayed on the classroom's projector, by selecting the answer on their devices (Figure 3).
Depending upon how accurately and quickly the student answered, a score was provided and subsequently displayed on a leader board after each question. Podium positions at the end of the quiz identify the highest scorers (Figure 4), these students were awarded a small prize at the end of the assessment. Kahoot formative assessments were all multi-choice format and students were afforded 30 seconds only to answer each question.
On alternate weeks, students were provided with a paper-based assessment, which had a mixture of multiple choice and short answer questions. One minute was afforded to complete each of the paper-based formative assessment questions. Feedback was provided on performance the following week, during which time students would review transcripts and discussed answers collectively.
The course had both paper-based and Kahoot formative assessments delivered throughout the module’s duration, there were a total of two paper-based quizzes which were conducted on weeks 2 and week 4, and four Kahoot quizzes were conducted weeks 3, 5, 6 and 9.
Upon completion of all the formative assessments, semi-structured interviews were conducted designed to elicit the students’ perceptions of the utility of the game-based platform as a formative assessment tool in comparison to conventional paper-based formative assessment. Activity theory was applied as a theoretical framework to this study, to organise the researcher’s thinking and make the complexities of the activity visible (Lingard et al., 2012). As such, Mwanza's (2002) eight-step model to translate activity theory components into research questions was utilised to help frame interview questions, to illicit the students’ perceptions of Kahoot, and to determine the impact of this digital game-based learning tool upon their learning.
It was hoped by using this model contextual factors promoting or impacting upon formative assessments delivery via Kahoot would be illuminated. In highlighting these findings through the deployment of an activity theory lens, it was theorised that this would permit discussions beyond the confines of this study, thereby, enabling a generalised understanding of the impact of incentivised learning spaces within the UAE.
Interviews conducted varied between 15 to 32 minutes in length, dependent upon the level of engagement of the interviewee with the questions. These interviews were recorded on a digital recorder, anonymised and downloaded onto a password-protected computer.
Ethics approval was granted from both Lancaster University and the UAE university where the students were enrolled. All research participants provided informed written consent for their data to use within this study.
Utilising activity system elements as a coding frame, semi-structured interviews were undertaken in tandem with sampling. Transcripts were produced after each interview and were coded by constant comparison. General themes which were uncovered were then assigned to the components parts of the activity system. The software NVivo 12 was utilised to aid in the organisation of emerging themes.
Data coded from the interviews was correlated against the corresponded components of the activity system (Engeström, 1987), and analysed for the presence of contradictions.
The possible limitations of this study to acknowledge are that results obtained may not be generalised to all context, as participants were only recruited from one higher education facility within the UAE and utilised only female students. Students enrolled in this study were able to self-select as to whether they would participate, which may have therefore introduced selection bias (List & Rasul, 2011).
The test format between the two formative assessment methods varied, as did the frequency of the exposure to the different interventions. However, the purpose of this qualitative study was not to provide absolute truths about the world in general, but rather provide a view that may help in understanding how technology may remediate context-specific activity system. Thereby providing a snapshot of debate surrounding the utilisation of digital game-based learning as a formative assessment tool in a higher educational facility.
Given the time constraints of this project, the data collected was not able to be triangulated with other sources.
The first part of the following analysis describes the elements of the activity system. The subsequent section describes the dynamics of that system—i.e., the contradictions within the system.
The object of the activity as expressed by students was to sustain their GPAs and successfully pass the course to graduate. The educator’s/researcher’s motive for introducing digital game-based learning into the classroom was to motivate students’ engagement in meaningful learning for formative assessment. However, this motivation was not expressed by the students in the interviews.
When discussing their wider educational experiences, students voiced their responses in largely personal terms. Students described studying in personal silos outside and inside the classroom for paper-based formative assessments. As such, their participation in this activity started out as a personal endeavour in which they intended to study at home, or within the institute’s facilities, but always on their own.
‘I think it was an ego thing that made me want to do well. I'm so competitive….so that’s why I studied.’
Conversely, although students still studied for Kahoot’s alone outside the classroom, they associated themselves as a collective entity, learning together when undertaking Kahoot quizzes in class. This was evident in their descriptions of how they would work together to find the correct answers during Kahoot assessments, and how they would collectively provide feedback to one another (and through their teacher) immediately after their responses were graded by Kahoot.
‘I want to hear other people discussing the question [after a Kahoot]. I want to hear you talk about it so that I can relate things to what I have studied. But when I'm alone looking at my paper [quiz], I am just like, Oh, this is my mistake and then I would forget about it.’
The students’ responses, therefore, highlighted that the present activity system was associated with a more collective subject formation than evident in paper-based assessment and, indeed, in ‘normal’ academic endeavours in the course.
All students (n=10) unanimously stated that using Kahoot as a formative assessment made the sessions feel ‘Less stressful’ and ‘fun.’ That seems important because the actual object of activity (discussed above) is high-stakes assessment and therefore typically regarded as stressful. It is evident from these documented perceptions that the student’s level of test anxiety diminished when this method of formative assessment was applied compared to paper-based formative assessment.
‘Kahoot felt like a game, it wasn't like an exam, and we were challenging each other, so it was okay if I didn’t do well….. ‘
One aspect of the Kahoot artefact that attracted some comment from students was how ‘scores’ were displayed. Whilst all students did not mind scores being displayed during formative assessment gaming, many students (n=8) did not want their peers to see their paper-based results nor discuss questions they had not answered correctly. Students explained these concerns in terms of its impact upon their self-esteem, and how others may perceive their individual performance.
‘Often in the paper base, I don't want to share my answer if I got it wrong or didn't understand the question, I will feel ashamed. So, I stay quiet, most of us stay quiet.’
Through the incentivising of learning, students within this activity system were influenced by both institutional and national policies designed to maintain and regulate academic achievements. These socio-economic structures continued to influence how students acted within the activity system and encouraged the students to focus on academic achievement.
‘I know I will get a grade […] and feel it will play an important role in my future, and then my GPA will be affected by my marks, and that’s not funny, it’s not good – that’s… it’s so stressful for us’
The level of test anxiety may have also been related to the reduced enforcement of these more typical class and assessment rules, which continued to exert influence alongside the ‘gaming’ as oppose to the ‘assessment’ atmosphere which Kahoot created.
However, students did suggest that the classroom rules were changed as a result of the introduction of Kahoot, through the creation of a more collaborative ethos. This transformation in the Kahoot classroom was conveyed by one student who stated:
‘When we do a paper-based quiz…..because we're not able to help each other talk to each other about the answers, it makes us feel helpless; as opposed to the Kahoot where as soon as I answer the question, I turned to my friend and asked her, What did you put?!!’
Students explained that the introduction of Kahoot as a formative assessment tool meant that feedback sessions became more interactive and collaborative in nature. This Collaborative environment mediated by Kahoot resulted in learners viewing themselves as less subordinate to their teacher as they not only wanted to hear what the teacher had to offer but wanted to question each other and their own ideas to construct knowledge together. Hence, oppose to passive acceptance of right and wrong, students described debates and disagreements which aided in the construction of their knowledge.
When asked how students felt about this change to normal formative assessment rules, students explained this had resulted in them becoming more engaged with the feedback process and had provided an impetus to engage in deeper learning practices within the classroom.
‘With the paper-based, we just take the answer, and want to know if it's right or wrong, but for the Kahoot …….when somebody puts the wrong answer, they will shout out No, Why? All of us like the challenge, and we really want to know why we answered incorrectly.’
On the other hand, students realised that this remediation of the assessment process meant that they could easily cheat (by conferring with neighbouring students) during the formative assessments if they wished. This behaviour of collusion during tests is at odds with the norms of assessment, strategies within the institute (and educational norms at large).
Yet such cheating was not always viewed negatively. Students could each easily recall incidents of cheating/collusion that they had experienced or witnessed during a Kahoot assessment and one student recalled incidents of how this had aided in the learning and recall of information.
‘A girl helped me with a question in the Kahoot, then something related to it came up in the exam. I remembered it because I thought, "Oh, my friend told me this before in the Kahoot, yeah, I remember now." So cheating was definitely beneficial for me.’
Students, on the whole, voiced their appreciation of collaborative learning practices and highlighted how this practice was at odds with the normal paper-based formative assessment they had experienced.
Students described their work in the activity in ways that made reference to a variety of other stakeholders, including peers on the course, other students not enrolled on the course, teachers, family members such as parents, etc. It was apparent that wider socio-economic structures governed the student’s intersubjective and socio-cultural environment.
Many of the comments described the decision to choose the course. One student expressed these concerns by describing the socio-cultural environment she was immersed within.
‘Other girls had already said this course was very hard, which made us worry about it…we were like oh my, it’s going to be so hard, so we were so stressed out…. all of them said this was the hardest course and that we would get bad marks.’
Indeed, one student having discussed her future plans with the wider student community to enrol in the module described her concerns around maintaining her GPA.
‘They said don’t do the course, your GPA will fall, and you will get kicked out of the college…. then, I imagined myself as a housewife, not doing anything…staying at home with children. So, yeah, I was worried.'
It was clear that students viewed their work in the course as relevant against a wider backdrop of expectations and reactions, which significantly influenced their working in the course.
One core influence of the introduction of Kahoot was an increased focus on studying in class and less focus on studying outside class time for the assessment. The intended outcome of the activity system from the teacher’s perspective was that the introduction of Kahoot would give students an increased impetus to study for formative assessments. Yet predominantly students (n=7) identified that paper-based assessments had a greater influence on their desire to study outside of the classroom (Figure 5).
The reason was that a fear of failure within paper-based assessments appeared to have been the prior motivation for students to study for the assessment out of class. There was a strong meaning for the students that a paper-based formative assessment was more serious, perhaps due to it being more commonly associated with summative GPA dependent tasks, (and as such, stipends and bursaries).
Whilst the population size in this study was only small (n=10) these findings demonstrate that 70% of students did not invest as much time into the preparation of formative assessments conducted by Kahoot compared to paper-based assessments. Students also described how they believed others might judge their failures on paper-based formative assessments more severely.
‘…when I did the paper-based (if I got a bad mark) my classmates would say “Oh, you got a very low mark” “Oh, my, oh my, you should do better in the next quiz” “Oh, my, you will fail” “Oh my, you should repeat the course.”…. So, it felt it was something so serious when we did paper-based.’
Nonetheless, Students claimed that despite the lack of out of class preparations for these tests, they felt they learnt and retained more knowledge from Kahoot formative assessments than they did from more conventional paper-based assessments. Hence, although their motivation to study for Kahoot formative assessments was not increased, they believed their attainment was improved by the use of Kahoot. This was a surprising finding, given the competitive nature of these environments, it was anticipated that the tool would have stimulated greater engagement in preparation for these assessments.
Students also expressed that they unexpectedly 'enjoyed' the digital game-based learning and said that as a result, they had a more positive view of the course and their teacher than they had anticipated before the module had begun.
Students worked much more intensely in the classroom to uncover meaning and understanding from the formative assessments provided through Kahoot. However, students described how they felt they did not need to invest as much time in studying for Kahoot assessments outside of the classroom, as this type of assessment was not viewed as serious to them. Hence, there are primary contradictions evident with the amount of work deemed desirable within different contexts when Kahoot was introduced: outside of the classroom—less work, inside the classroom—more collective and engaged learning.
Another primary contradiction within the Kahoot mediated activity system was evident in the students' discussions surrounding cheating and collusion. Within the Kahoot environment, these behaviours were not frowned upon, as they made interactions more fun and aided in the gaming atmosphere. Students described how these unusual behaviours helped motivate greater understanding from feedback mechanisms and assisted in recall during summative assessments.
Clearly, the aims of the teacher to shift the perception of the object of activity were unsuccessful despite the deployment of Kahoot quizzes. The cultural values surrounding the course are very deeply embedded and students persisted in seeing the object as being about sustaining GPAs.
It is, however, noteworthy, that students identified a range of other issues when discussing outcomes from the course, such as how they enjoyed the course more, participating differently, although they did continue to value GPAs above everything else.
This secondary contradiction between the object and the outcome of the activity appears to be unresolved at the time of writing, but it is interesting that students seemed to enjoy the course partly because they felt they were able to do something fun without damaging their GPAs.
Pressure expressed by the students to individually maintain their GPA scores, and to individually conform to community expectations surrounding GPA attainment, appears at odds with their collective behaviour within the Kahoot formative assessments. This is an interesting secondary contradiction, as despite both the individuals and the community working to reinforce the individualistic longstanding object of GPA maintenance, when students engaged in Kahoot assessments they did so as a collective group.
Thus, the subjects and rules, on the one hand, are in contradiction with community and object, on the other hand. This is an unresolved contradiction that shows introducing the Kahoot tool and its game-based elements exacerbates existing tensions about individual attainment.
Students expressed considerable anxiety surrounding assessments and the implications they may have upon their GPAs. It was difficult to prevent students from comparing the Kahoot assessment to the normal paper-based assessments. Likewise, it was difficult for students to accept that paper-based formative assessments were not something to be overly anxious about. It is theorised that students struggled to assimilate to the new formative assessment activity system in which they felt less anxious because this way of working was at odds with their cultural norms associated with assessments. These actions, therefore, evidenced tertiary contradictions within the system between the rules and community.
In broad terms, activity theory has been used to describe relationships between students, teachers, instruments and rules of collaboration (Barab et al., 2002) to gain insight into the dynamics of an activity. It is the intent of the following section to detail these relationships as perceived by subjects involved in digital game-based learning for formative assessments. As such, the components of the activity system at odds with one another as a result of the introduction of the digital game-based formative assessments shall be considered within the subsequent discussion. This discussion shall, therefore, illuminate female UAE undergraduates’ perceptions of Kahoot, and describe its impact upon their learning when used as a formative assessment tool through the transformation of system contradictions.
The primary outcome of undertaking the course was expressed by the students as the maintenance of GPA and the successful passing of the course, an unexpected outcome expressed as a result of digital game-based learning formative assessments was increased student satisfaction (of both their teacher and the course content). This secondary contradiction whilst not the main focus of this study is important to note, as future users of Kahoot may choose to do so in light of the double-natured contradiction which may be produced in an activity system by this technology.
Indeed, the semi-structured interviews evidenced that students and teacher had different intended outcomes of this activity. Whilst the teacher wanted to construct deeper understanding and learning through engagement in formative assessments. Students were transfixed on GPA maintenance and successful course completion before commencing the course. Through the use of Kahoot, the students’ actions were, therefore, rearticulated towards the teacher’s goals of the activity system – as they engaged in meaningful learning which also translated to their goals to sustain their GPAs. Blunden (2010) summarises such rearticulation of motives and goals within an activity system as being ‘dependent upon the representation of the activity through mediation of social relations.’ (p.178)
It is, therefore, evident that Kahoot mediated a transformation in formative assessment engagement by learners, which aligned with the teacher's motives for delivering the assessment tasks in this way. As such, the utilisation of Kahoot to shift pedagogical practices within the classroom to suit both teacher and students' needs, may be worthy of future investigation.
Previous studies have revealed that Kahoot is a fun form of digital game-based learning which was capable of serving as both a formative assessment tool and revision aid (Ismail & Mohammad, 2017; Wang & Tahir, 2020). In the context where this study was conducted students supported these claims and also highlighted that when this method of formative assessment was employed compared to paper-based formative assessments, the mood in the classroom was lightened; and transformed into an enjoyable fun learning experience.
Interestingly this study also highlights that in utilising Kahoot as a formative assessment tool, the rules and classroom norms were transformed—as classroom formalities associated with tests were removed and replaced with a collaborative environment where feedback upon performance was actively sought and debated. This is an important consideration when planning formative assessments, which ensure student agency is maximised (Lyon et al., 2019).
Students explained their lack of engagement in feedback when not engaged within Kahoot formative assessments was routed in concerns for their self-esteem and others’ perceptions of their performance. Their descriptions of discomfort and embarrassment attached to the possible failure of an assessment are symptoms associated with test anxiety (Minor & Gold, 1985). These findings highlight secondary unresolved contradictions within the system between the subjects and the rules, and the community. Conversely, the introduction of Kahoot transformed the formative assessment process by resolving/remediating the fears of the learners—through the gamification of formative assessment. Hence, within these environments, students felt safer and no longer felt the need to protect their anonymity or feel embarrassed about gaps in their knowledge.
Whilst cheating would not normally be welcomed within an assessment process, it is postulated from the review of the transcribed data, that these actions were beneficial in aiding memory retention. These memorable moments of breaking classroom rules and cheating may have mediated recall of facts and knowledge gained during these incidents, and thereby created more superior knowledge retention. These actions of collaboration, cheating and collusion within the Kahoot tool-mediated formative assessment, therefore, produced primary contradiction with the activity system between the rules and the subjects. These contradictions, thus, lead to Kahoot transforming the teaching environment to an interactive and collaborative domain, compared to the silent, and solitary paper-based formative assessment environment.
Socio-economic structures which governed the student’s intersubjective and socio-cultural environment were evident from the student’s descriptions of how the wider community viewed the need to comply with these societal norms in which the activity system was grounded. This was evident in one interview where the student expressed concern for the possible implications of her failure to maintain her GPA might mean she may need to adjust her career goal from paramedic to housewife. This narrative evidenced the degree to which the student felt uncomfortable about the possibility of her failure, and the heightened degree of tension and worry this was causing her. It also evidences the wider socio-cultural environment which was perpetuating these concerns in the students and details the economic reality of non-compliance with local and national policies to the students within this study.
Collaborative practices are highly sought graduate attributes, and as such there have been calls within the literature to recruit existing technologies that may re-mediate areas of student learning to enable collaborative learning to occur (Crook, 2010). It would appear from this case study that Kahoot as a formative assessment tool may be able to offer such a remediating solution, as students claimed that whilst their out of class learning practices on the whole were reduced by the introduction of Kahoot, their learning practices within the classroom were transformed.
Students perceived that whilst the introduction of Kahoot did not mean they needed to devote more time to out of class study for these formative assessments, as they were less serious; they recognised that the introduction of Kahoot meant that feedback sessions became more interactive and collaborative in nature. Thus, students were utilising technology to negotiate meaning through the social world of collaboration, rather than only within the individual's minds (Stahl, Koschmann, & Suthers, 2006).
This Collaborative environment mediated by Kahoot resulted in learners viewing themselves as less subordinate to their teacher as they not only wanted to hear what the teacher had to offer but also wanted to question each other and their own ideas to construct knowledge together. As such a new form of activity was mediated via the introduction of Kahoot as a formative assessment tool.
Given contextual and demographic differences between this study and others, it is not possible to refute claims as to the utility of Kahoot to motivate out of class study (Bicen & Kocakoyun, 2018). It is, however, a clear example of the importance of considering the socio-cultural, economic and intersubjectivity influences; and also highlights the importance of comparing technological tools directly against pre-existing conventional ones (Hainey et al., 2016).
Similar to earlier research, this study revealed that digital game-based learning conducted with Kahoot produced a fun, and collaborative atmosphere, which helped alleviate symptoms of test anxiety. However, unlike previous studies, through the application of an activity theory lens, this study permitted the influence that contextual factors had upon the utility of this technology to be evaluated. In doing so, this study revealed the presence of contradictions between the rules and norms of assessment policies, resultant from introducing Kahoot into the incentivised learning environment. These primary contradictions within the system produced by the utilisation of Kahoot, permitted collaboration and collusion to be fostered within learners who would usually work alone under formal exam conditions. Students identified that this was beneficial to their knowledge construction as it created memorable moments that aided knowledge recall, and made them feel less anxious.
Students did not, however, identify that digital game-based formative assessment, conducted with Kahoot stimulated greater out of class learning. Indeed, students identified that within their incentivised mediated context, that standard paper-based formative assessments motivated out of class engagement more. This was borne out of fear for this type of assessment being psychologically linked to GPA attainment, and fears that the wider student community would view their failure. Such findings may be limited to the incentivised learning context of the UAE, as within this environment poor performance in formally structured assessments can have both implications upon academic progression and financial bursaries received.
Hence, the findings in this study may be different from other context results due to the socio-economic and intersubjectivity of this system, which incentivises learning within the UAE context. However, the results of the activity system examined indicated that through collaboration and collusion Kahoot may permit superior knowledge retention, without increased investment in out-of-class studying. Further studies are required to identify whether such practices have tangible effects on students performance. However, this study evidenced that digital game-based learning with Kahoot did produce marked effects on student satisfaction rates expressed towards both the teacher and the subject content. Formative assessment conducted by digital game-based learning may, therefore, offer an attractive solution to educators and curriculum designers who wish to engage students in formative assessments; and also, positively affect perceived levels of student’s satisfaction.
This paper draws on research undertaken as part of the Doctoral Programme in E-Research and Technology Enhanced Learning in the Department of Educational Research at Lancaster University.
Maxine Minton, Department of Educational Research, Lancaster University, Lancaster, United Kingdom.
Maxine Minton is a PhD student of the Centre for Technology Enhanced Learning at Lancaster University. She is a Paramedic who has worked both within the United Kingdom and the United Arab Emirates within the prehospital field and currently lectures in Paramedic science within the UAE. She has a keen interest in utilising technology within the classroom to enhance students’ engagement and professional development.
Email: [email protected]
ORCID: 0000-0002-6589-5365
Brett Bligh, Department of Educational Research, Lancaster University, Lancaster, United Kingdom.
Brett Bligh is a Lecturer in the Department of Educational Research, Lancaster University, and co-Director of the Centre for Technology Enhanced Learning. His research interrogates the nexus of technology mediation, physical environment, and institutional change in higher education. Brett’s work prioritises Activity Theory conceptions of human practice, and interventionist methodologies.
Email: [email protected]
ORCID: 0000-0003-4591-8897
Twitter: @BrettBligh
Article type: Full paper, double-blind peer review.
Publication history: Received: 19 February 2021. Revised: 29 March 2021. Accepted: 30 March 2021. Published: 16 April 2021.
Cover image: mentatdgt via Pexels.
Andrade, H. L., Wang, X. L., Du, Y., & Akawi, R. L. (2009). Rubric-referenced self assessment and self-efficacy for writing. Journal of Educational Research, 102(4), 287–301.
Bahati, B., Fors, U., Hansen, P., Nouri, J., & Mukama, E. (2019). Measuring learner satisfaction with formative e-assessment strategies. International Journal of Emerging Technologies in Learning, 14(7), 61–79. https://doi.org/10.3991/ijet.v14i07.9120
Barab, S. A., Barnett, M., Yamagata-Lynch, L., Squire, K., & Keating, T. (2002). Using activity theory to understand the contradictions characterizing a technology-rich introductory astronomy course. Mind, Culture, and Activity, 9(2), 76–107. https://doi.org/10.1207/s15327884MCA0902_01
Bicen, H., & Kocakoyun, S. (2018). Perceptions of Students for Gamification Approach: Kahoot as a Case Study. International Journal of Emerging Technologies in Learning, 13(2), 72–93.
Biggs, J. B., & Tang, C. (2011). Teaching for quality learning at university (4th ed.). Maidenhead: Society for Research into Higher Education & Open University Press.
Black, P., & Williams, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31.
Bligh, B., & Coyle, D. (2013). Re-mediating classroom activity with a non-linear, multi-display presentation tool. Computers & Education, 63, 337–357. https://doi.org/10.1016/j.compedu.2013.01.001
Bligh, B., & Flood, M. (2015). The Change Laboratory in Higher Education: research-intervention using activity theory. In J. Huisman, & M. Tight (Eds.), Theory and Method in Higher Education Research (pp. 141-168). Emerald Group Publishing Ltd. https://doi.org/10.1108/s2056-375220150000001007
Bligh, B., & Flood, M. (2017). Activity theory in empirical higher education research: choices, uses and values. Tertiary Education and Management, 23(2), 125–152. https://doi.org/10.1080/13583883.2017.1284258
Blunden, A. (2010). An interdisciplinary theory of activity. Leiden: Brill.
Cameron, J. (2001). Negative effects of reward on intrinsic motivation, a limited phenomenon: Review of Educational Research, 7(1), 29–42.
Castro, J. F., G, Y., Contreras, H., Linares, F., & Watson, H. (2018). The Effect of Monetary Incentives on Cognitive Effort, Emotions and Test-Solving Performance. Peruvian Economic Association.
Crook, C. (2010). Versions of computer-supported collaboration in higher education. In Ludvigsen, S. R., Lund, A., & Rasmussen, I. (Eds), Learning across sites: New tools, infrastructures and practice (pp. 156–171). Routledge.
Duty, S. M., Christian, L., Loftus, J., & Zappi, V. (2016). Is cognitive test-taking anxiety associated with academic performance among nursing students? Nurse Educator, 41(2), 70–74.
Ebner, M., & Holzinger, A. (2007). Successful implementation of user-centred game-based learning in higher education: an example from civil engineering. Computers & Education, 49(3), 873–890.
Embassy of the UAE. (2021). Education in the UAE. Retrieved March 28, 2021, from https://www.uae-embassy.org/about-uae/education-uae
Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Helsinki: Orienta-Konsultit.
Engeström, Y. (1999). Activity theory as individual and social transformation. In Y. Engeström, R. Miettinen, & L. Pumamaki (Eds.), Perspectives on activity theory (pp. 19–38). Cambridge: Cambridge University Press. Retrieved from http://dx.doi.org/10.1017/CBO9780511812774.003
Engeström, Y. (2001). Expansive learning at work: Towards an activity-theoretical reconceptualization. Journal of Education and Work, 14(1), 133–156.
FCHS. (2021a). Incentives. Incentives for UAE Nationals. Retrived April 19, 2021, from https://www.fchs.ac.ae/En/Perspective%20Students/Pages/Stipends.aspx
FCHS. (2021b). Scholarship. For Non-UAE nationals. Retrived April 19, 2021, from http://www.fchs.ac.ae/En/Perspective%20Students/Pages/Scholarship.aspx
Federal Government UAE. (2010). UAE Vision 2021. Retrieved from https://www.vision2021.ae/en
Fry, H., Ketteridge, S., & Marshall, S. (2015). A handbook for teaching and learning in higher education: Enhancing academic practice (Fourth Edition). Abingdon, Oxon: Routledge.
Hainey, T., Connolly, T. M., Boyle, E. A., Wilson, A., & Razak, A. (2016). A systematic literature review of game-based learning empirical evidence in primary education. Computers & Education, 102, 202–223. https://doi.org/10.1016/j.compedu.2016.09.001
Hardman, J. (2005). An explanatory case study of computer use in a primary school mathematics classroom: New technology, new pedagogy? Perspectives in Education, 23(4), 99–111.
Ismail, M. A.-A., & Mohammad, J. A.-M. (2017). Kahoot: A Promising Tool for Formative Assessment in Medical Education. Education in Medicine Journal, 9(2), 19–26. https://doi.org/10.21315/eimj2017.9.2.2
Iwamoto, D. H., Hargis, J., Taitano, E. J., & Vuong, K. (2017). Analyzing the efficacy of the testing effect using Kahoot on student performance. Turkish Online Journal of Distance Education, 18(2), 80–93. https://doi.org/10.17718/tojde.306561
Kahoot. (2018). New year, new heights: Kahoot! grows by 75% to reach 70 million unique users. Retrieved from https://kahoot.com/press/2018/01/18/kahoot-grows-reach-70-million-unique-users/#:~:text=About Kahoot!&text=The game platform now hosts,in more than 200 countries.
Kamel, S. (2014). Education in the Middle East: Challenges and opportunities. In N. Azoury (Ed.), Business and Education in the Middle East (pp. 99–130). London: Palgrave Macmillan UK. https://doi.org/10.1057/9781137396969
Keeley, J., Zayac, R., & Correia, C. (2008). Curvilinear relationships between statistics, anxiety and performance among undergraduate students: Evidence for optimal anxiety. Statistics Education Research Journal, 7(1), 4–15.
KU. (2021). Khalifa University Undergraduate Scholarships. Retrieved April 19, 2021, from https://www.ku.ac.ae/scholarships-undergraduate
Licorish, S., Owen, H. E., Daniel, B., & George, J. I. (2018). Students perceptions of Kahoot!’s influence on teaching and learning. Research and Practice in Technology Enhanced Learning, 13(9), 1–23.
Lingard, L., McDougall, A., Levstik, M., Chandok, N., Spafford, M. M., & Schryer, C. (2012). Representing complexity well: A story about team-work, with implications for how we teach collaboration. Medical Education, 46(9), 869–877.
List, J. R., & Rasul, I. (2011). Field experiments in labor economics. In Handbook of Labor Economics. Elsevier.
Lyon, C. J., Oláh, L. N., & Wylie, E. C. (2019). Working toward integrated practice: Understanding the interaction among formative assessment strategies. The Journal of Educational Research, 1–14. https://doi.org/10.1080/00220671.2018.1514359https://doi.org/10.1207/s15326985ep2802_5
Meguid, E. M. A., & Kahalil, M. K. (2017). Measuring medical students motivation to learn anatomy by cadaveric dissection. Anatomical Science Education, 10(4), 363–371. https://doi.org/10.1002/ase.1669
Minor, S. ., & Gold, S. (1985). Worry and emotionality components of test anxiety. Journal of Personality Assessment, 49(1), 82–85.
Morch, A. L., Nygard, K. A., & Ludvigsen, S. R. (2010). Adaption and generalisation in software product development. In H. Daniels, A. Edwards, Y. Engeström, T. Gallagher, & S. Ludvigsen (Eds.), Activity theory in practice: promoting learning across boundaries and agencies (pp. 184–205). London: Routledge.
Morillas-Bario, C., Munoz-Organero, M., & Sanchez-Soriano, J. (2016). Can gamification improve the benefits of student response systems in learning? IEEE Transactions on Emerging Topics in Computing, 4(3), 429–438.
Murphy, E., & Rodriguez-manzanares, M. A. (2013). Activity theory perspectives on technology in higher education. IGI Global.
Mwanza, D. (2002). Towards an activity-orientated design method for HCI research and practice. The Open University Press. Retrieved from http://www.idemployee.id.tue.nl/g.w.m.rauterberg/conferences/INTERACT2003/INTERACT2003-p1045.pdf
Nkhoma, C., Nkhoma, M., Thomas, S., & Tu, L. K. (2018). Gamifying a flipped first-year accounting classroom using Kahoot! International Journal of Information System and Engineering, 6(2), 93–115. https://doi.org/10.24924/ijise/2018.11/v6.iss2/93.115
Pachler, N., Daly, C., Mor, Y., & Mellar, H. (2010). Formative e-assessment: Practitioner cases. Computers & Education, 54(3), 715–721. https://doi.org/https://doi.org/10.1016/j.compedu.2009.09.032.
Papastergiou, M. (2009). Digital game-based learning in high school computer science education: impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1–12.
Parra-Santos, T., Molina-Jorda, J. M., Casanova-Pastor, G., & Maiorano-Lauria, L. P. (2018). Gamification for formative assessment in the framework of engineering learning. In The sixth international conference on technological ecosystems for enhancing multiculturality (pp. 61–65).
Patronis, M., Ishtaiwa-Dweikat, F. F., Al Awad, M., & Aburezeq, I. M. (2019). Attitudes and perceptions towards summative e-assessment for free-text responses: A case study of a UAE university. International Journal of Information and Communication Technology Education, 15(1), 13–28. https://doi.org/10.4018/IJICTE.2019010102
Putwain, D. W., Langdale, H. C., Woods, K. A., & Nicholson, L. J. (2011). Developing and piloting a dot-probe measurement of attentional bias for test anxiety. Learning and Individual Differences, 21(4), 478–482. Retrieved from https://doi.org/10.1016/j.lindif.2011.02.002
Quinn, B. L., & Peters, A. (2017). Strategies to Reduce Nursing Student Test Anxiety: A Literature Review. Journal of Nursing Education, 56(3), 145-151. https://doi.org/doi:10.3928/01484834-20170222-05
Schwarzer, R. (1984). Worry and emotionality as separate components in test anxiety. International Review of Applied Psychology, 33(2), 205–220.
Shieh, J., & Cefai, C. (2017). Assessment of learning and teaching in higher education: a case analysis of a university in the south of Europe. Malta Review of Educational Research, 11(1), 29–47.
Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning : An historical perspective. In Cambridge handbook of the learning sciences (pp. 409–426). Cambridge, UK: Cambridge University Press.
Sundre, D. L., & Kitsantas, A. (2004). An exploration of the psychology of the examiner: Can examinee self-regulation and test-taking motivation predict consequential and non-consequential test performance? Contemporary Educational Psychology, 29(1).
Wang, A. (2015). The wear out effect of a game-based student response system. Computers & Education, 82, 217–227.
Wang, A. I., & Tahir, R. (2020). The effect of using Kahoot! for learning – A literature review. Computers and Education, 149(January), 103818. https://doi.org/10.1016/j.compedu.2020.103818
Yin, R. K. (2009). Case study research: design and methods (4th edition). Thousand Oaks, CA: Sage.