The European Digital Kitchen Project

This article reports on the European Digital Kitchen, an EU-funded language learning project which promotes learning of languages, cultures and cuisines in digital interactive kitchens. The project involves taking a normal kitchen and specifically adapting it for language learning using the next generation of digital technology, namely activity recognition and sensor technology. We intend that learners will be able to learn aspects of the language whilst performing a meaningful real-world task and will simultaneously experience the cultural aspect of learning to cook a foreign dish. The article starts by outlining the project background, including rationale, motivation and aims. We then explain in detail how the technology works (using photographs) and outline our design methodology, which blends Task-Based Language Teaching (TBLT) and Human-Computer Interaction (HCI). We then present 3 extracts of Digital Kitchen interaction to illustrate the type of learning which takes place.


2
will provide for the use of 7 different European languages.The project supports development of innovative ICT for language learning, mobility across EU countries and the integration of migrant language learning.In Finnish and Catalan, our project promotes the acquisition of less-used languages.
EU agendas include the Inclusive Growth priority of Europe 2020, Agenda for New Skills and Jobs and Youth on the Move.
Our project involves taking a normal kitchen and specifically adapting it for language learning using the next generation of digital technology, namely activity recognition and sensor technology.We constructed a purpose-built kitchen that communicates with learners in a European language and gives them step-by-step instructions on how to prepare cuisine and teaches aspects of target language.The first generation of this technology was produced by the French Digital Kitchen project at Newcastle University, as reported in Seedhouse et al, 2013.The French Digital Kitchen project was the result of collaboration between computer scientists working on the development of assistive technology for pervasive environments, namely the construction of an Ambient Kitchen used to support people with dementia (Olivier et al, 2009;Pham & Olivier, 2009) and applied linguists working on how digital technology can be combined with a task-based approach to language learning (Seedhouse & Almutairi, 2009).We chose cooking as a relevant task as there is currently huge interest throughout the European Union in cooking, as can be seen in the number of cookbooks sold and the number of cooking programmes on TV.Many adult learners are motivated to learn European languages through their interest in cuisine and culture, and this project taps into this motivation.
There are a number of well-known problems relating to classroom foreign language teaching addressed by this project.These include, firstly, the universal problem of classroom language teaching, namely that students are rehearsing using the language, rather than actually using the language to carry out actions such as buying a train ticket; secondly, the difficulty of bringing the foreign culture to life in the classroom.In the digital kitchen environment, we intend that learners will be able to learn aspects of the language whilst performing a meaningful real-world task and will simultaneously experience the cultural aspect of learning to cook a foreign dish.Seedhouse, et al. Bellaterra Journal of Teaching & Learning Language & Literature. 7.1 (Feb-Mar 2014) ISSN 2013-6196 3

How does the European Digital Kitchen work?
Constructing the European Digital Kitchen involved drawing on an existing technologically-enhanced kitchen (the Ambient Kitchen) which was originally developed at Newcastle University to support older people and those with dementia in their everyday kitchen activities.The term 'ambient' refers to the nature of the technology used in the kitchen, which is absorbed or hidden in that environment and, similarly to a car satellite navigation system, is designed to guide and support the user in an everyday setting.The Ambient Kitchen was designed to provide situated support in the form of written or audio prompting during a kitchen-based activity such as making a cup of tea.It does this by detecting actions and linking these to the possible intentions of the user.
In the European Digital Kitchen, this technology was developed so that the kitchen speaks to the learners in a range of languages, providing step-by-step cooking instructions in relation to learners' completion of the cooking steps.It can also detect what the learners are (or are not) doing and this information is used by the kitchen programme to provide feedback, such as a reminder that help is available, or to provide more details about a certain cooking action as a 'tip'.Embedded or hidden digital sensors were developed and inserted in or attached to all the equipment (for example, a peeler, a mixing bowl, a whisk or even the oven door) and ingredients (for example, a bag of flour, sugar or a tub of butter) as in Figure 1.The sensors use a technology similar to the Nintendo Wii™.Learners are also able to communicate with the kitchen, using an interactive screen or Graphical User-Interface (GUI), where they can request audio and visual help along the way and the ability to move back and forward between the cooking instructions of needed, as in Figure 2. To sense and recognize activities relevant to the cooking process, we instrumented the objects used for cooking with small, inexpensive acceleration sensors (see Figure 3).These wireless sensors are integrated into the handles of cooking utensils, incorporated into containers that hold ingredients, and directly attached to kitchen appliances (e.g., oven door, weighing scales).
When a sensor detects movement it starts transmitting the raw acceleration data to a nearby receiver, which is connected to a host computer.To recognize activities from the accelerometer data, we employ a technique that reports motion if certain thresholds in the signal's energy and the magnitude of its power spectrum are exceeded.Motion events are generated if kitchen objects, e.g., food containers or the oven door, are moved.
We have produced a 'portable digital kitchen' for the project, made up of tablet PC with touch screen and a set of utensils with embedded sensors and additional sensors for other kitchen equipment.
This portable digital kitchen can function in any existing kitchen setup, ideally where there is access to kitchen equipment such as a counter, sink and an oven etc.Following standard practice in Task-Based Language Teaching (TBLT), kitchen users work in pairs; we normally paired users with skills in a European language together with users with skills in cookery so that they were able to exchange skills.Users followed the 3-stage task cycle detailed in the following section.

Task-Based Language Teaching Principles
In the previous section, we described the practical functioning of the kitchen.In this section, we explain our design methodology for constructing and trialling the kitchen, which blends pedagogy and technology, specifically Task-Based Language Teaching (TBLT) and Human-Computer Interaction (HCI).
The pedagogical design of the European Digital Kitchen employs TBLT, a well-established approach to language learning which prompts learners to achieve a goal or complete a task (Skehan, 1998;2003).Much like real-world tasks, such as asking for directions, TBLT seeks to develop students' language through providing a task and then using language to solve it.Some of the main features of TBLT are that: meaning is primary (language use rather than form); there is some communication problem to solve; a classroom task relates directly to real world activities; the assessment is done in terms of outcomes (Ellis, 2003).Willis (1996, p. 1) defines the aims of tasks as "to create a real purpose for language use and to provide a natural context for language study".It is generally assumed (Ellis, 2003, p. 263) that tasks are carried out in pairs or small groups in order to maximise interaction and autonomy.There has been a substantial programme of research in relation to TBLT, summarised in Skehan (2003).Ellis (2003, p. 320) suggests that "there is a clear psycholinguistic rationale (and substantial empirical support) for choosing 'task' as the basis for language pedagogy."Skehan (1998, p. 95) suggests that transacting tasks "…will engage naturalistic acquisitional mechanisms, cause the underlying interlanguage to be stretched, and drive development forward".TBLT has so far predominantly been based on tasks to be undertaken within the classroom which simulate real-world tasks.Some innovations in TBLT have combined language learning with other, non-linguistic skills in a similar way to this project.Paterson & Willis's (2008) English through Music, for example, aims to help children to absorb English naturally as they enjoy making music together.There have been few attempts to employ TBLT in naturalistic settings outside the classroom; the project described here is innovative in combining TBLT and digital technology in a naturalistic kitchen setting outside the classroom.
In order to operationalize TBLT in this setting we adopted Skehan's (1998) framework in which tasks are divided into 3 phases: pre-task, during-task and post-task.This provided a clear design structure for materials.The pre-task functions as a preparation stage for the activity to be carried out in the during-task phase.This may include the presentation of new language, the mobilisation of existing language knowledge and clarification of the type of knowledge that would be required (Skehan, 1998, p. 138).The during-task phase involves the performance of the task set.It is in this phase of the task that Skehan claimed learners' attention can be specifically manipulated through a range of features such as time pressure, support and surprise.The post-task phase is designed to manipulate attention through the analysis of during-task performance and reflection, as a period of evaluation and consolidation after the completion of the task.This is similar to the 'plenary' section of a school lesson where a teacher goes through the learning objectives of a lesson and pupils identify 'what they have learned'.The previous section explains how the task cycle was implemented during a kitchen session.

Human-Computer Interaction Design Principles
HCI employs 'bottom-up' approaches, where development of the technology is based on direct observation and investigation of usability of an initial prototype.As Abras et al. state, "the role of the designer is to facilitate the task for the user and to make sure that the user is able to make use of the product as intended and with a minimum effort to learn how to use it" (2004, p. 763).To do this, HCI designers go through a range of iterative processes to produce a design which is based on user activity.
In the European Digital Kitchen, HCI design focused on 'ambient displays' on the kitchen walls (Figure 5) and the provision of a Graphical User-Interface (GUI) (Figure 3), as well as an in-built hidden speaker system.
Figure 5: The Ambient Kitchen (Pham & Olivier, 2009) How did the HCI design support the TBLT framework?The 'ambient displays' provided a location from which to show the recipe preparation video and the vocabulary slideshow, and the in-built speakers streamed the audio information attached to this visual information.The speaker system also provided the list of ingredients in the pre-task.The GUI was specifically designed to support learning processes.In the pre-task, the GUI offered three types of scaffolding: a photo or video of the object or process to use, a repetition request and the option of moving back and forwards through the list of ingredients.In the during-task phase, the GUI provided learners with photo/video, repetitions and the option to move around the cooking instructions.In classroom-based TBLT, the learners carry out tasks themselves, but can call on the teacher as a resource if they require some kind of help or support.In a similar way, the GUI provides scaffolding for learners if and when they require it.
Learning was also supported through the activity recognition sensor technology, which was designed to provide the different steps of the cooking instructions in a timely manner, that is, as and when learners were ready in terms of how they were progressing through the recipe operations.Further scaffolding was provided in terms of prompts, consisting of alternative versions of instructions, often reformulated in terms of 'tips' about cooking technique.The prompts were designed in such a way as to occur in response to two alternatives: a) after a period of non-activity where the sensor technology was able to detect that an operation had not been carried out even though a cooking instruction had been communicated or b) if the incorrect food item or kitchen equipment had been moved as a result of miscomprehension.Similarly to the technological affordances for the pre-task, the post-task exercises were also embedded in the kitchen using the 'ambient displays' (Figure 5).The design of the kitchen was therefore based on a blend of pedagogical and technological principles, and the next section provides examples of how learners use the kitchen, as well as how they interact and learn during a task.

What is learnt in the European Digital Kitchen?
In this section we illustrate how learners use the resources of the kitchen to carry out their tasks and analyse the types of learning which can occur.The data come from the previous French Digital Kitchen project, because learning data from the European project are not yet available.We used Conversation Analysis to analyse the interaction which provided us with a holistic way of documenting the momentto-moment verbal and non-verbal activity in the tasks as the learners interacted with each other in pairs.
Transcript 1: Making a hole Transcript 1 is an extract from a session which involved a catering student (S1) who had very little prior knowledge of French and an advanced learner of French (S2).In the opening to this episode, we see that when the instructions are provided by the Kitchen (AR), S2 immediately offers a translation.From lines 5 to 11, we then see that S1, the catering student offers his own input into the task.First he introduces a catering term, "well" and then goes on to give S2 (who is making the well) more information about the technique and guides her in the cooking action.After a short gap, in line 12, S1 then asks S2 to provide the French word for 'mix'.Here, we see the transition or rather, return, to a new focus on language.
Importantly, this switch to a focus on language form has initiated a further action from S1 to reuse this knowledge to create a new phrase 'mélanger oeufs'.In line 16, S2 offers further feedback about the French instruction 'réalisez un puits' by reformulating it and thus breaking down the instruction to make it more explicit.
What is especially noticeable in this example is how learning in the European Digital Kitchen has the potential to provide for productive cross-curricular opportunities in French with other areas of the curriculum and more specifically here, Food Technology.Both learners, or 'experts' (one in catering and one in French) support each other through this particular stage of the task.S2 as French expert, demonstrates and practices her knowledge of French whilst at the same time developing her ability to apply language skills (for example, by helping S1).S1, as catering expert and French novice, guides S2 through the cooking task activity and also develops his language skills by asking questions and manipulating the language to create new phrases.
Transcript 2: Peeling the Pears This episode involves an advanced level learner (University undergraduate student) and beginner level learner of French.Neither has any professional cooking experience.The episode opens with a prompt from the kitchen (AR) to 'peel the pears'.This type of prompt demonstrates how the sensor technology in the kitchen is designed to guide the learners and appears when the inbuilt programme has registered that an action has yet to be performed, even though an instruction has been previously been provided.In line 2, S2 identifies the action 'éplucher les poires' as an unknown phrase.She uses the translation facility on the interactive screen to assist her.As a result of help, she then identifies the word causing her the problem, 'éplucher' and acknowledges the translation.Next, S2 notices the 'peeler' as the utensil needed to complete the action and reads the words 'couteau éplucheur' off the label attached to the 'peeler'.In line 8 , S1 returns to the conversation by reusing the word 'éplucheur' and adds the comment 'very helpful'.The real-world nature of the task means that the users need to access the right equipment to carry out the right actions for the task; it is not just a question of understanding the input provided in the audio messages.From line 8, we can see how, in this task-based learning environment, knowledge of language and cooking is interdependent.S1 demonstrates an additional orientation to the meeting of new language knowledge concerning the verb 'éplucher' (to peel).This sequence shows how S1 has both confirmed the learning of 'éplucher' and is additionally able to apply it to a new (but related) linguistic context.S1 applies the recently noticed verb 'éplucher' to identify the adjective 'éplucheur' in the noun phrase 'couteau éplucheur' (peeler, or directly translated, peeling knife).S1 does this by using a tangible support provided by the kitchen, namely the labelling of utensils and ingredients in the target language.S1's turn demonstrates a dual focus on the on-going cooking action, where a peeler is now needed, and a self-initiated focus on language.
Extract 2 demonstrates how the pedagogical and technical design supports the autonomous learning processes engaged in by the users.It also illustrates how learning in the European Digital Kitchen involves using language skills and strategies to complete a stage in the task.These skills and strategies are applied by S1 through the use of the interactive screen to make links with English and by S2 through the use of the labelling of the utensils.In this vocational context the learners are focused on the dual pedagogic goals of the task (language and cooking) and their production demonstrate their different levels of expertise in cooking and French.Thus, the language focus of this particular activity becomes one of 'talking about the language' rather than through it as might happen in a traditional classroom set-up.
The example also shows the collaborative nature of strategy use in that both learners pay attention to and benefit from each others' learning behaviours.Observations from this session allowed us to see how the task provided learners with an appropriate context for autonomous learning where they could experiment with French and both use and develop language learning strategies and skills.Extract 2 also shows how the pedagogical and technological design allows for the initiation and application of new language which can be made potentially relevant at any point in the during-task phase.
Transcript 3: Weighing the Sugar Again, the interaction between the learners is initiated by an instruction from the kitchen.In line 3, S2 focuses on the amount of sugar needed, ninety grams, which shows how he has broken down the instruction into its relevant content.The talk that follows this relates specifically to the cooking activity in that S2 is weighing out the sugar whilst S1 keeps a close eye and uses French to indicate to S2 his progress in terms of pouring the sugar into the weighing dish.
This third example provides an insight into the ways in which the cooking task allows learners to reuse language, in the case of S2, and adapt language for different purposes, as in the case of S1.
Importantly, this example also shows how such authentic language use is produced in a context of completing actions and real world communication between paired learners.
The examples above show how learners used French and engaged in specific types of learning processes to complete the cooking task.These processes involved a focus on attention to new language and applying existing language knowledge.As such, they demonstrate the kinds of learning behaviours which are central to key concepts, processes and curriculum opportunities promoted in classroom learning for French.They show how a 'real-world' task-based learning environment is able to foster and develop learners' knowledge, creativity, strategies and skills outside the French classroom.Importantly, this is made possible in the French Digital Kitchen by providing an autonomous context for learning which is supported by the user-centered technology seen in these examples through the use of the interactive screen and the provision of timely cooking instructions.Figure 6 below presents a summary of learning processes we observed in our trials to support these claims: Learners noticed key words and phrases by: • Listening to the instructions provided by the French kitchen • Using the learning supports (labelling of equipment and ingredients in French) • Listening to the appropriate or 'timely' feed--back from the kitchen in the form of reminders and reformulations • Hearing their partner use words or phrase in their own creative use of French • Using the translation or repetition facility on the interactive screen Learners then manipulated these key words and phrases whilst talking with their partner, this involved: • Reusing key words during a cooking action • Creating new phrases from key words during a cooking action • Creating new phrases from existing knowledge to communicate personal meanings about a cooking activity and food.
Learners also demonstrated existing knowledge of French including key words and phrases, this involved: • Using language to communicate personal meanings about food or a cooking activity • Repeating words and phrases communicated by the kitchen • Helping their partner to understand words and phrases by offering feedback on meaning, pronunciation

Figure 1 :
Figure 1: Digital sensors attached to cooking equipment

Figure 2 :
Figure 2: Communication with the kitchen via the Graphical User Interface

Figure 4 :
Figure 4: The portable digital Kitchen with a selection of the utensils

Figure 6 :
Figure 6: Summary of the learning behaviours identified by observing the learners in the French Digital Kitchen