تجارب با وظایف پشتیبانی شده توسط یک سیستم شناختی یادگیری الکترونیکی در مهد کودک: مدلسازی و آموزش در حافظه کاری و کنترل توجه

Abstract Improvements in executive functions appear to have a strong impact on preschool learning activities and academic performance. This paper presents some experiences in training working memory and attentional control supported by an educational software system called APRENDO. The aims were to assess the suitability of the APRENDO system as a computer-based learning system in terms of improvements in these two executive functions, and to establish whether the tasks help children – especially children with the poorest performance – in a school task requiring the use of both processes. The participants were 52 four-year-old children, divided into a control group and an experimental group. The experimental group trained with two types of APRENDO visuospatial exercises: “Find the different objects” and “Find the intruder”. Before and after the training phase, all the children performed a school task with similar psychological demands selected from their ordinary school materials. The results of both APRENDO exercises showed positive correlations between the same variables over the training sessions, demonstrating the suitability of the exercises. On the other hand, in the experimental group, there were significant differences between some of the variables analyzed, e.g., the time needed to complete the exercises (“Answering time”) or the number of clicks on the correct images (“Answer accuracy”) in both exercises during the sessions. The results indicate that the children who obtained lower scores in the pre-test phase were those who benefitted the most from training. The analysis of the errors made by the children in both tasks suggests that these errors are due to the incorrect application of the same cognitive abilities. The implications for educational practice are discussed.

Introduction The educational and psychological literature shows that the early years of schooling, from 3 to 5 years, are crucial to the development of educational and developmental processes. At this stage, academic demands require the use of skills (perceptual, linguistic, attentional, memory, etc) that are more complex than the ones usually needed in the family setting (Boujon and Quaireau, 1999, Diamond et al., 2007, Gerardi, 1997, Rueda et al., 2005 and Welsh et al., 2010). Attentional control and working memory are two of the key functions needed to perform many school tasks (Miyake et al., 2000).There are studies demonstrating that executive functions can be improved in pre-schoolers in regular classrooms and the benefits are transferred to other activities (Diamond et al., 2007, Diamond and Lee, 2011 and Cogmed Working Memory,). Teachers can train the cognitive functions using the tools and techniques common to primary education as well as computer programs specifically designed for this purpose (Diamond and Lee, 2011, Grunewaldt et al., 2013 and Rueda et al., 2005) There are some interesting systems based on psychological principles (Aleven and Koedinger, 2002, Arroyo et al., 2006 and Roll et al., 2011), although few systems have been specifically created to aid in the cognitive development of young children at school (Sung et al., 2008). We have implemented PATIO, a generic computer-based learning framework (described in more detail in Section 1.2). It has been designed specifically for early childhood education. It provides a set of generic services for defining, delivering, assessing and monitoring learning activities. It includes learning tools specialized in different educative areas for small children (such us writing, reading and training cognitive skills) that use those generic services. One of these tools is APRENDO (Trella et al., 2008) that focuses on basic cognitive skills such as attention or memory development. This paper describes a research done with APRENDO tool. The main aims of this study were (1) to test the suitability of the APRENDO system as a learning system for training working memory and attentional control in preschool children (Markopoulos et al., 2008), and (2) to conduct a pilot study to explore whether APRENDO can improve attentional control and working memory in younger children. This paper describes an experiment conducted with children between 4 and 5 years old in preschool who used the APRENDO system activities between typical school tasks. This study used a repeated measures pre-test/training/post-test design with an experimental group (EG) and a control group (CG). In the pre-test and post-test conditions, both groups performed a pen and paper task or compulsory and curricular school tasks with psychological requirements similar to those of the APRENDO exercises. The experimental group trained with two types of APRENDO visuospatial exercises: “Find the different objects” and “Find the intruder”. The rest of the paper is organized as follows. Section 1.1 reviews previous studies performed in the pre-schoolers executive functions training domain. Section 1.2 describes the basics of PATIO educative framework. At the end of this introduction, the concrete objectives of the present study are presented. Afterward, the method (participants, procedure and measures) and the results (statistical studies) of the experiences are explained in detail in 2 and 3 respectively. Finally, we conclude by analyzing and discussing the results in Section 4, and we consider the recommendations for future studies, and the implication for educational practice in Section 5. 1.1. Computer-based learning systems to promote attentional control and working memory in preschool children School learning is a cumulative process in which knowledge is built up year by year with an increase in the number and complexity of cognitive processes and strategies needed for school tasks. Thus, it is important that these processes and strategies are active and responsive to the needs of learning from infancy onwards. Attentional control and working memory are the two key executive functions needed to perform many school tasks that require concentration, the inhibition of distractions, remembering the characteristics of the stimuli or known information, and giving accurate and rapid answers (Miyake et al., 2000). There is a considerable body of work on the complexity of these two capacities (Baddeley, 2006, Bialystok and Martin, 2003, Callejas et al., 2004, Fan and Posner, 2004, Gathercole et al., 2006, Gathercole et al., 2004, Miyake et al., 2000, Posner, 2004, Posner and Petersen, 1990 and Rueda et al., 2005) and their importance for information processing and school learning (Bull and Scerif, 2001, Bull et al., 2008, Deustch and Deustch, 1963, Dunham, 1995, Foster and Watkins, 2010, Gathercole et al., 2006, Jankowski et al., 2001, Jones et al., 2003, Loe et al., 2008, Ruff and Rothbart, 1996, Thorell et al., 2009, Wassenberg et al., 2005, Welsh et al., 2010 and Wickens, 1984). Developmental research shows that executive control begins to develop from 3 to 6 years onward and that, as the children grow, attention improves in three dimensions: control, flexibility, and planning strategies, although studies on normally developing preschool children remain scarce (Rueda et al., 2004). Furthermore, these studies rarely take into account the initial signs in preschool that predict major attentional difficulties, such as attention deficit hyperactivity disorder. Thus, attentional control has been shown to be an essential element in academic learning outcomes, although it also influences emotional regulation which, in turn, has a direct impact on academic outcomes in primary school (Buhs et al., 2006, Chang and Burns, 2005, Friedman-Weieneth et al., 2007, Normandeau and Guay, 1998, Posner and Rothbart, 2007, Rose et al., 1999, Rothbart and Posner, 2006 and Vasey et al., 1996). Working memory is a key function needed for cognitive tasks, but has been little studied in normally developing younger children. Working memory consists in simultaneously maintaining and processing information over a short period of time. It has a close relationship with attentional control because it appears to be related to the ability to resist distractions and irrelevant stimuli and the ability to concentrate on information relevant to completing tasks (Gathercole et al., 2012). On the other hand, school is an essential context for child development at early ages, not only because initial difficulties in the learning process can be early detected, but because interventions can be conducted to improve the psychological skills and strategies involved in these problems (Chang and Burns, 2005, Grunewaldt et al., 2013, Ladd et al., 2006, Loe et al., 2008 and McClelland et al., 2006). Executive functions can be improved in pre-schoolers in regular classrooms and the benefits are transferred to other activities (Diamond et al., 2007 and Diamond and Lee, 2011). Teachers can train the cognitive functions needed for school tasks using the tools and techniques common to primary education: classroom curricula, pen and paper exercises, motor games, aerobic exercise, music, poetry, drama, and cognitive or linguistic enrichment tasks, etc., as well as computer programs specifically designed for this purpose (Diamond and Lee, 2011, Grunewaldt et al., 2013 and Rueda et al., 2005). The theoretical model underlying many of these interventions often assume a Vygostkian approach of mediated learning and cognitive modification (Calero, 1995, Bodrova and Leong, 2007, Feuerstein et al., 1979, Holdich and Chung, 2003 and Vigotsky, 1978). Training can be treated as an active phase with monitoring and guidance, with the aim of coaching the students in those basic strategies that lead to better performance in a specific domain. Such training is carried out between two sessions addressing the same task, formal or otherwise, that functions as a test and helps to detect academic progress. Thus, although there is an increase in the complexity of academic and behavioral tasks during childhood, skills improve if the learning conditions follow the orientation of this model (Holdich and Chung, 2003, Kolhberg, 1986, Kitchener, 1986 and Vigotsky, 1978), that is, when the learning tasks: (1) are motivating and contextualized; (2) include prior knowledge or hints and clues that the child knows; and (3) are carried out in the presence of more experienced children. Computer-based learning systems can form part of the settings which promote these conditions. After our experience with small children in the classroom, we have observed that the inclusion of these tools in the classroom has important advantages: (1) interactive tasks can be performed that cannot be done with traditional materials; (2) computers motivate children and the feedback received after each action is interpreted as part of the task and not as a penalty; (3) the content and materials can be reused; and (4) the tasks are modeled as a problem-solving workflow composed of a set of steps. Thus, the learning process can be monitored and situations can be detected in which help and intervention can be directly provided by the teacher or automatically, by the tool itself; and (5) both the learning process and results can be studied and analyzed. However, different researchers have questioned the suitability of the use of computers by children aged from 3 to 6 years. As stated by Plowman and Stephen (2003), the question is not “At what age should children use computers?” but “What are appropriate and meaningful uses of technology with children?”. If the technology is used properly it can be a useful tool in the development and learning of young children (Abbott et al., 2001 and Bolstad, 2004). Some studies have proposed guidelines for the development of software for children (D´Mello et al., 2012, Gelderblom, 2004, Isomursu et al., 2011, Mooij, 2007 and Park and Hannafin, 1993), although these guidelines are generic and therefore difficult to apply when the system must be customized to a specific domain and psychological theory of learning. Computer training has been shown to improve working memory and reasoning in children aged 4–5 years, but experiences related to inhibitory control have not been completely successful (Rueda et al., 2005, Thorell et al., 2009, Diamond, 2012 and Gathercole et al., 2006). Few systems have been specifically created to aid in the cognitive development of young children at school (Sung et al., 2008). Recently Cogmed has been the most studied cognitive training software for children with normal development (Cogmed Working Memory, 2014). Cogmed Working Memory Training Software focuses exclusively on training working memory and it is built around three age-specific software applications: JM for pre-schoolers, RM for school-age and QM for adults. Cogmed JM contains a set of visuo-spatial and verbal memory tasks embedded into video games that progressively increase working memory demands. In Thorell et al. (2009) the authors present two computerized training programs developed in collaboration with the company Cogmed. They were used to train both inhibitory control and working memory in pre-schoolers. The system allows the level of difficulty to be set according to two parameters: the time allowed to respond in the first case, and the number of items the student has to remember in the second case. Their study showed that the use of this system to train working memory led to positive effects in training and non-training tasks. Although the children improved in inhibitory control in the trained tasks, the effects could not be generalized to non-trained tasks. In Rueda et al., 2005 and Rueda et al., 2012 a set of computerized exercises specifically designed to train executive attention is presented. The program consists on eleven exercises classified in five categories: tracking/anticipatory, attention focusing/discrimination, conflict resolution, inhibitory control and sustained attention. The results of this study suggest that attention training transfers to fluid intelligence. The scientific evidence showing how computer learning can improve executive functions in young children is normally based on evaluating the effect of training by using psychological intelligence tests or by specific standardized tasks (Diamond and Lee, 2011, Grunewaldt et al., 2013 and Thorell et al., 2009). However, ecological tasks are rarely used, such as the tasks or exercises performed during daily classroom activities. Thus, there are various pending issues such as the training time needed to obtain results, whether inhibition behavior can be trained in preschool children, and whether training could have an effect not only on standardized tests, but also on compulsory school tasks, etc. (Rueda et al., 2005 and Thorell et al., 2009). 1.2. PATIO, a computer based learning framework for pre-schoolers The preschool curriculum is organized into domains related to the children׳s environment such as home, school or family; topics such as colors, sizes, seasons and weather are taught within the context of these domains. At the same time, children have to practice different cognitive skills such as visual perception, spatial orientation, memory and attention and begin to learn reading and writing. At this stage, they practice important social skills such as collaboration, listening to others, and conflict resolution. Therefore, the curriculum has a multidimensional structure. The teacher organizes classroom work using different tasks, games and activities in which subjects, topics, and the practice of different skills are combined. This learning method guided the design of PATIO (Fig. 1) a generic computer-based learning framework for early childhood education. It contains a set of tools for defining (1. Authoring tool), delivering (2. Learning tools), assessing (3. Assessment tool) and monitoring (4. Monitoring tool) learning activities. Its architecture is based on Intelligent Tutoring Systems (Woolf, 2009). PATIO framework architecture. Fig. 1. PATIO framework architecture. Figure options PATIO includes four learning tools: APRENDO (Trella et al., 2008), LEO, ESCRIBO and TRAZO (Barros et al., 2008 and De Diego-Cottinelli and Barros, 2010). LEO and ESCRIBO focus on reading and writing, respectively. TRAZO was created to assist in the development of handwriting skills. APRENDO focuses on other curricular aspects such as basic cognitive skills (e.g. attention or memory development), maths (sets, numbers, etc) and music. 1.2.1. Authoring tool (Fig. 2) Preschool teachers require digital materials that can be adapted and personalized both to the characteristics of individual children and to specific contents. They claim that learning is more effective if the exercises contain characters that are familiar to children, if they use a particular drawing style or if they include aspects related to the local cultural context (food, animals, clothes, etc.). On the other hand, learning is based on an ongoing crossectional approach; that is, children practice the same tasks throughout the course using different contents according to the topic they are focussing on. This working method requires a large set of exercises, mainly because the exercises are short and the children can memorize and repeat them correctly without applying the skills actually demanded by the task. For example, “mark which of these foods are fruits” or “draw a circle around the winter clothes” are typical activities in a preschool class. If we generalize their problem-solving workflow we can identify the pattern “make sets of items that share feature X”. The task model of PATIO provides a set of patterns called tasks. Using the authoring tool the teacher can instantiate a task to create different exercises by including different multimedia resources. The level of difficulty of the task can be adapted to each child by setting parameters such as time (between images, permanence of an image on the screen), amount of visual or audio stimuli or the presence or absence of help and reinforcements. The set of configurable parameters is specific to each task. The domain model includes a set of tags related to the domains and topics of the curriculum that are used to semantically tag the multimedia resources and exercises. Authoring tool. Edition example: task “Find the different objects”, exercise ... Fig. 2. Authoring tool. Edition example: task “Find the different objects”, exercise “Which are not toys?”. Figure options 1.2.2. Learning tools (Fig. 3) These tools are interfaces that (a) introduce children to the exercises, that can be delivered through different media such as a PC, an interactive whiteboard or a PC tablet; and (b) collect all the information regarding the child׳s interaction with the system (Fig. 4). The exercises are stored as .xml files. The learning tool reads the corresponding files and presents the exercises to the child. PATIO provides user management and allows teachers to define the sessions for a single child or for a group. A session is defined as a set of exercises that will be presented to the child one by one in the order specified by the teacher. Learning tool for children. Fig. 3. Learning tool for children. Figure options Log file generated by the learning tool corresponding to the execution of a ... Fig. 4. Log file generated by the learning tool corresponding to the execution of a “Find the intruder” exercise. Figure options APRENDO is one of those learning tools. It consists in an interface that delivers a set of exercises specifically designed for training cognitive skills. The exercises are generated, using the Authoring tool, by adding content (from Multimedia resources database) to tasks (templates from task model). While the child is doing the exercise, APRENDO records each click and event (appearing/disappearing images, beeping sounds, etc.) and stores it in a.xml log file (Fig. 4), which is subsequently processed by the assessment tool of PATIO in order to generate the student model The specific templates and exercises used in the experiments of this paper are described in detail in Section 2.3.1. 1.2.3. Assessment tools The student model consists of a set of pairs (variable, value) that model the skills trained in each task. Thus, the model changes from one task from another. Teachers and psychologists assisted the PATIO designers in identifying a set of observable variables for each of the tasks of the task model. The values of these variables are automatically calculated by the assessment tools based on the child׳s interaction with the system. Thus, for each exercise there are two files describing the child׳s actions during their activities: the log file with the fine-grained data and the student model that contains the values inferred from it. Section 2.4.1 provides details of the user model generated by the attention and working memory exercises used in this paper. The student model can be exported to be analyzed by statistical software. 1.2.4. Monitoring tools (Fig. 5) PATIO offers a set of visual tools to help analyze learning sessions. A log file is created when each child completes each activity and the log is evaluated and stored in the system. Thus, all the exercises done by students can be reproduced and studied (Fig. 5c). The analysis and visualization depends on the type of task. Monitoring tool. Example of a “Find the different objects” exercise results: (a) ... Fig. 5. Monitoring tool. Example of a “Find the different objects” exercise results: (a) variables/values of the student model; (b) actions in the exercise timeline; and (c) reproduction of child interactions based on data stored in the log. Figure options In summary, we present APRENDO to the scientific community as a classroom tool that makes use of the advantages offered by the PATIO learning framework as a tool for development and automatic assessment. We were particularly interested in exploring the suitability of APRENDO and its potential as a training tool for certain cognitive skills needed to succeed in school work, i.e., in the early processes of school learning in normally developing children. As discussed in this section, we evaluated the effect of training on a compulsory task, using the most ecologically meaningful tasks, such as those used by preschool children in the classroom. 1.3. The present study. Objectives This study describes an experiment conducted with children between 4 and 5 years old in preschool who used the APRENDO system activities between typical school tasks. A pre-test/training/post-test experimental design was applied using experimental and control groups. The main aims of this study were as follows: • Objective 1. To test the suitability of the APRENDO system as a learning system for cognitive training in preschool children (Markopoulos et al., 2008). • Objective 2. To conduct a pilot study to explore whether APRENDO can improve two executive functions (attentional control and working memory) in younger children. This improve is evaluated by the effects that APRENDO can produce on a compulsory task. The hypotheses were as follows: • Hypothesis 1. Children in the experimental group will improve their performance during the APRENDO training sessions. That is, the results of the last session will be better than the first. • Hypothesis 2. The APRENDO training will have a positive effect on the executive functions analyzed, as measured by a compulsory school task that requires the use of these abilities, i.e., the children in the experimental group will obtain better post-test results than the children in the control group (no training group).

3. Results The following sections address the aims outlined above. 3.1. Objective 1: to test the suitability of the APRENDO system as a learning system for cognitive training in preschool children As we have seen some variables from actions in the system have been defined. With those variables we have calculated some statistical measures for each training session (S1, S2 and S3). As shown in the descriptive statistics of the first training session (Table 3), a comparison of the children׳s responses shows that there were striking differences between the two APRENDO tasks regarding the same variables. For example, in the “Find the different objects” task, the children took longer to react to the cognitive demand [M (FI-A10)=17.69, M (FD-A4)=22.83], took longer to give a wrong answer [M (FI-A8)=0.48, M (FD-A5)=37.02], and naturally took longer to finish the task than in the “Find the intruder” [M (FI-A)9=29.84, M (FD-A8=67.65)]. Finally, we found that children were less accurate in this task [M (FI-A4)=0.07, M (FD-A3)=1.13] because they clicked more often on incorrect images. Table 3. Descriptive statistics and repeated measures t-test for “Find the intruder” and “Find the different” in the three training sessions. S1 S2 S3 S1–S2 S2–S3 S1–S3 N M SD N M SD N M SD t(29) Sig t(29) Sig t(29) Sig Intruder FI-A1 30 0.70 0.466 30 1.00 0.000 30 1.00 0.00 −3.525 0.001⁎ −3.525 0.001⁎ FI-A2 30 0.00 0.000 30 0.00 0.000 30 0.00 0.00 FI-A3 30 0.00 0.000 30 0.10 0.305 30 0.00 0.00 −1.795 0.083 1.795 0.083 FI-A4 30 0.07 0.365 30 1.83 2.13 30 0.97 2.01 −4.479 0.000⁎ 2.264 0.031⁎ −2.377 0.024⁎ FI-A5 30 0.03 0.183 30 1.17 1.76 30 0.67 1.69 −3.495 0.002⁎ 1.542 0.134 −2.027 0.052⁎ FI-A6 30 1.00 0.000 30 1.00 0.000 30 1.00 0.00 FI-A7 30 17.79 16.30 30 25.66 34.11 30 15.10 8.40 −1.180 0.248 1.776 0.086 0.705 0.487 FI-A8 30 0.48 2.65 30 10.99 20.74 30 4.59 9.41 −2.761 0.010⁎ 1.928 0.064 −2.252 0.032⁎ FI-A9 30 29.84 23.08 30 25.66 34.11 30 15.10 8.40 0.566 0.576 1.776 0.086 3.226 0.003⁎ FI-A10 30 17.69 16.31 30 19.15 23.33 30 13.36 7.14 −0.301 0.766 1.387 0.176 1.202 0.239 FI-A11 30 0.0481 0.26 30 1.73 2.63 30 0.36 0.87 −3.491 0.002⁎ 2.812 0.009⁎ −1.859 0.073 Different FD_A1 30 0.23 0.430 30 1.00 0.00 30 1.00 0.00 −9.761 0.000⁎ −9.761 0.000⁎ FD_A2 30 0.86 1.04 30 0.10 0.40 30 0.06 0.25 3.516 0.001⁎ 0.372 0.712 4.252 0.000⁎ FD_A3 30 1.13 1.41 30 2.60 2.86 30 1.43 1.92 −2.772 0.010⁎ 1.803 0.082 −0.794 0.434 FD_A4 30 22.83 29.06 30 22.94 25.22 30 19.45 35.08 −0.015 0.988 0.449 0.657 0.387 0.701 FD_A5 30 37.02 31.16 30 18.45 27.48 30 14.71 37.29 2.106 0.044⁎ 0.438 0.665 2.579 0.015⁎ FD_A6 30 1.16 6.29 30 1.41 6.39 30 6.88 36.14 −0.151 0.881 −0.809 0.425 −0.848 0.403 FD_A7 30 0.00 0.00 30 0.00 0.00 30 7.78 11.33 −3.761 0.001⁎ −3.761 0.001⁎ FD_A8 30 67.65 1.10 30 34.37 34.07 30 32.78 50.05 1.578 0.125 0.147 0.884 1.729 0.094 FD_A9 30 1.12 6.09 30 0.00057 0.0031 30 0.00 0.00 1.004 0.324 1.000 0.326 1.004 0.324 FD_10 30 0.223 0.864 30 2.489 4.006 30 2.451 6.272 −2.979 0.006⁎ 0.032 0.974 −1.931 0.063 (⁎) p<0,01 Table options Second, Spearman׳s correlation was calculated to determine the relationships between the variables associated with the two APRENDO tasks over the three training sessions. The results are shown in Table 4, which shows how the variables evolved as the children progressed over the training sessions (S1, S2 and S3). As shown, the number and variety of related variables increased over the three sessions in both tasks. Table 4. Find the intruder and Find the different objects Spearman׳s correlation. Find the intruder Session 1 Session 2 Session 3 FI-Ai 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11 1 0.805** 0.805** 2 3 0.383* −0.366* 4 1.00** 1.00** 1.00** 0.958** 0.503** 0.869** 0.503** 0.865** 0.862** 0.585** 0.990** 0.585** 0.840** 5 1.00** 1.00** 0.525** 0.791** 0.525** 0.883** 0.587** 0.834** 0.587** 0.974** 6 7 0.999** 0.568** 1.00* 0.636** 0.554** 0.601** 1.00** 0.759** 0.568** 8 1.00** 0.568** 0.776** 0.601** 0.816** 9 0.636** 0.554** 0.759** 0.568** 10 11 Find the different object Session 1 Session 2 Session 3 FI-Di 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 −0.425** −0.643** 0.484** 2 −0.641** 0.802** 0.492** 0.998** 0.719** 0.999** 3 −0.467** −0.365* 0.586** 0.511** 0.698** 0.843** 0.546** 0.475** 0.856** 4 −0.584** 0.670** 0.774** 0.403** 0.461** 0.904** 5 0.402** 0.773** 0.629** 0.655** 0.639** 0.616** 6 0.377* 0.670** 7 0.616** 0.394* 8 0.538** 0.410* 9 10 Variables (from Table 1): FI-A1: Number of clicks on correct images, FI-A2: Number of clicks outside the image before the intruder is shown, FI-A3: Number of clicks outside the image after the intruder is shown, FI-A4: Number of clicks on wrong images, FI-A5: Number of consecutive clicks on a wrong image, FI-A6: Drop out, FI-A7: Positive latency, FI-A8: negative latency, FI-A10: Total time, FI-A11: Time average between two consecutive wrong clicks, FD-A1: Number of clicks on correct images, FD-A2: Number of clicks outside the images, FD-A3: Number of wrong images, FD-A4: Reaction time, FD-A5: Negative latency, FD-A6: Not meaningful latency, FD-A7: Time average between two consecutive correct clicks, FD-A8: Total time, FD-A9: Time average between two consecutive clocks outside the image, and FD-A10: Time average between two consecutive wrong clicks. Table options The results of Spearman׳s correlation of the final session are summarized in Fig. 11 and Fig. 12. The children׳s actions demonstrated that the relationships between variables are the same in both exercises. That is, variables׳ relationships follow the same pattern in both cases. As shown, the time needed to complete the task in both tasks (total time: FI-A9, FD-A8) correlates with (i) the time until the first click (reaction time: FI-A10, FD-A4); (ii) the time until clicking on the first wrong image (negative latency: FI-A8, FD-A5); and (iii) the average time between two wrong consecutive clicks (FI-A11, FD-10). Moreover, the number of clicks on wrong images (FI-A4, FD-A3) correlates, in both exercises, with the average time between two wrong consecutive clicks (FI-A11, FD-10). Nevertheless, in “Find the intruder” (Fig. 12), there are more variables that correlate with erroneous clicks. Significant relationships between actions and variables in “Find the different ... Fig. 11. Significant relationships between actions and variables in “Find the different objects”. Figure options Significant relationships between actions and variables in “Find the intruder”. Fig. 12. Significant relationships between actions and variables in “Find the intruder”. Figure options 3.2. Objective 2. Can APRENDO improve attentional control and working memory in younger children? 3.2.1. Hypothesis 1. Children in the experimental group will improve their performance during the three APRENDO training sessions This section presents the results related to the first hypothesis. It was predicted that children in the EG would improve their performance, memory, and attentional processes over the training sessions. Independent within-subjects t-tests were conducted for S1–S2, S2–S3, and S1–S3. This consisted in comparing the EG training sessions for each of the two tasks conducted with APRENDO, using the variables or actions described in Table 1. The results are shown in Table 3. First we present the results for “Find the intruder”, next for “Find the different objects” and finally we show classification of children by their performance. Overall, the results show progress between training sessions in the “Find the intruder” task regarding the following actions: number of clicks on the correct image (FI-A1); number of clicks on the wrong image (FI-A4); number of consecutive clicks on a wrong image (FI-A5); time until click on the first wrong image or “negative latency” (FI-A8); time needed to complete the task or “total time” (FI-A9); and the average time between two clicks on wrong images (FI-A11). Progress was observed in the “Find the different objects” task regarding the following actions: number of clicks on the correct image (FD-A1); number of clicks outside the images (FD-A2); time until the first click on a wrong image or “negative latency” (FD-A5); and average time between two consecutive correct clicks (FD-A7). In addition, it can be observed that, in terms of the children׳s performance, more variables reach a statistically significant difference between S1 and S3 than between S1 and S2. The results of the variables “Answer accuracy” and “Answering time” show statistically significant positive differences in both tasks. In the “Find the intruder” task, the children take longer to fail, i.e., to select an incorrect image, between S1 and S2/S3. On the other hand, in the “Find the different objects” task, there was a statistically significant reduction in time taken between S1 and S2/S3, i.e., the children took more time to select an incorrect stimulus in S1 and less time in S2 and S3. There was also a statistically significant change in the variable measuring the average time taken between clicks on incorrect images (FI-A11) in the “Find the intruder” task; the average in S2 was higher than in S1 [t (29)=−3.491; P=0.002], and the average in S2 was lower than in S3 [t (29)=2.812; P=0.009]. Clearly, the average in S3 was higher than in S1 [t (29)=−1859; P=0.073]. Thus, a comparison of the training sessions shows there was a reduction in the time needed to complete the APRENDO task. However, the results are statistically significant only for the “Find the intruder” task. Moreover, in this task there was also a statistically significant difference in the number of clicks on a previously-clicked incorrect image (FI-A5) as shown by the difference in averages from S1 to S2 [p<0.01 t (29)=−3.495], and also from S1 to S3 [p<0.05 t (29)=−2.027]. As shown in Table 3, there was a significant decrease between training sessions in the variable “number of clicks outside the images” (FD-A2) in the “Find the different objects” task, i.e., there was a decrease in the number of clicks that were unrelated to the stimuli. The children rarely clicked outside the images before the intruder or after the intruder appeared. An interesting result was that in “Find the intruder” task there was no statistically significant change in the variable “number of clicks outside the image before the intruder is shown” (FI-A2) in S1–S2, S2–S3, or S1–S3. The average was zero in all these sessions. There was no change in the variable “Drop out” (FI-A6) in “Find the intruder” task between all the training sessions. Significant changes were found regarding the following actions: click on the right image; do not click on the wrong images; time until the wrong image is clicked; average time between clicking on wrong images; time needed to complete the exercise; repeated wrong clicks (in “Find the Intruder”); and clicks outside the images (in “Find the different objects”). The children were classified as “fast or slow” and “accurate or inaccurate”, according to their performance in each exercise. This was done by obtaining the Z score of the children for two selected sub-variables used to calculate answer accuracy (V1) and answering time or speed (V2) described in Table 1. To classify the children as “fast or slow” we used: (1) the time needed to complete the task (actions FI-A9 and FD-A8, for “Find the intruder” and “Find the different objects”, respectively); and (2) the number of clicks on a wrong image (actions FI-A4 and FD-A3, for “Find the intruder” and “Find the different objects”, respectively). The values were normalized, taking into account the Z value for the “time needed to complete both exercises”. If Z<0.5 the child was classified as “slow” and if Z>0.5 as “fast”; the children with intermediate values were not taken into account for two reason. First, because it is a normal and recognized method in the measure of variables in child psychology on order to classified children in many developmental or educational areas (e.g. IQ, Intelligence Quotient) to aiming to make and plan many evaluation and training programs. Second, it is a initial study and, due we had few children, we want now know and follow or monitor the performance of extremely groups. For others new studies we could study the performance of the intermediate values. The same method was used to classify children as “accurate” or “inaccurate”. The results are shown in Table 5. Table 5. Number of children. Classification of the participants: fast-slow, accurate-inaccurate. Training session S1 S2 S3 Intruder Speed Fast 23 27 28 Slow 7 3 2 Accuracy Accurate 1 11 6 Inaccurate 0 12 0 Different Speed Fast 24 25 28 Slow 6 5 2 Accuracy Accurate 2 6 5 Inaccurate 5 13 14 Table options As shown in Table 5, over the training sessions the number of children classified as “fast” increased and those classified as “slow” decreased. This occurred in both tasks. However, the variable “answer accuracy” (V1) shows that there were more “accurate” children in S2 than in S1. Nevertheless, the value of this variable decreased between S2 and S3 rather than continuing to increase. In “Find the intruder” the decrease was greater. This is related to the decrease in the number of “inaccurate” children between S2 and S3. In the next step, the Wilcoxon signed rank test was applied as a statistical method to compare the progress over the three training sessions. The results are displayed in Table 6, which shows that there were no significant differences between sessions in speed or accuracy in any of the exercises. As Table 5 shows, although children changed from one group to another during the training sessions, there was not the same progress at the within-subject level. In the “Find the intruder” task there was a trend toward statistical significance in the variable “answer accuracy” (V1) from S1 to S3. Table 6. Within-subject Wilcoxon signed rank test. S1–S2 S2–S3 S1–S3 Z Sig Z Sig Z Sig Intruder Speed −1.414 0.157 −0.577 0.564 −1.667 0.096 Accuracy −0.408 0.683 −1.400 0.162 −1.890 0.059 Different Speed −0.333 0.739 −1.134 0.257 −1.414 0.157 Accuracy 0.785 0.433 −0.371 0.710 −1.189 0.234 Table options 3.2.2. Hypothesis 2. The APRENDO training will have a positive effect on the executive functions analyzed, as measured by a compulsory school task that requires the use of these abilities The second hypothesis was that children from the EG would perform better in the school task during the post-test session than children from the untrained CG. We have calculated a two-way ANOVA for the quantitative variables and descriptive statistics for the qualitative ones. The results displayed in Table 7 show that the CG and EG did not significantly differ in any of the variables of the school task. Table 7. Two-way ANOVA for the variables of the post-test school task in the EG and CG. CG EG N M SD N M SD F(1) Sig CA 22 7.8636 2.12234 30 7.9667 2.09241 0.030 0.862 CA1 22 3.1364 1.69861 30 3.4667 1.56983 0.524 0.472 CA2 22 3.7727 0.42893 30 3.5667 0.85836 1.068 0.306 CA3 22 0.8636 0.88884 30 0.9000 0.92289 0.020 0.887 M 22 8.2727 4.15396 30 8.1667 6.25373 0.005 0.945 M1 22 3.6364 1.94068 30 3.4667 1.54771 0.123 0.727 M2 22 0.2727 0.55048 30 0.4000 0.85501 0.373 0.544 M3 22 3.6364 3.61933 30 3.2000 5.05419 0.119 0.732 M4 22 1.1364 0.94089 30 1.0667 0.90719 0.073 0.789 CAM 22 −0.4091 5.64556 30 −0.2000 7.80981 0.011 0.915 Variables (from Table 2): CA: Number of correct answers, CA1: Number of times the child “crosses out the absurd”, CA2: Number of times the child “paints the right clothes”, CA3: Number of times the child “does not color the wrong clothes”, M: Number of mistakes, M1: Number of times the child “does not cross out absurd objects”, M2: Number of times the child “does not color the clothes right”, M3: Number of times the child “crosses out wrong or irrelevant objects, or aimlessly doodles”, and CAM: Difference between correct answers and mistakes. Table options We calculated progress in the children׳s level of attention, the strategies used to deal with the task, and the final results. In addition, we calculated improvements in the attentional level (AL) and strategies (S) described in Table 2, leading to three categories (Table 8): AL-I (Attentional Level Improvement), S-I (Strategies Improvement), and P (Performance). Each category had four possible values: “−“ (became worse), “=” (no improvement), “+” (average improvement) and “++” (strong improvement). Table 8. Descriptive statistics of the school task (attentional level, strategies and performance). CG EG Count % in AL-I % in Group Count % in AL-I % in Group AL-I – 2 33.3% 9.1% 4 66.7% 13.8% = 11 45.8% 50.0% 13 54.2% 44.8% + 9 42.9% 40.9% 12 57.1% 41.4% Total 22 43.1% 100.0% 29 56,0.9% 100.0% Count % in AS-I % in Group Count % in AS-I % in Group S-I – 3 42.9% 13.6% 4 57.1% 15.4% = 7 41.2% 31.8% 10 58.8% 38.5% + 12 50.0% 54.5% 12 50.0% 46.2% Total 22 45.8% 100.0% 26 54.2% 100.0% Count % in P % in Group Count % in P % in Group P – 0 0.0% 0.0% 2 100.0% 6.9% = 8 53.3% 36.4% 7 46.7% 24.1% + 14 41.2% 63.6% 20 58.8% 69.0% Total 22 43.1% 100.0% 29 56.9% 100.0% Variables: AL-I. Attentional Level Improvement, S-I: Strategies Improvement, and P: Performance. Table options A comparison of the percentages shows that more children in the EG than in the CG improved their attentional level and performance in the school task between the pre-test and post-test sessions. Similar percentages of children from the EG and CG improved their strategies, although more children in the EG maintained their scores. In total, 57.1% and 42.9% of children in the EG and CG, respectively, improved their strategies. Finally, regarding the final score, 58.8% of the children who improved were in the EG and 41.2% were in the CG. We have realised that a group of children in the experimental group obtained poor results in the pre-test school task. It is particularly important for teachers and educational psychologists to know the potential of underperforming children who have not yet been diagnosed with any disorder or deficit to catch up with their peers. Therefore, we decided to look at those children in more detail in the post-test task. Children who were one standard deviation below the average in each variable of the school task were defined as having poor performance and their results were analyzed. Thus, these children were identified and their performance was studied on a case-by-case basis in the post-test session (Table 9). The data show that more than half of these children improved during the post-test. Thus, there were improvements in the majority of the evaluated variables. This allowed us to reclassify children from the group of “poor performance children” to the “good performance children”. For example, 75% of the 4 children in the pre-test session improved their performance for the variable “number of correct answers” (CA) and were reclassified into the normal group. Similarly, 60% of the 5 children with difficulties in the variable “number of mistakes” (M) in the pre-test session improved in the post-test session. Similar outcomes occurred regarding the other variables in the school task. On the other hand, the majority of the children did not improve in two variables: the “number of times the child does not color the wrong clothes” (CA3); and the “number of times the child colors the wrong clothes” (M4). In CA3, only 22.3% improved in the post-test session and in M4 only 23% improved. Table 9. Number and percentage of children who passed from Pretest “difficulties group” to the Postest “normal group” according to the variables observed in the compulsory school task. Variables observed Number of children in “the difficulties group” Percentage of children who improved: i.e. passed from the “difficulties group” to “normal group” Pretest Postest CA 4 1 75% of all children improved M 5 2 60% of all children improved CAM 5 2 60% of all children improved CA1 4 4 0% of all children improved CA2 3 0 100% of all children improved CA3 9 7 22.3% of all children improved M1 4 1 75% of all children improved M3 5 2 60% of all children improved M4 13 10 23% of all children improved Variables (from Table 2): CA: Number of correct answers, CA1: Number of times the child “crosses out the absurd”, CA2: Number of times the child “paints the right clothes”, CA3: Number of times the child “does not color the wrong clothes”, M: Number of mistakes, M1: Number of times the child “does not cross out absurd objects”, M3: Number of times the child “crosses out wrong or irrelevant objects, or aimlessly doodles”, and CAM: Difference between correct answers and mistakes.