Interaction design for learning

Engagement points for interaction designers to make technology count for education.

“It’s nothing short of a miracle that modern methods of instruction have not yet entirely strangled the holy curiosity of enquiry.” ~ Albert Einstein

Recent technological developments in the field of ICT have a major impact on our work and social experiences. With the help of interaction designers and their strategies to make technology work for the human condition, we can rightly claim that ICT tools add real value to our lives in various areas. Unfortunately, this is not the case when looking at education. Here, it seems as though things are moving slower than anywhere else. In the following article, which is based on my presentation at IxDA conference this January in Toronto, I’d like to propose various engagement points for interaction designers to make technology count for education.

A “user experience” of schooling

I did not enjoy school. To me, primary and high-school were not stimulating environments, they did not motivate me to learn and grow. School seemed like a stagnant bureaucracy of content, a prison of ideas, where everybody lived and thought in little boxes. The strange thing is; that most of what I can remember is linked to social experiences and interaction. In my memory, clearly, good or bad experiences were construed around the aura of specific teachers and their means to teach, not around the learning topics. In fact, I do remember most teachers but not essentially what they taught; I don’t remember any “aha” moments.

Today, researchers call for 21st century learning, which teaches skills around media literacy, collaboration and communication, the ability to constantly challenge the status quo and innovate if necessary, problem solving, responsibility and ethics. It’s not about content anymore; it’s about getting the tools and motivation for life-long learning.
But 30 years after my own personal learning experience, my daughter Maia started Grade 3. It is hard to believe, how little had changed. The lesson plans look oh-so-familiar with 90 minutes of English, 90 minutes of math, and 90 minutes of art class after that.

So, how come that our education system seems to be so change resistant? I won’t be able to answer this in this article, but maybe some of the interaction concepts and technologies I mention later could be positively disruptive to the education dinosaur; just the way our social and work life have enjoyed the ripples of the technological novelties of the last decade. It is clear that change needs to happen, when summarizing the dominant factors of the prevalent learning experience.

The prevalent learning experience

• Learning is seen as work, even by primary school kids
• Learning is disconnected from ‘real-life’ experiences of learners
• Learning is usually restricted to a classroom
• Learners are divided into age groups; irrespective of personal development
• Learners have little or no influence on learning content and methods
• An insufficient integration of information technology and interfaces

My wish list for 21st century learning experience

• Learning should be rewarding and propelled through curiosity inherent to human nature
• Learning should be linked to real world experiences and challenges
• Learning should be open to the activities outside of the classroom (e.g. flipped classroom)
• Learn groups should be flexible toward personal interest and progress
• Learners should influence what they learn and how
• Learners should have access to state of the art technology

So, what is the scope of interaction design for learning?

Whilst everybody talks about changing learning through technology, I am more concerned about the successful integration of technology to support the condition of learning through interaction design. As we speak policy makers spend huge budgets to fit out computer labs for the sake of introducing technology to schools oblivious to the fact that computer labs and the particular interaction this enforces are a thing of the past.
A report by the New Media Consortium (NMC) released this January suggests six key technologies for learning in pre-college education (K-12) to be adopted within 1-5 years. It becomes obvious that the effectiveness of these technologies is tightly linked to interaction design.

Short-term (within 12 months)

• Mobile devices and apps
• Tablet computing

Mid-term (within 2-3 years)

• Game-based learning
• Personal learning environments (PLE’s)

Long-term (within 5 years)

• Augmented reality
• Natural user interfaces

In the following passage, I am going to explain the potential of these technologies laying out some challenges for interaction design whilst bringing several examples of our work at Formula D interactive.

Mobile devices and apps & Tablet computing for learning

How easily tablet computers integrate into existing classroom scenarios has been proven many times. In one example of iPads being used in a school in South Africa (A project by the Core Group), kids in Grade 7 even created their own apps. The unobtrusive functionality and interaction with tablet computers is a huge leap forward in a classroom setting, where pupils had to move into special computer labs in the past in order to use computers.

On the other hand, tablet computers with educational apps have an even greater potential; specifically in the light of limited access to formal education in developing countries, which has been proven by Nicholas Negroponte’s latest experiment in 2 (two) rural villages in Ethiopia. 20 illiterate rural children aged 8 to 10, who reportedly had not even seen road signs before, had been given tablet computers with educational apps and absolutely now instructions. 6 months later, the kids were found inseparable from their devices and being able to spell and pronounce English words. One child even bypassed a security setting which locked specific apps to the home screen and limited access to others. The 10 year old who could not read and write, reportedly activated the camera (which was disabled) and started to take video of his family. I find this a very inspiring example. It shows that kids can learn for themselves when given the right tools.

Game-based learning

Play and games are fundamental to learning. When my younger daughter had her first day at school last week, the Dean of the school addressed the 7-year olds at the welcome function. I almost burst out in laughter when he exclaimed, that this now, would be a new chapter in the life of the kids. “The chapter of learning.” Sadly, this view underlines a serious problem; there is still a perception out there that plays and games (the main activity of the kids before school), are somewhat inferior to the “real” learning in school. In fact play and games are predominantly about learning. Soon after birth kids engage in play, which can be categorised as functional, constructive and role play. All these interactions are important learning experiences. Starting from the age of 4, kids “up their game” and start to play to a prior agreed set of rules: They now use games to learn. Discipline, comprehension, negotiation, knowledge application, cognitive and sensory skills, and abstraction; All of these skills are required and mediated by almost any game. With this in mind I can confidently claim, that even the most violent computer games (despite the side-effects) have some positive learning in them. The understanding of the great potential of learning games has reached the highest level of policy makers up to the presidential office in the US, who is promoting and advocating the field of Computer-based learning games.

For example, we developed the Eskom Energy Planner Game for the main South African parastatal energy provider. Players in this SimCity-type environment are tasked to manage energy supply and demand of a city. By building power stations and keeping an eye on budget, environment and citizen happiness, players learn about advantages and disadvantages of different energy sources. However, energy demand can be influenced by implementing programs and equipment to save energy. The game mechanics of the Eskom Energy Planner are not new. There are hundreds of building and breeding games out there. The interesting aspect of this project is that data deciding the game play is based on realistic figures. We worked closely with one of the client’s engineers with the result that the game is pretty close to a simulation. Initially thought to be part of a public awareness campaign, the game is currently being rolled out to schools to evaluate the learning value in a school environment.

Personal learning environments (PLE’s)

One could describe the current K-12 educational system as “Learning Highway”. Everyone goes pretty much in the same direction at the same speed and is evaluated using the same one-fits-all assessments. In primary school, it is sometime a single teacher who is in sole control of the learning path of kids over 2 years old. Obviously, in such a system there is very little room to accommodate individual learning needs. In fact there is a danger of learners being put-off learning and utterly misguided. Learning in this closed system is unsuitable for the acquisition of an essential 21st century skill set.

The NMC horizon report mentions Personal Learning Environments as one of the key technologies to be adopted in formal education within 2-3 years. The idea of a PLE is fascinating; to me it seems this direction could have a major impact on learning in the near future. Although there seem to be a different interpretation of what a PLE should comprise of with manifold affordances, the need for smart technology integration with adequate interaction design to support personalized learning is evident. Research confirms that people learn differently. Also, the learning process cannot be contained to a classroom anymore, it happens everywhere, and “just in time”. In fact, I believe that an electronic personal learning environment could one day replace what we know as schools today.

A few years back, a South African Bank who runs an academy preparing smart kids to become smart bankers, approached us with the challenge to design a collaborative electronic learning platform to allow for content sharing, publishing and debating. To my knowledge the project has never gone live, but there are a few ideas, which I think could stimulate discussion around personal learning environments. Inspired by Netvibes which had just launched at the time, we aimed at creating a personalised front end with data feeds according to user activity and profile settings. The other characteristic of the interface is a 3D data space with icons representing different topics. Topic clusters of interest to a user would dynamically move closer to the user’s individual “world view” as we called it. Users would also be able to save and change between multiple “views” or share these spaces.
Maybe the most interesting features in this concept are that information preferences or filters set by users where fuzzy instead of on/off. Users would have sliders which could be dragged left and right on a bar to indicate a “felt” level of interest for a specific topic. The 3D icon map or their “world view” would instantaneously. However, a good personal learning environment should be able to do a lot more.

A sketch of the future Personal Learning Environment

Imagine a learning environment that connects and harnesses a host of distributed activities and projects across the city and web, which can be selected or booked by learners according to their self-guided, teacher-guided, and data-informed learning paths. Activities like reading a blog or a book on Google Books, signing up for a math class, visiting a museum or doing an internship at a local company, would be logged in the learning path, acknowledged and credited. Traditional schools would focus on producing good content and exciting projects that could be individually booked by learners, instead of trying to fit everything into one curriculum.

The system would be equipped with powerful data-mining tools, which would dynamically evaluate and visualize the impact of personal learning decisions and activities with respect to an assumed career path if applicable. Scheduled peer, tutor or master assessments would certify key learning points. Gamification strategies would allow for virtual and real-world rewards for progress. Already at a young age, good students could become tutors or teachers in fields they are good at. Peer to peer interaction would always end with a rating of the interaction, so learners and teachers will gain credibility points just like vendors on eBay for successful transactions.

I imagine the main components of a Personal Learning Environment would be:

• Education “Health” status: Ambient visualization of Learning progress (learning ‘garden’, tree, tamagotchi)
• Learning timeline: “Cradle to grave”
• Learn Zone / ‘here and now’ focus area with various content feed, publishing and collaboration tools (visible to invited peers or tutors)
• Module mall (market place for learning activities ranging from real-world activities like museum visits to global online projects, learning games etc)

• Scheduling assistant
• Peer network and interaction tools
• Learning path tool mining live data from current careers
• Evaluation Centre: learning check points or evaluation from peers and teachers. Compare to self-assessments.

Augmented Reality for learning

During the past holiday, my kids and I used AR to learn about the stars we saw when we took a break on a farm near Cape Town, which was far enough from the city lights to expose views on the most amazing night skies. How often in my life had I looked at stars or constellations and had no clue what I was looking at. With SkyView and similar apps this has changed. Telling by the effort that has gone into the great animated MARS discovery movie, one can tell that NASA has realized that science communication is critical in order to stay funded by the public sector. What about an AR multiplayer game, which involves tracking and navigating of virtual space shuttles across a real sky whilst informing players about science facts?

No doubt, Augmented Reality will be another powerful technology used for learning. The NMC report estimates the adoption of AR into mainstream learning within 2-3 years. AR allows us to embed digital content into our physical world with increasingly slimmer and less intrusive interfaces. Mobile devices with cameras, GPS and gyroscope are increasingly abundant opening the field for AR applications. The benefit for learning is obvious. Through AR, knowledge and information is highly contextualised. Real objects, spaces and events can be tagged or augmented with media and personal experiences “in situ” allowing learners to make immediate connections and learn “just in time”.

In other AR or mixed reality interaction concepts, physical objects or tools that are familiar to the users are being augmented with technology driven functions that often involve multimedia content.

In our Recycling floor projection app, players use a physical broom to sweep virtual waste into the right containers. Our De Bono app teaches user the principle of De Bono’s different thinking hats by showing scenarios which change according to which colour hard hat a user wears.

Natural user interfaces for learning

A lot of the downfalls of traditional WIMP interaction become even more evident in learning environments if compared to professional settings.
Two aspects are particularly problematic and are for me central to developing more intuitive interaction frameworks:

Co-located collaboration

Most devices and apps are still only supporting single user experiences.

Multi-user environments are one of the most interesting challenges I can think of in an interaction design, since this requires us to design interaction frameworks and Information architecture where users get individual access, can share and collaborate within the same space, which is completely different to the ways we are used to using computers.

Inclusive interfaces

Secondly, there is inclusivity. The fact that thanks to more direct interaction via touch, we see toddlers, old and illiterate people finding access to computer technology, shows the power of natural user interfaces for inclusivity. Still, we can’t stop here: Interactions will be based on voice recognition, gesture, mimics and eye-tracking, all of which are keys to natural user interface technology.

When looking at a developing country like South Africa with 11 official languages and a high level of illiteracy, natural user interfaces could provide marginalised communities with access to technology.

As early adopters of multi-touch technology (amongst the first in South Africa), we introduced our first touch wall interface at the Two Oceans Aquarium in Cape Town, just a few months before the iPhone landed in South Africa.

The Frog wall teaches visitors about local frog species’. The experience is dominated by a shared environment, which comprises of an animated immersive landscape. From here, up to 3 users can explore various activities simultaneously. Users can either explore together or find their own corner on the screen. Even, when engaging in different activities, the environment still supports awareness of experiencing the same space.

Every now and then a central activity (thread quiz) freezes all other interaction on the screen to encourage the collaboration of users.

Another example for a multi-user application is a 5-metre touch table for a museum in Saudi Arabia, which works for up to 12 users simultaneously. Surely, these applications have not been designed for classrooms, yet, they give us an idea what kind of collaborative platforms for learning we could be developing in the near future.

Tangible interfaces

Manipulating objects is one of the most elementary forms of interaction. Already babies engage with objects exploring their properties through functional play. This is why natural user interface concepts should always consider tangible interaction. One of the huge advantages of digital technology in learning

environments is to provide access to contexts and material without requiring direct access to the authentic situation or object. Many tangible interaction concepts for learning make use of reference objects which are substitutes for “the real thing”. In comparison to Graphical User Interface elements like icons, these references can be far more descriptive, making use of texture, weight, temperature, position in 3D space and other properties. This not only can help make interaction clearer, it is also known to increase the ability to memorise content and contexts.

As a flexible prototyping environment for certain tangible interactions, we adopted the ReacTIVision platform in several projects, the most recent being the Virtual Chemistry Lab table.

The Virtual Chemistry Lab Table is an interactive multimedia learning tool that uses tangible objects to navigate and control a simulated chemistry lab. Young learners can explore chemical experiments and demonstrations without having access to physical laboratory facilities and equipment. The system is designed to cater for up to 2 learners at the time that can trigger chemical reactions by placing tokens that represent chemical elements and catalysts in proximity to each other. The reactions can then be further explored with tool tokens.

Having received good feedback from Science Centre visitors, we explored variations of the Lab that would be more suited for schools. Considerations included cost implications as well as flexible use in a classroom setting. So we created a smaller low cost version (a webcam in a box + reference objects + software), that could plug into existing computer hardware.

Although this concept seems to be a step back toward the desktop and screen setup, the feedback and activity displayed by these girls from a Cape Town Junior School, suggest that the interface works very well in collaborative situations.

We are currently working on converting the software to be usable on the Microsoft surface tables with PixelSense, a technology which will overcome major constraints of camera-based systems like focal distance to the object, and ambient light.

When discussing the project with science teachers, it became clear that a tool like this is not a true alternative to actual chemical experiments. Pre-rendered chemical experiments are suitable for demonstrations and introductions to chemistry, but cannot be used for actual scientific experimentation, since the outcomes are predetermined.

However, the addition to a content management system, currently in development, empowers learners to create their own virtual experiments. With this functionality the tool becomes a modeling and demonstration tool, which would be used to document and share chemistry research using other media or in a real Chemistry Lab.

Conclusion

Formal education is in a crisis. Educators and activists call for a radical shift to new learning and teaching methods. As technology tools promise new opportunities to innovate learning, it is up to us interaction designers to design the right interfaces, which are centered on the learner and development of skills relevant for the 21st century.