KIT724 Assignment2 -AR

2 years ago by xwu26

Share this post:

A Report on the “Play and Learn Vocabulary” AR Application

1. Introduction

For education purposes, this “Play and Learn Vocabulary” application could integrate with certain vocabulary books, to help learners like kids and those who learn English as a second language to arouse their interest and have fun while studying.

2. Application Description

Traditional vocabulary books are simple and boring, the word lists (shown in Figure1) and the words with images (shown in Figure2) are less attractive. Learners can hardly pay much attention to them. Memorizing the meanings and spellings is sometimes painful and not efficient. There are no engagements and interactions between learners and the books.

The “Play and Learn Vocabulary” application is designed to improve this situation. A lot of research has indicated that real-world environments could significantly increase the efficiency of learning new stuff. Obviously, virtual 3D models are more interesting, lively and vivid than plain text.

To achieve the above goals and build attractive visualizations, this application involves AR technologies, presents a game-like environment, and provides an immersive experience for users. The idea is to build multiple interfaces such as various animation avatars, environments, effects, actions, and audios that are relevant to the content. And users can operate the models by clicking, spinning, zooming in, sliding the screen and so on.

Figure 1. A word list from an English learning website (Tree Valley Academy 2022)

Figure 2. A page from an English vocabulary book (AliExpress 2022)

3. Interaction Design

Following is a simple storyboard of one of the use cases in which the “Play and Learn Vocabulary” application is designed to be implemented.

For kids and those who just start to learn English, interesting materials can make them enjoy learning and develop an active attitude. Users are barely engaged with traditional 2D books. Augmented reality can add 3D models and many user-engaged interactions. It is highly interactive with the actual environment in real-time. It can enhance the perception between users and books. The 3D objects, visualized change and playable environment can stimulate the brain and nervous system to deep reinforce the learning process.

In the submitted demo project, there are three cards (shown in Figure 3) that stand for parts of the image on these certain books. After installing the application on their computer or mobile phone, the user turns on the camera, cheerful game background music will arise. If the user puts the camera above the first card, a cute 3D car model will pop up with the text “Car”, if the user presses the virtual button “Green”, the car model will change to green colour (shown in Figure 4). The other two virtual buttons and the other two cards have the same functions.

A lot more interactions could be developed and implemented in this application. For example, when the English words show up or are chosen by users, the pronunciation audios will arise. When users learn animal words, the 3D dynamic animal will appear, and users can operate them to jump, run or fly. Real-life situations and sentences that involved the words could be built.

Figure 3. Three cards that used in the sample project.

Figure 4. A screenshot of the running application.

4. Technical Development

The submitted project is developed with Unity (3D) using Vuforia and AR camera. It is developed in a Windows environment and tested using a webcam. When the camera captures the card, 3D models will appear at the same position as the vehicle in the image, along with the English words at the top of the model. When any of the virtual buttons is pressed, the model will change to the corresponding colour. The background music, three cards with three English words, three 3D vehicle models, nine virtual buttons and nine render colour interactions are working successfully during the test. The sources of models, audio and images are listed in the references section.

5. Descriptions of 3D Models

Figure 5 is a small thumbnail image of the three 3D models used. Their cute and adorable appearances, which are very like real-world car toys, fit this project perfectly.