Multimodal interface for mobile applications for older adults

This project was the center point of my MSc thesis, and it was a part of ongoing research conducted at the Delft University of Technology. The target of the project was to develop and evaluate a multimodal application for older adults on a mobile platform. The project required a lot of different skills to be applied: graphic and audio design, software and database engineering techniques, knowledge of usability in general and in relation to older adults in particular and knowledge of varied evaluation methods. An experimental, but fully functional application was built in order to study effects of multimodal feedback.  The project had as an ultimate goal to understand if a multimodal interface would support the use of a Diet Diary for older adults.

Application:  The application was a diet diary, designed to help the users to keep a log of their nutritional intakes and provide advice and suggestions.  The application was written in C# for Windows Mobile.  The application data was stored in locally as XML. The graphics and sound in the text was designed specifically to convey the same information that was present on screen in text. This means that all the buttons and concepts in the application were accompanied with a representative icon and all the text on the screen was also presented through pre-recorded speech. The speech was pre-recorded instead of synthesised in order to minimize misrecognition and to appeal to older adults using native Dutch speaker tone.  These modalities could be turned on and off individually, which allowed for a comparison in user experience between different combinations of feedback modes.

The Observation Room

Evaluation:  The application was evaluated with a large number of older adults. The main areas of interest for the evaluations were: effectiveness, efficiency and satisfaction.  The evaluation took place in the TNO Experience Lab in Soesterberg.  It consisted of a living room equipped with two cameras and fully furnished. It also provided an observation room with all the necessary equipment.   A PDA with application was given to the 30 users to conduct the test.

All participants were over 45 years old and they had a basic experience in using computers. Demographic questionnaires were used to select participants.   The within subjects experiment had 4 conditions and the data collected came from observations, log files, NASA TLX results per scenario and interviews.

The Home Lab

Careful analysis of the data based on statistics showed no significant differences  between various combinations of modalities, but interesting results were concluded with qualitative analysis.  This analysis showed that these particular users displayed strong preference for graphics and generally disliked the speech modality.  It was shown that getting to know the users and spending time to understand them provides data that could be invisible  in statistical terms but invaluable in product and systems’ design.

I learned a lot during this project, about usability, about software development on mobile platforms and about methodology behind scientific research.

← Back