Usability and User Research


May 10, 2012 @ 14:30, Room: 12AB

Chair: Anna Cox, University College London, UK
Identifying Usability Issues via Algorithmic Detection of Excessive Visual Search - Paper
Community: user experience
Contribution & Benefit: Presents an evaluation of algorithms for the automated detection of excessive visual search, a technique that can be utilized to aid in the identification of usability problems during usability testing.
Abstract » Automated detection of excessive visual search (ES) experienced by a user during software use presents the potential for substantial improvement in the efficiency of supervised usability analysis. This paper presents an objective evaluation of several methods for the automated segmentation and classification of ES intervals from an eye movement recording, a technique that can be utilized to aid in the identification of usability problems during software usability testing. Techniques considered for automated segmentation of the eye movement recording into unique intervals include mouse/keyboard events and eye movement scanpaths. ES is identified by a number of eye movement metrics, including: fixation count, saccade amplitude, convex hull area, scanpath inflections, scanpath length, and scanpath duration. The ES intervals identified by each algorithm are compared to those produced by manual classification to verify the accuracy, precision, and performance of each algorithm. The results indicate that automated classification can be successfully employed to substantially reduce the amount of recorded data reviewed by HCI experts during usability testing, with relatively little loss in accuracy.
An Evaluation of How Small User Interface Changes Can Improve Scientists' Analytic Strategies - Paper
Contribution & Benefit: We presented results from a quantitative user study showing that controlled changes in the interface of an analysis system can be employed to correct deficiencies in users' analytic behavior
Abstract » Subtle changes in analysis system interfaces can be used purposely to alter users'
analytic behaviors. In a controlled study
subjects completed three analyses at one-week intervals
using an analysis support system. Control subjects used one interface in
all sessions. Test subjects used modified versions
in the last two sessions: a first set of changes
aimed at increasing subjects' use of the system
and their consideration of alternative hypotheses;
a second set of changes aimed at increasing the amount of evidence collected.
Results show that in the second session test subjects used the interface $39\\%$ more
and switched between hypotheses $19\\%$ more than in the first session.
They then collected $26\\%$ more evidence in the third than in the second session.
These increases differ significantly ($p<0.05$) from near constant control rates.
We hypothesize that this approach can be used in many real applications to guide analysts unobtrusively towards improved analytic strategies.
Mouse Tracking: Measuring and Predicting Users' Experience of Web-based Content - Paper
Contribution & Benefit: Demonstrates that mouse-tracking offers valuable signals about user attention and experience on web pages, and can even help detect user frustration and reading struggles. Applications include evaluating content layout and noticeability.
Abstract » Previous studies have used mouse tracking as a tool to measure usability of webpages, user attention and search relevance. In this paper, we go beyond measurement of user behavior to prediction of the resulting user experience from mouse patterns alone. Specifically, we identify mouse markers that can predict user frustration and reading struggles at reasonably high accuracy. We believe that mouse-based prediction of user experience is an important advance, and could potentially offer a scalable way to infer user experience on the web. In addition, we demonstrate that mouse tracking could be used for applications such as evaluating content layout and content noticeability; we apply this in particular to advertisements. More generally, it could be used to infer user attention in complex webpages containing images, text and varied content, including how attention patterns vary with page layout and user distraction.
Evaluating the Benefits of Real-time Feedback in Mobile Augmented Reality with Hand-held Devices - Note
Contribution & Benefit: Adding real-time feedback to a mobile Augmented Reality system to reflect the status of the physical objects being manipulated improves performance by reducing the division of attention.
Abstract » Augmented Reality (AR) has been proved useful to guide operational tasks in professional domains by reducing the shift of attention between instructions and physical objects. Modern smartphones make it possible to use such techniques in everyday tasks, but raise new challenges for the usability of AR in this context: small screen, occlusion, operation ``through a lens''. We address these problems by adding real-time feedback to the AR overlay. We conducted a controlled experiment comparing AR with and without feedback, and with standard textual and graphical instructions. Results show significant benefits for mobile AR with feedback and reveals some problems with the other techniques.
How Do We Find Personal Files?: The Effect of OS, Presentation & Depth on File Navigation - Note
Contribution & Benefit: A large scale study testing the effects of OS, interface presentation and folder depth on personal file navigation. Informs improved folder system design by increasing efficiency in finding files.
Abstract » Folder navigation is the main way that computer users retrieve their personal files. However we know surprisingly little about navigation, particularly about how it is affected by the operating system used, the interface presentation and the folder structure. To investigate this, we asked 289 participants to retrieve 1,109 of their own active files. We analyzed the 4,948 resulting retrieval steps, i.e. moves through the hierarchical folder tree. Results show: (a) significant differences in overall retrieval time between PC and Mac that arise from different organizational strategies rather than interface design; (b) the default Windows presentation is suboptimal � if changed, retrieval time could be reduced substantially and (c) contrary to our expectations, folder depth did not affect step duration. We discuss possible reasons for these results and suggest directions for future research.