Chair: Tessa Lau, IBM Almaden Research Center, USA
Tell Me More? The Effects of Mental Model Soundness on Personalizing an Intelligent Agent
Contribution & Benefit: A user study exploring the effects of mental model soundness on end users personalizing an intelligent agent. Can help designers understand the impact of providing structural information about intelligent agents.
Abstract » What does a user need to know to productively work with an intelligent agent? Intelligent agents and recommender systems are gaining widespread use, potentially creating a need for end users to understand how these systems operate in order to fix their agent's personalized behavior. This paper explores the effects of mental model soundness on such personalization by providing structural knowledge of a music recommender system in an empirical study. Our findings show that participants were able to quickly build sound mental models of the recommender system's reasoning, and that participants who most improved their mental models during the study were significantly more likely to make the recommender operate to their satisfaction. These results suggest that by helping end users understand a system's reasoning, intelligent agents may elicit more and better feedback, thus more closely aligning their output with each user's intentions.ACM
Pay Attention! Designing Adaptive Agents that Monitor and Improve User Engagement
Contribution & Benefit: Describes a novel technique to monitor and improve user attention in real-time using passive brain-computer interfaces and embodied agents. Will inform designers of adaptive interfaces, particularly for educational applications.
Abstract » Embodied agents hold great promise as educational assistants, exercise coaches, and team members in collaborative work. These roles require agents to closely monitor the behavioral, emotional, and mental states of their users and provide appropriate, effective responses. Educational agents, for example, will have to monitor student attention and seek to improve it when student engagement decreases. In this paper, we draw on techniques from brain-computer interfaces (BCI) and knowledge from educational psychology to design adaptive agents that monitor student attention in real time using measurements from electroencephalography (EEG) and recapture diminishing attention levels using verbal and nonverbal cues. An experimental evaluation of our approach showed that an adaptive robotic agent employing behavioral techniques to regain attention during drops in engagement improved student recall abilities 43% over the baseline regardless of student gender and significantly improved female motivation and rapport. Our findings offer guidelines for developing effective adaptive agents, particularly for educational settings.ACM
ReGroup: Interactive Machine Learning for On-Demand Group Creation in Social Networks
Contribution & Benefit: Presents ReGroup, a novel end-user interactive machine learning system for helping people create custom, on-demand groups in online social networks. Can facilitate in-context sharing, potentially encouraging better online privacy practices.
Abstract » We present ReGroup, a novel end-user interactive machine learning system for helping people create custom, on demand groups in online social networks. As a person adds members to a group, ReGroup iteratively learns a probabilistic model of group membership specific to that group. ReGroup then uses its currently learned model to suggest additional members and group characteristics for filtering. Our evaluation shows that ReGroup is effective for helping people create large and varied groups, whereas traditional methods (searching by name or selecting from an alphabetical list) are better suited for small groups whose members can be easily recalled by name. By facilitating on demand group creation, ReGroup can enable in-context sharing and potentially encourage better online privacy practices. In addition, applying interactive machine learning to social network group creation introduces several challenges for designing effective end-user interaction with machine learning. We identify these challenges and discuss how we address them in ReGroup.ACM
An Automatically Generated Interlanguage Tailored to Speakers of Minority but Culturally Influenced Languages
Contribution & Benefit: Describes a technique to compensate for resource-scarce languages in machine translation. Can assist in developing UIs tailored to speakers of minority languages.
Abstract » Automatic localization of cultural resources and UIs is crucial for the survival of minority languages, for which there are insufficient parallel corpora (or no corpus at all) to build machine translation systems. This paper proposes a new way to compensate for such resource-scarce languages, based on the fact that most languages share a common vocabulary. Concretely, our approach leverages a family of languages closely related to the speaker's native language to construct translations in a coherent mix of these languages. Experimental results indicate that these translations can be easily understood, being also a useful aid for users who are not proficient in foreign languages. Therefore this work significantly contributes to HCI in two ways: it establishes a language that can improve how applications communicate to their users, and it reports insights on the user acceptance towards the method.ACM
"Then Click 'OK!'" Extracting References to Interface Elements in Online Documentation
Contribution & Benefit: This paper presents a recognizer for identifying references to user interface components in online documentation. We enumerate various challenges, and discuss how informal conventions in tutorial writing can be leveraged.
Abstract » This paper presents a recognizer for identifying references to user interface components in online documentation. The recognizer first extracts phrases matching a list of known components, then employs a classifier to reject coincidental matches. We describe why this seemingly straightforward problem is challenging, then show how informal conventions in documentation writing can be leveraged to perform classification. Using the features identified in this paper, our approach achieves an average F1 score of 0.81, and can correctly distinguish between actual command references and coincidental matches in 93.7% of test cases.ACM