Deliverables

D1.1 Literature Review

Comprised of two major parts, namely an interdisciplinary review on “human sociality” aspects and a review on long-term relationships, the text is intended to provide insights into aspects of human-human communication and human-robot/agent communication that are relevant for the SERA project. In doing this, the first part is subdivided into micro-, meso- and macro-models and covers several prerequisites for human communication: verbal and nonverbal behaviour, the need to belong, common ground, perspective taking and theory of mind. The part on macro-levels features some sociological frameworks for interactions (e.g. affect control theory, identity control theory) and introduces systems theory. The second part of this literature review goes into detail about available systems (e.g. Valerie, Pleo, Paro, Nabaztag:tag) and covers a variety of experiments and results on social effects and longterm relationships. Furthermore, in a chapter on methodology, open and closed research questions and methods are explained in detail and examples related to the exploration of several aspects of robots/agents and human-robot/agent interaction are given. Finally, the document provides a summary with implications for the SERA project.

D1.2a SERA Theory Framework

Since social engagement is a complex phenomenon, the creation of long-term appealing robots/agents requires the integration of sound interdisciplinary theoretical foundations. Starting from knowledge from human-human relationships, we present an integrated theoretical framework of prerequisites for human-agent and human-robot-communication. Besides this, a first review of the data collected in iteration 1 helps to identify problems arising from the lack of theory of mind abilities of the system. In this respect we present alternatives for how the interaction could work if the artificial interaction partner had theory grounded abilities that go beyond the actual implementation. Additionally, we suggest to what extent theory can lead data analysis.

D1.2b Revised Journal Papers

A major outcome of the project is a special issue of a peer-reviewed scientific journal, with papers covering the major research and insights of the project. Six papers were submitted and underwent peer review. The deliverable contains revised versions of the papers, extracts from the reviews, and the authors' comments on the reviews. The papers will be published as issue (25)6 of the journal Applied Artificial Intelligence.

D1.3b Data Analysis Results

The deliverable contains a summary about the data analysis results of the three field trials as well as of additional user studies and draws general conclusions about human-robot interactions and relationship building between humans and robots.

D2.3 Corpus Description

This report gives an overview of the data collected in the course of the project. Data comprise video recordings, interviews (audio files and transcripts), and questionnaires. The report also contains the plan for the use of these data, which constitute foreground of the project.

D2.5 Speech Technology

This document accompanies the dialogue manager voice design for SERA field iterations 2 and 3. It summarises why that particular voice was chosen, and it gives background information on agent character decisions (in the form of the literature review provided in the annex). Part of the experiments that have been conducted (in collaboration with University of Twente) have aimed at establishing a better measure of consistency between voice and agent interface than currently available. Part of the deliverable has been submitted as a paper to Interspeech 2010.

D3.1 SERA Design Guidelines

Abstract (for dissemination) This deliverable considers two questions: what would social agents and robots have to look like and what skills should they have. We probe these aspects by looking at the recommendations and experience laid down in the literature and by conclusions drawn from the preliminary data analysis of the first iteration. Rather than presenting a full survey of the literature, we discuss what we believe to be a relevant sample that is sufficient for current purposes to identify the range of dimensions to consider in designing social agents or robots.

D3.2 SERA Reference Architecture

For Workpackage 3 of the SERA project two reports are planned as deliverables. The first report “Design Guidelines” has already been delivered. This report constitutes the second report on what has been called the “Reference Architecture”. The design guidelines and reference architecture documents are concerned with questions related to three aspects of social agents or robots: functionality, appearance and components. The first aspect, functionality, deals with the question what behaviours and actions social agents and robots should be capable of performing to properly be called “social”. Appearance deals with the question what they should look like. These two aspects were considered in deliverable D3.1. The current deliverable, D3.2, deals with the third aspect: “how should they be built”. As for the questions addressed in D3.1, we provide a first answer by looking at descriptions of robots and agents that have been designed and built for social interaction. The functionality that is shared by the systems – building conversational systems – leads to a common basis in the architecture: there are functions for input and output and for decision making. The emphasis on social aspects leads furthermore to consider components that go further into understanding the user than merely task-related aspects: Affect recognition, affect modelling, user modelling in general (through specialised modules) is considered as well. Two kinds of types of systems can be distinguished. The systems that try to model explicitly the cognitive systems that underlie human behaviour and the systems that take a more behavioural approach.

D3.3 Implementation of the SERA Core Architecture

The software that we present in this deliverable consists of a set of modules that allows one to specify the core part of a dialogue system. Besides the software modules, we also present a demo implementation of one of the essential skills that any interactive system needs to handle, namely turn-taking and topic management.

D4.1 Showcase Scenario Description

This document provides a first characterisation of the SERA showcase: its place and purpose within the SERA project, identified requirements and use cases. It includes a first draft of the kind of background story to be used to situate and contextualise the research findings to be presented in the showcase.

D4.2 Showcase and Video & D4.3 Showcase Test Report

This document provides an overview of the prototype delivered by WP4, the portable showcase. This showcase is anchored by a website that describes project results, installation and setup instructions for interactive demos, their hardware and software requirements, as well as the intended use cases. The text content of the initial distribution of this showcase is, in part, included in this deliverable, for the full text, we refer to the public website. Finally, we report on a questionnaire intended for future feedback and results of testing the system during the development phase.

D5.2a Proceedings of Dissemination Event

As one of the dissemination events of the project SERA, a symposium was held in the framework of EMCSR 2010 (Vienna, Austria) and is reported here, including the papers presented by project partners.

D6.3a Self-Assessment Checklist

The report covers the quality assurance procedure adopted for the project. The core process is a "270-degree evaluation" which includes definition of criteria as well as review of results by both self (WP leaders) and other (partners).