Lecture “Affective Computing”

Affective Computing is an interdisciplinary research area that uses models of affect to recognize emotional expressions, simulate affect, and create emotional behavior.

This course gives an overview of theories of human emotion and how emotions, social values, cognitive processes, and bodily expressions are connected. Also, hardware and computational techniques for modeling human emotion processes as well as for recognizing and synthesizing emotional behavior are presented and discussed.  Further, we will discuss how to extend traditional HCI systems to create Social HCI systems. Master students will gain a strong background in the theory and practice of human-centered computing as it relates to games, immersive environments, and pedagogical applications.

Instructor: Patrick Gebhard (patrick.gebhard@dfki.de)

Dates: April 10, 17, 24; Mai 8, 15, 22, 29; June 5, 12, 19, 26; July 3

Time: 10 am -12 am

Location: HS 003, E1 3

Grades: Exam

Current Objectives for Participants

Grades for the 1st exam can be obtained in LSF/HISPOS. 

Results for the 2nd exam can be obtained in LSF/HISPOS.

Exam inspection is handled individually. Please contact the instructor via email.

Registration for the Exam

The registration for internal use is closed. Registered students will be informed about the exam (e.g., exact date and place) and about updates via email. Please note that you have to officially register in HIPOS/LSF two weeks before the exam.

Course Outline

April 10Introduction and Overview

  • What is Affect?
  • History and Current State
  • Typical Areas
  • Examples
  • Course Organisation

Introduction and Overview – Slides (PDF)

April 17Basic Models of Affect

  • Identify Affective Phenomena
  • Need for Theory
  • Basic Model of Affect

Theory of Emotion I – Basic Models of Affect – Slides (PDF)

Video for Lecture Experiment

Support Media (Interview Prof. Lisa Feldman Barrett about Emotions)

Preparation Reading: Loewenstein, George, and Jennifer S. Lerner. The role of affect in decision making.

April 24Emotion as Information

  • Appraisal Theory
  • Influence on Cognitive Processes
  • Rational vs. Emotional Decisions

Theory of Emotion II – Emotion as Information – Slides (PDF)

The Ultimatum Game (Youtube)

Preparation Reading: Whiten, Andrew, editor, Natural Theories of Mind, Evolution, Development and Simulation of Everyday Mindreading

chapter 1chapter 2chapter 3

May 8: Theory of Mind, Advanced Definition of Emotions

  • Theory of Mind
  • Advanced Definition of Emotion
  • Regulation of Emotions

Theory of Emotion III – Theory of Mind, Advanced Definition of Emotion – Slides (PDF)

Theory of Mind of a 3 year old child (Youtube)

Theory of Mind of a 5 year old child (Youtube)

Preparation Reading: McLeod, Psychology as a Science, 2008

May 15Experimental Design, Methodology, and Analysis

(Guest lecturer: Tanja Schneeberger)

  • Basic Understanding of Empirical Methods
  • Design, and Conduct Studies
  • Dependent, Independent Variables

Methods and Techniques – Experimental Design, Methodology and Analysis – Slides (PDF)

Preparation Reading: The Oxford Handbook of Affective Computing,
Chapter 7 – „Social Signal Processing“

May 22Experimental Design, Methodology and Analysis (Step 4: From Experiments to Results) and Emotional Social Signals

(Guest lecturer for the first part: Tanja Schneeberger)

  • From Experiment to Results
  • Overview of Social Signals Processing
  • Challenges, Models, and Tools
  • Recognition Approaches

Methods and Techniques – Experimental Design, Methodology and Analysis (Step 4: From Experiment to Results) and Emotional Social Signals – Slides (PDF)

Support Reading: Gebhard et al., Exploring Interaction Strategies for Virtual Characters to Induce Stress in Simulated Job Interviews, 2014

Preparation Reading: Dastani et al., Programming Agents with Emotions, 2014

May 29Computational Model of Affect

  • Overview on Models
  • Component View on Affect Simulation
  • Script-Based and Plan-based Appraisal
  • Creating Believable Behavior

Computational Model of Affect – Slides (PDF)

ALMA web page,Basic Appraisal Tags, OCC variables, elicited Emotions

Videos: ALMA OCEAN-PAD MappingALMA Mood Change (Pull) and Amplification (Push)

Support Reading: Vinayagamoorthy et al., Building Expression into Virtual Characters, 2006

Preparation Reading: The Oxford Handbook of Affective Computing,
Chapter 34 – „Emotion in Games“, Chapter 36 “Affect in Human-Robot Interaction”

June 5Systems of Affective Computing

  • Workflow
  • Software Components
  • Modelling Approaches
  • Tools and Examples
  • Design of AdapTris – Part 1

Systems of Affective Computing – Slides (PDF)

VideosSimple Head Tilt BehaviormodelComplex HeadbehaviorIDEAS4Games Overview

Support Reading: Gebhard, Kipp, Mehlmann, Visual SceneMaker -A Tool for Authoring Interactive Virtual Characters, 2012

Preparation Reading: Nunnari, Heloir, Generation of Virtual Characters from Personality Traits, 2017

June 12: Authoring Virtual Characters for Social Interactions

(Guest Lecturer: Fabrizio Nunnari)

  • Body Modeling
  • Animation of Characters
  • Concepts and Tools

Authoring Virtual Characters for Social Interactions – Slides (PDF)

Support Reading: André et al, Exploiting Model of Personality to Control the Behavior of Animated Interactive Agents, 2000

Preparation Reading: Gebhard et al., MARSSI: Model of Appraisal, Regulation, and Social Signal Interpretation, 2018

June 19Computationally Model Human Emotions

  • Design of AdapTris – Part 2
  • Advanced Computational Models
  • Coping, and Regulation
  • Connection between Model and Social Signals

Computationally Model Human Emotions – Slides (PDF)

Support Reading: Christina Conati, Heather Maclaren, Emirically building and evaluating a probabilistic model of user affect, 2009

Preparation Reading: David DeVault et al., SimSensei Kiosk: A Virtual Human Interviewer for Healthcare Decision Support, 2014

June 26Systems of Social Computing

  • Design of AdapTris – Part 3
  • Definition Social HCI
  • Social User Modeling
  • System Examples and Discussion

Systems of Social Computing – Slides (PDF)

Cafaro, Glass, 16, Interruption of 2 Agents (Voice) from Keynote of C. Pelachaud (Video, min 32)

Real-time Barge-In Virtual Character (Vide0)

Real-time Handling-Barge-In (VSM) (Video)

Real-time Conflict Detection (VSM) (Video)

Real-time Overlap Detection (VSM) (Video)

Support Reading: Bart et al., The Influence of a Social Robot’s Persona on How it is Perceived and Accepted by Elderly Users, 2016

Preparation Reading: all lectures

July 3Recapitulation

  • Important Concepts 
  • Functions of Emotions
  • Areas of Affective Computing
  • Typical Methods and Techniques

Recapitulation with Student Survey – Slides (PDF)

July 17Exam