CSIM is composed of a wide set of courses that capture the interdisciplinary nature and vocation of this master program. The table below summarises the structure of the syllabus.

The core of the program is formed by courses that have been especially conceived for the CSIM master and some that are shared from other master programs in our department.  These courses are composed into the following three groups:

  • Compulsory Courses: These courses are meant to give a solid base and a transversal view onto Cognitive Systems and Interactive Media.
  • Main Optional Pool Courses: These courses are meant to provide a more in depth view on specific branches of Cognitive Systems and Interactive Media.
  • Optional Expansion Pool Courses: These courses are meant to provide an expansion into more applicative branches of Cognitive Systems and Interactive Media.
    NOTE: These courses are technically and mathematically quite demanding and are therefore only feasible for students that already have a strong background in mathematics and engineering.
 

 

Trimester  Course Code Belongs to: ECTS
Compulsory Courses
1 Cognitive Science & Psychology: Mind, Brain and Behaviour 30846 CSIM 5
1 Research Methodologies in Humanities and Science 30845 CSIM 5
1 Interaction Models 30847 CSIM 5
2 Systems Design, Integration and Control 30863 CSIM 5
Main Optional Pool
(Choose minimum 3)
2 Adaptive Behaviour 30864 CSIM 5
3 Advanced Concepts and Methods in Cognitive Systems 30877 CSIM 5
2 Advanced Interface Design 30853 CSIM 5
2 Cognitive Systems: theory and models 30860 CSIM 5
3 Education, Games and Entertainment 30857 CSIM 5
1 Real Time Interaction 30876 SMC 5
3 Sound Communication 30852 CSIM 5
Optional Expansion Pool
(Choose maximum 1)
1 Audio and Music Processing 30226 SMC 5
1 Autonomous Systems 31641 MIIS 5
1 Machine Learning 31645 MIIS 5
1 Mobile Robotics 31643 MIIS 5
2 Natural Language Interaction 31642 MIIS 5
Master Thesis
1, 2 & 3 Project 30849 CSIM 20

 

 

All courses are worth 5 ECTS (European credits), while the Project is worth 20 ECTS.
To attain the masters diploma the student must get a total of 60 ECTS.
Therefore, students must choose:

  • All the Base Courses (20ECTS) 
  • A minimum of 3 courses from the Main Optional Pool Courses (mínimum 15ECTS)
  • A maximum of one course Optional Expansion Pool Courses (maximum 5ECTS).
  • The Master Thesis Project (20ECTS)

This adds up to either:     4x5 + (4+0)x5 + 20 = 60 ECTS      or     4x5 + (3+1)x5 + 20 = 60 ECTS

 

Below you may find the description of each course:

Cognitive Science & Psychology: Mind, Brain and Behaviour - 30846 Cognitive Science & Psychology: Mind, Brain and Behaviour - 30846

Professor: Paul Verschure


Description

This course exposes students to the central disciplines that form traditional cognitive science (philosophy, psychology, linguistics, computer science, mathematics, anthropology) and will show how the concepts and paradigms of these disciplines bring complementary visions of mind, brain and behaviour.

Course Objectives

Learn about theories, methods and discoveries in cognitive science, the historical context and the philosophical roots that allowed the rising of this multidisciplinary field of studies.
To help students develop general scientific thinking and study skills that will be an important requirement for all the master courses.
To help students understand cognitive science application to real world artifacts.
To help students to develop a critical approach to scientific research and literature.

Readings

There is no official text book for this course. Every week two selected readings related to the topic of the lecture will be assigned and made available in the Moodle page. You are expected to read and critically comment the papers in the form of a short essay to submit through the Moodle system before each lecture.

Class attendance

Regular attendance to the classes is mandatory since this is the only way to learn the material.

Research project

As part of your training you will have to design, conduct and present a real experimental investigation related to one of the topics covered in class. Projects will be performed in small groups (max. 4 people) and they are shared with the Research Methodologies in Humanities and Science (30845) course.

Evaluation Criteria

  1. Homework (weekly essay on selected readings):  50%
  2. Research project: project proposal (30%) + project presentation and report (70%): 50%

Delays in delivering the homework assignments without a valid motivation will be penalised and the maximum mark that can then be awarded is the minimum pass mark. Plagiarism will not be tolerated in any of its forms.

Course Structure

Week 1

Introduction to the course

Readings:

  • Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognising one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-34.

Week 2

Philosophical and Historical roots (part I)

Main topics:

  • The knowledge problem in science
  • Science wars
  • Mind vs matter
  • Evolutionary aspects of brain
  • The bicameral mind hypothesis of Jaynes
  • The mind-body problem
  • Rationalism and empiricism
  • Plato's cave allegory

Readings:

  • Plato, The Republic, Book VII, translated by B. Jowett

Week 3

Philosophical and Historical roots (part II)

Main topics:

  • More on the mind-body problem: Socrates, Plato, Aristoteles
  • Galen and the localisation of brain functions
  • Fernel and the brain doctrine
  • Cartesian dualism
  • Physicalism
  • Hume's empiricism and the problem of causality
  • Evolutionism

Readings:

  • Ghost in the shell (movie), Masamune Shirow, Kazunori Itō, 1995.

Week 4

Structuralism and early behaviourism

Main topics:

  • The experimental study of psychology
  • Fechner, Von Helmoltz, and the origins of psychophysics
  • The stage model of Donders
  • Wundt and the rise of structuralism
  • Pavlov: reflexes and the origin of classical conditioning
  • Thorndike and the laws of goal oriented learning
  • Classical vs operant/instrumental conditioning
  • The birth of behaviourism

Readings:

  • Köhler, W. (1959). Gestalt Psychology Today. American Psychologist, 727-734.
  • Watson, J. (1913). Psychology as the Behaviourist Views it. Psychological Review, 1-11.

Week 5

Behaviorism

Main topics:

  • Watson's little Albert experiment
  • Skinner and operant conditioning

Week 6

Cognitive behaviorism

Main topics:

  • Tolman and cognitive maps
  • Hull and adaptive behaviour
  • More on classical conditioning
  • Rescorla Wagner and the behavioural law of associative competition

Readings:

  • Breland, K., & Breland, M. (1961). The misbehavior of organisms. American Psychologist.
  • Lettvin, J. Y., Maturana, H. R., McCulloch, W. S., & Pitts, W. H. (1959). What the frog's eye tells the frog's brain. Proceedings of the IRE, 47(11), 1940-1951.

Week 7

The fall of behaviorism and the cognitive revolution

Main topics:

  • The cognitive revolution

Readings:

  • Chomsky, N. (1959). A review of Skinner's verbal behavior. Language, 142-143.

Week 8

Mind as computation (part I)

Main topics:

  • Wiener and the origins of Cybernetics
  • Feedback and self regulating systems (Cannon, Ashby, Walters)
  • From analytic engine to computers via the Turing machine
  • The computer metaphor

Readings:

  • Searle, J. (1980). Minds, brains, and programs. Behavioral and Brain Sciences.
  • Turing, A. (1950). Computing machinery and intelligence. Mind, 1-21.

Week 9

Mind as computation (part II)

Main topics:

  • Theory of mind: functionalism
  • Searle's Chinese room argument

Week 10

Flux and synthesis

Main topics:

  • New AI
  • Embodiment and morphological computation
  • Connectionism
  • Synthesis: how does the brain solve the mind-brain problem?

Projects' presentation

TOP

Research Methodologies in Humanities and Science - 30845 Research Methodologies in Humanities and Science - 30845

Professor: Alberto Betella & Rosa Cerarols

 

Description

This course exposes students to the essential research methods in humanities and science including quantitative and qualitative research methods and statistics

Course Objectives

  1. Know the different research methodologies used in the ambits of humanities and science, understand their characteristics, differences and utilities.
  2. Be able to apply these methodologies within the student's own research.

Readings

Hugh Coolican. Research Methods and Statistics in Psychology. 
Hodder Arnold publisher, 4Rev Ed edition
, ISBN-10: 0340812583

Class attendance

Regular attendance to the classes is mandatory

Evaluation Criteria

Grades will be based on

  • Regular class presence
  • Assignments and Tutorial Exercises: 50%
  • Project proposal (30%) and presentation (70%): 50%

Course Structure

Week 1

  • Research in Humanities and Science
  • Epistemiological background
  • Experimental methods

Week 2

  • True experiment and Quasi Experiment
  • Barriers to Scientific Method
  • Questionnaires, Attitude Scales, Test

Week 3

  • Ethics and Subjects
  • Data management

Week 4

  • Project presentation

Week 5

  • Descriptive Statistics

Week 6

  • Inferential Statistics

Week 7

  • Testing for differences: ANOVA & CO

Week 8

  • Correlation and Regression
  • Nonparametic Statistics
  • How to write a paper

Week 9

  • Qualitative methods in research

Week 10

  • Qualitative methods in research

Research project

As part of your training you will have to design, conduct and present a real experimental investigation related to one of the topics covered in class. Projects will be shared with the Cognitive Science and Psychology (30846) course and will be performed is small groups.

TOP

System design integration and control - 30863 System design integration and control - 30863

Professor: Armin Duff

 

Description

This course exposes students to paradigms habitual within design, integration and control of truly feasible complex systems, with a special stress on neuromorphic principles underlying biological, interactive, cognitive and emotive systems.

Course Objectives

  1. Understand the basic principles of organization of perception and control in biological systems and its neurological substrate.
  2. Learn the design principles necessary to build and analyze a biological grounded synthetic system.
  3. Master the tools to implement and control a biomimetic system, including neural network simulator IQR.

Readings

Readings related to the topics of the lecture will be made available through the course webpage.

Class attendance

Regular attendance to the classes is mandatory

Evaluation Criteria

Grades will be based on

  • Regular class presence
  • IQR tutorials (40%)
  • Final project competition and report (60%)

Course Structure

Week 1

  • Introduction to Biomimetics

Week 2

  • Neurons the basic elements of the Brain
  • IQR Introduction: Basics and Neurons

Week 3

  • Synapses and Brain Circuits
  • IQR Introduction: Connections

Week 4

  • C++ primer
  • IQR: Developer

Week 5

  • System Design Concepts and Tools
  • IQR: Advanced Features

Week 6

  • Introducing: SDIC Robot Challenge

Week 7

  • The Human Visual System
  • IQR: Perception and Denoising

Week 8

  • Motor Control
  • IQR: Simple Control Mechanisms

Week 9

  • Learning and Memory
  • IQR: Hebbian Learning

Week 10

  • SDIC Robot Challenge

Research project

As part of your training you will have to design, and implement a swarm of foraging agents controlled in IQR following biomimetic design principles. In the final lecture you will compete for limited resources in a simulated environment against your peers in small groups.

TOP

Cognitive Systems: theory and models - 30860 Cognitive Systems: theory and models - 30860

Professor: Marti Sanchez Fibla & Riccardo Zucca

 

Description

This course presents a panorama of theories and models on cognition, emotion and personality in combination with methods for evaluation and testing.

Course Objectives

  1. Theoretical and practical knowledge on artificial neural networks
  2. Theoretical and practical knowledge on reward based learning
  3. Theoretical and practical knowledge on control architectures in particular the Distributed Adaptive Control (DAC) Architecture

Readings

  • David Kriesel: A Brief Introduction to Neural Networks (online_ http://www.dkriesel.com/_media/science/neuronalenetze-en-zeta2-2col-dkrieselcom.pdf)
  • Richard S. Sutton and Andrew G. Barto: Reinforcement Learning:An Introduction (online_ http://webdocs.cs.ualberta.ca/~sutton/book/the-book.html?)
  • Paul F.M.J. Verschure: Distributed Adaptive Control: A theory of the Mind, Brain, Body Nexus (online_ http://www.sciencedirect.com/science/article/pii/S2212683X12000102)

Class attendance

Regular attendance to the classes is mandatory

Evaluation Criteria

Grades will be based on

  • Regular class presence
  • Performance on the assignments
  • Research project: paper presentation

Course Structure

Week 1

  • Introduction to Artificial Neural Networks

Week 2

  • Advanced Concepts of Artificial Neural Networks

Week 3

  • Application in Artificial Neural Networks

Week 4

  • The physiology of Reward

Week 5

  • Introduction to Reinforcement Learning

Week 6

  • Applications of Reinforcement Learning

Week 7

  • Introduction to Control Architectures

Week 8

  • Distributed Adaptive Control (DAC): Reactive Layer

Week 9

  • Distributed Adaptive Control (DAC): Adaptive Layer

Week 10

  • Distributed Adaptive Control (DAC): Contextual Layer

TOP

Advanced concepts and methods in cognitive science - 30877 Advanced concepts and methods in cognitive science - 30877

Professor: Riccardo Zucca

 

Brief Description

This course will provide an interdisciplinary overview on theories of consciousness from the perspective of modern science through the critical discussion of selected readings from a broad range of fields (philosophy of mind, cognitive psychology, biology, artificial intelligence among others)

Description

One of the major challenges that scientific research is now facing is understanding the nature of consciousness. In this module we will consider this problem from several perspectives and analyse the various solutions proposed within different theoretical frameworks. 
We will not try to address the question merely from a philosophical point of view but we’ll try to gain a better understanding of consciousness form a perspective of modern science. 
We will initially face the epistemological issues related to the study of consciousness (the hard/easy problem) and what is the function of consciousness. We will then learn how these questions have been addressed by evolutionary, experimental, clinical, computational and synthetic studies and what are the solutions they provided. Finally, we’ll provide an integrative approach about consciousness in the context of the Distributed Adaptive Control theory of mind, brain and body.
Every week two articles will be selected for open discussion and critically evaluated in a typical journal club format.

Readings

There is no official text book for this course. Every week two selected readings related to the topic of the lecture will be assigned and made available in the Moodle page. Students will have to read the literature provided and submit a critical review of the article in the form of a short essay before each class through the Moodle system.

Class attendance

Regular attendance to the classes is mandatory given the open discussion format of the lectures.
Every week one of the students will prepare a keynote presentation of a selected reading that will open the discussion.

Evaluation Criteria

Grades will be based on

  • Regular class presence
  • Homeworks: 60%
  • Class presentation: 40%

Delays in delivering the homework assignments without a valid motivation will be penalised and the maximum mark that can then be awarded is the minimum pass mark.
Plagiarism will not be tolerated in any of its forms

Course Structure

Week 1

  • What are the epistemological issues in the study of consciousness? The “hard problem” of consciousness and the reductionist approach

Readings:

  • Nagel, T. (1974). What is it like to be a bat? The Philosophical Review, 83(4), 435–450.
  • Lem, S. (1974). The Seventh Sally, or How Trurl’s Own Perfection Led to No Good. The Cyberiad.

Week 2

  • What is the function of consciousness? Surprising dissociations between behavior and conscious experience. Confabulations and timing.

Readings:

  • Libet, B. (1993). Unconscious cerebral initiative and the role of conscious will in voluntary action. Neurophysiology of Consciousness, 529–566.
  • Sperry, Roger W., Michael S. Gazzaniga, and Joseph E. Bogen (1969). "Interhemispheric relationships: the neocortical commissures; syndromes of hemisphere disconnection." Handbook of clinical neurology 4.273-290.

Week 3

  • What is the CDAC answer to the function of consciousness? H4W/H5W. Evolutionary pressures and the Cambrian revolution.

Readings:

  • Feinberg, T. E., & Mallatt, J. (2013). The evolutionary and genetic origins of consciousness in the Cambrian Period over 500 million years ago. Frontiers in Psychology, 4(October), 667.
  • Parker, A. (2003). In the blink of an eye. Biological Psychiatry. Cambrisge (MA): Perseus Publishing.

Week 4

  • More into the H5W problem: Intentionality and social context. Step from H4W to H5W.

Readings:

  • Dennett, D. C. (1981). True believers: The intentional strategy and why it works. In Mind Design II (pp. 57–79)
  • Graziano, M., & Kastner, S. (2011). Human consciousness and its relationship to social neuroscience: a novel hypothesis. Cognitive Neuroscience, 2(2), 98–113.

Week 5

  • Returning to H4W: Sensory motor contingencies, mind/body/environment nexus.

Readings:

  • O'Regan, J. Kevin, and Alva Noë. "A sensorimotor account of vision and visual consciousness." Behavioral and brain sciences 24.05 (2001): 939-973.

Week 6

  • Predicting brains: Predicting brains and simulation theory of cognition

Readings:

  • Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. The Behavioral and Brain Sciences, 36(3), 181–204.
  • Hesslow, G. (2002). Conscious thought as simulation of behaviour and perception. Trends in Cognitive Sciences, 6(6), 242–247.

Week 7

  • Experimental studies of consciousness: Control of mental activities by internal models. Conscious and unconscious information processing.

Readings:

  • Ito, M. (2008). Control of mental activities by internal models in the cerebellum. Nature Reviews. Neuroscience, 9(4), 304–13.
  • Dehaene, S., & Naccache, L. (2001). Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition, 79(1-2), 1–37.

Week 8

  • Psychiatric studies of consciousness: Agnosias, drug effects, subjectivity of experience, confabulation and delusion

Readings:

  • Blackwood, Nigel J., et al. "Cognitive neuropsychiatric models of persecutory delusions." American Journal of Psychiatry 158.4 (2001): 527-539.
  • Schnider, Armin. "Spontaneous confabulation and the adaptation of thought to ongoing reality." Nature Reviews Neuroscience 4.8 (2003): 662-671.

Week 9

  • Computational studies of consciousness: Connectomics, modeling psychophysics.

Readings:

  • Balduzzi, D., & Tononi, G. (2008). Integrated information in discrete dynamical systems: motivation and theoretical framework. PLoS Computational Biology, 4(6), e1000091.
  • Barrett, A. B., & Seth, A. K. (2011). Practical measures of integrated information for time-series data. PLoS Computational Biology, 7(1), e1001052.

Week 10

  • Synthetic studies of consciousness: Why, how and when do we apply C-DAC to machines

Readings:

  • Verschure, Paul FMJ. "Distributed adaptive control: a theory of the mind, brain, body nexus." Biologically Inspired Cognitive Architectures 1 (2012): 55-72.

TOP

Advanced interface design - 30853 Advanced interface design - 30853

Professor: Martí Sánchez Fibla

 

Description

This course focuses on paradigms, methods and tools used in the construction of complex multimodal interfaces between users and artefacts

Course objectives

Students will learn to build and use interfaces and artefacts that can engage subjects during perceptual/behavioural task and will be given the tools to be able to capture and measure different characteristics of the performed tasks and deliver feedback.
Covering the different phases of this closed loop experience we will learn:

  • the basics of sensing technologies and we will learn how to use existing devices and how to build our own
  • how transfer data through communication protocols and how to process it with examples of measures that we can extract
  • how to deliver feedback through actuation using motors, displays, leds

Robots are great tools to learn and practice sensor actuator loops. We will learn basic principles of robotic systems using a mobile Arduino based robotic platform that we developed at SPECS group.

Readings and materials

In the course we provide tutorials for each weekly session as well as source example code.
We will also provide hardware material to use.
The main hardware tools that will be covered in the course are Arduino, RaspberryPi, Kinect, Console Devices like the Wiimote, Eye tracking, Physiological signals.
Processing and Python will be the main programming languages used during the course.
The course is project oriented: from an early stage a project will be structured and developed in accordance with the teacher

Class attendance

Regular attendance to the classes is mandatory.

Evaluation Criteria

Evaluation of the progress of the students is carried on during the different phases of the project development.

  • Initial project draft: 15 %
  • Work in class, group: 15%
  • Final presentation, group: 40%
  • 4-page final report, one per group: 30%

Individual contributions have to be indicated explicitly.

Course Structure

Week 1 and Week 2

Introduction and Arduino / Processing basics

We will give a broad overview of the state of the art in interface technologies. Arduino electronics and programming will be introduced.
The Processing programing is also used for rapid prototyping of applications interfacing with sensors and actuators which may need basic visualization capabilities. The advantage of using processing is that programs can be very easily transferred to Android mobile phones.

Week 3 and Week 4

Raspberry Pi, Communication protocols

Project guidelines will be presented and students will need to start focusing on which direction they want to take. During week 4 projects start to be developed.
Raspberry Pi is a linux based mini computer with the capability of interfacing with sensors and actuators and Arduino itself. We will introduce it in the course so that we realize the similarities and differences with Arduino. 
Communication protocols are the basic tools to plug together sensors of different nature or systems.For this purpose we introduce TUIO and OSC.
An example is developed through the class which consists of a video sequencer and controller that can be interfaced with Arduino sensor data.

Week 5

Robotic and control applications

We will learn the basics of robotic systems through different examples.
We will use an Arduino/RaspberryPi robotic platform developed in SPECS.
We will also explain a case study of an Arduino self-balancing robot that can be controlled via bluetooth through processing. 
Projects will be developing through the rest of the weeks with feedback and support.

Week 6

Computer Vision and RGB-D Applications

We will deal in this class with a variety of camera sensing technologies including use of normal cameras and Kinect devices.

Week 7

Interfacing with physiology signals: Arduino E-Health Sensor board

This is a class dedicated to learn how to interface with physiology signals and how to compute arousal and valence measures from them. The class covers a diverse variety of physiological signals : from complex EEG recordings to heart rate and galvanic skin response.

Week 8

Audio Processing

We deal with several audio processing applications like beat detection, frequency based analysis and others. We will learn how to control interfaces with different audio extracted parameters.
Projects start reaching

Week 9

Acquiring and sharing data through internet and the internet of things

Examples of how to share sensor data through internet applications are provided. Being the last class before the presentations the main part of the class will be devoted to finalizing projects.

Week 10

Project Presentations

Project presentations include a demo of the built device. Examples of previous years projects:

  • A sonar sensor based sensitive stick with haptic feedback for blind navigation.
  • The vibrating belt : an 8 motor vibrating belt based on Arduino
  • A learning gestural frequency selector and filtering system
  • An arduino sensing skate used for monitoring skateboard tricks
  • A fitness system using multimodal interfaces

TOP

Interaction models - 30847 Interaction models - 30847

Professor: Sergi Jordà & Roc Parés & Narcis Parés

 

Brief summary

This course will present a range of different views on interaction that are represented by structures and properties, design cycles and strategies, mediation and communication flows and modes, interface configurations, user approaches, etc., that constitute, either explicitly or implicitly, models for each type of interaction.
These interaction models show how different interaction can be across the range, so as to actually represent different media with distinct specificities and application adequacy.

Description

In all fields we tend to talk of "Interaction" as if it were one single concept or technological option. However, there are many configurations of human-computer interaction technology that lead to very different relationships between the users and the "system". This implicit uniformisation leads, on the one hand, to confusion of terms used in interaction design, interface design, evaluation, etc. On the other hand, it ignores the huge differences in potential that these different configurations provide. 
In this course we will first introduce this problem and start looking at some differences between technology configurations. We will start to move from a purely technological view to a communicational view to try to understand the specificities of the different configurations as interactive "media". 
The course will then analyse properties and evaluation criteria in desktop and web-based interaction. 
A second block will introduce the specificities of Tangible Interaction. 
A third block will explore Virtual Reality and the notion of Presence. This will lead on to the global field of AMVR (augmented, mixed and virtual reality) and will analyse their common specificities. 
Finally the course will close with an comparative analysis of all the exposed interactive media to understand where they stand with respect to each other.

Class attendance

Regular attendance to the classes is mandatory

Evaluation Criteria

Grading will be done through participation in class, assignments in the form of small analytical work as short texts or class presentations. 
Each teacher in the course will define its own type of assignments (please see the weekly program) and the specific evaluation s/he will do over those assignments. 
The final grade will be calculated as per the following formula:

  • Sergi Jordà: 10%
  • Roc Parés: 10%
  • Narcis Parés: 80%

Course Structure

Week 1

Interaction Models Introduction (Narcis Parés)

We will introduce the topic and the course structure. 
Our goal will be to understand what is specific of the different types of interaction, how they can be modelled and how are interactive experiences mediated. 
We will then understand why models might be important in interaction study and design by unfolding the different aspects of the different types of interaction. 
We will focus on two main types of interactions, namely AMVR & Tangible. 
We will then enter the realm of Virtual Reality and will review it's technological aspects.

Recommended readings:

Week 2

Interaction Models VR (Narcis Parés)

We will review a panorama of VR Applications to understand what VR has been used for. 
We will then analyze the definition of Interface.

Recommended readings:

Week 3

Tangible Interaction I (Sergi Jordà)

Suggested readings and videos: 
In the next two sessions we will focus on "tangible interaction". For preparing the first session, I would suggest that before the class you watch the following video playlist (constituted of 12 videos). The first and longer one (32'30") is a documentary made during the "7th International Conference on Tangible, Embedded and Embodied Interaction", which took place last February 2013 in Barcelona, chaired by Narcís and myself. The following videos (3 to 12) are much shorter and include interviews done during the same conference, to some of the most relevant researchers in the field. These interviews were taken by the organizers of the Discussion Panel "The beauty of the paradigms encounter" (which is slightly documented in the longer documentary) for detecting some of the more relevant topics for discussion. With a little more than 60' for all, I do really believe that these playlist constitutes an excellent and condensed introduction to the field.

Complementary readings:

  • Fernaeus 08
  • Jorda 08

Week 4

Tangible Interaction II (Sergi Jordà)

In this class we will discuss potential TUI applications.

Week 5

Virtual Reality: Definitions and Afine Technologies (Narcis Parés)

  • We will analyze a number of definitions of VR to apply critical thinking to this topic.
  • We will then analyze afine technologies such as Augmented, Mixed and Artificial Reality.
  • We will finally apply critical thinking to a very important aspect of VR: the notion of Presence/li>

Required readings:

Week 6

VR as a Medium (Narcis Parés)

  • We will analyze the specificities of Virtual Reality as seen from a Media Studies standpoint.
  • We will understand the importance of the notion of "Real-Time Generation" of stimuli.
  • We will also see and compare two main interaction design strategies.

Required readings:

  • Parés, N., Parés R., "Interaction-driven virtual reality application design. A particular case: 'El Ball del Fanalet or Lightpools'." PRESENCE: Teleoperators and Virtual Environments. Cambridge, MA: MIT Press, Vol 10.2. Pag. 236-245, 2001

Week 7

Virtual Subjectiveness (Narcis Parés)

We will analyse the paper referenced below to try to understand the steps taken towards a model that explains mediation of a VR experience.

Week 8

Critical models (I) (Roc Parés)

In these two sessions we will discuss how new media art is questioning the ways in which the conventional uses of interactive technologies determine our lives as individuals in our contemporary societies. A critical analysis model will be presented and used in several short class assignments.

Required readings:

  • Specific objects, essay originally published in Arts Yearbook 8, 1965 by Donald Judd.
  • Realtime art manifesto, Auriea Harvey & Michaël Samyn, Directors,  Tale of Tales, 2006

For more than 20 years I have been developing a personal, poetic and sometimes humorous body of work through the critical use of interactive media with a strong emphasis on the experimentation with tangible, embedded and embodied interfaces.

"Tangible, Embedded and Embodied have been attributes of art since the origin of mankind.  My work as an art educator, as an art researcher and as an art practitioner is to incorporate these ancient art attributes into the presently primitive human-computer interaction systems of our time. In this beginning of the XXI Century, art is questioning many of the assumptions which underlie the early desktop, laptop and even handheld interaction models. Such models have become conventional standards by imposing cartoonish metaphors which mimic the human relations with the objects in the world: Icons of folders, helm-wheels or tool-boxes, paper-clip assistants and miniature trash bins have accompanied us humans in our first steps into the first computational environments and cyberspaces. Within these corporation-operated worlds, human interaction with such simulated objects has been used more as a means to retrieve digitized contents and less as a meaningful and emotionally fulfilling experience by itself. But the more that we assume that computers are taking over our entire live times (from our first preschool sensory awareness apps to our posthumous twitts) the more we long for that time to be filled with engaging interactions that can make us participants more human." (From the curator's presentation text of the Arts Track Exhibition at TEI 2013 http://www.tei-conf.org/13/artstrack)  

Week 9

Critical models (II) (Roc Parés)

In these two sessions we will discuss how new media art is questioning the ways in which the conventional uses of interactive technologies determine our lives as individuals in our contemporary societies. A critical analysis model will be presented and used in several short class assignments.

Required readings:

  • Specific objects, essay originally published in Arts Yearbook 8, 1965 by Donald Judd.
  • Realtime art manifesto, Auriea Harvey & Michaël Samyn, Directors,  Tale of Tales, 2006.

Week 10

Interaction Models V (Narcis Parés)

This class will be dedicated to presentations of the students' assignments.

TOP

Education, Games and Entertainment - 30857 Education, Games and Entertainment - 30857

Professor: Narcis Parés

 

Brief summary

In this course we will have a look at why and how Embodied Interaction, Play and Interactive Technology can have an impact on applications that seem so far apart as, for example, Lego Mindstorms, state of the art interactive themed attractions such as Disney's Virtual Pirates of the Caribbean, Exergames such as the Interactive Slide, or experiences to help children with Autism such as ECHOES or Pico's Adventures. In the course we will also design a Full-body Interaction Learning Environment and an Interactive Themed Attraction. And we will take a look at the theories that inform the design and evaluation of these applications.

Description

This course focusses on three main application areas: Learning, Public Space Entertainment and Special Needs. These three areas are analysed from three main transversal research fields: Play, Embodiment and Technology. This means that we will analyse how these three latter research fields inform the three former application areas. We will focus on Interaction Design for these application areas and will especially try to understand how Embodied Interaction can provide very interesting benefits for them.

The course will therefore concentrate on how Embodied Interaction informs and helps in:

  • Technology Enhanced Learning such as Manipulatives and Full-Body Interaction Learning Environments (FUBILEs)
  • Large Scale Interaction for Location-Based Entertainment (LBE) and Theme Parks
  • Special Needs in Health, Motor and Cognitive Disorders such as through Exergames and Virtual Rehabilitation Systems

We will cover theoretical aspects, technological solutions, as well as interaction design strategies that can provide meaning generation, rich interaction and entertaining experiences to help users learn, have fun or have a more autonomous life.

Class attendance

Regular attendance to the classes is mandatory

Evaluation Criteria

This course will be evaluated using a hands-on approach through the work done in two main large design exercises:

  • Design of a Full-body Interaction Learning Environment: 50%
  • Design of a Large-scale Interactive Experience: 50%

These two works will be done in teams of three or four students to foster brainstorming, collaborative work, critical thinking and creativity.

Course Structure (Approximate structure which may vary slightly)

Week 1

  • Presentation
  • Introduction to Play & Game
  • From Froebel & Montessori to Manipulatives

Recommended readings:

Week 2

  • Constructionism and its Origins
  • Designing For/With Children

Recommended readings:

  • Papert, S. (1980) Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, Inc., New York, NY, USA.
  • Papert, S. (1987) Microworlds: transforming education, In Artificial intelligence and education, Vol. 1, pp. 79–94
  • Resnick, M. 2002. Rethinking Learning in the Digital Age. In The Global Information Technology Report: Readiness for the Networked World, edited by G. Kirkman. Oxford University Press
  • Ackermann, E. (2004) Constructing Knowledge and Transforming the World. In M. Tokoro & L. Steels (Eds.), A learning zone of one’s own: Sharing representations and flow in collaborative learning environments (pp. 15–37). IOS Press
  • Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds.). Cambridge,MS: Harvard University Press
  • Scaife, M., & Rogers, Y. (1998). Kids as informants: Telling us what we didn’t know or confirming what we knew already? In A. Druin (Ed.), The design of children’s technology (pp. 27–50). San Francisco: Morgan Kaufmann.
  • Scaife, M., Rogers, Y., Aldrich, F., & Davies, M. (1997). Designing for or designing with? Informant design for interactive learning environments. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’97 (pp. 343–350). New York, New York, USA: ACM Press. doi:10.1145/258549.258789
  • Druin, A., (1999) Cooperative inquiry: developing new technologies for children with children, In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '99), ACM, New York, NY, USA, pp. 592-599
  • Bekker, M.M. and Antle, A. (2011) Developmentally Situated Design (DSD): Making Theoretical Knowledge Accessible to Designers of Children’s Technology, CHI 2011 conference, Vancouver, Canada.
  • Scratch: http://scratch.mit.edu/

Week 3

  • Space & Body
  • Interaction and Embodiment
  • Full-body Interaction Learning Environments
  • Assignment of Design of FUBILE

Recommended readings:

  • Grudin, J. (1990). The computer reaches out: the historical continuity of interface design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vol. Seattle, W, pp. 261–268). New York, NY, USA: ACM
  • Merleau-Ponty, M. (2005) Phenomenology of Perception, Trans: Colin Smith, Routledge, London
  • Cerbone, D. R. (2006) Understanding Phenomenology. Acumen. Understanding Movements in Modern Thought Series
  • Gibson, J. J. (1979) The Ecological Approach to Visual Perception, Boston: Houghton Mifflin. ISBN 0898599598
  • Dourish, P. (2001). Where the Action Is: The Foundations of Embodied Interaction. Cambridge: MIT Press
  • Papert, S. (1980) Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, Inc., New York, NY, USA
  • Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds.). Cambridge,MS: Harvard University Press
  • Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645
  • Lakoff, G. and Johnson, M. Metaphors We Live By. University of Chicago Press, Chicago, IL, USA, 1980.
  • Johnson, M. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason, Chicago Press, Chicago, IL, USA, 1987
  • Goldin-Meadow, S. (2011). Learning through gesture. Wiley Interdisciplinary Reviews: Cognitive Science, 2(6), 595–607
  • Antle, A. N., Corness, G., & Bevans, A. (2013). Balancing Justice : Comparing Whole Body and Controller-based Interaction for an Abstract Domain. Internation Journal of Arts and Technology, 6(4), 1–21
  • Carreras, A., & Parés, N. (2004). Designing an Interactive Installation for Children to Experience Abstract Concepts. In New Trends on Human-Computer Interaction (pp. 33–42)
  • Charoenying, T., Gaysinsky, A., & Ryokai, K. (2012). The choreography of conceptual Development in Computer Supported Instructional Environments. In Proceeding of the 2012 International conference on Interaction design and children - IDC  ’13 (Vol. 4, pp. 162–167)
  • Kynigos, C., Smyrnaiou, Z., & Roussou, M. (2010). Exploring rules and underlying concepts while engaged with collaborative full-body games. In Proceedings of the 9th International Conference on Interaction Design and Children - IDC  ’10 (p. 222). New York, New York, USA: ACM Press

Week 4

  • Participatory Design & "Wizard of Oz" on a Full-Body Interactive installation:

Students will learn the design methods of Participatory Design and WoZ adapted for full-body interaction in a workshop activity in which they will act as children participating in the design of an exergame

Practical session in the Multi-purpose room of the second underground floor of Tanger building. 
We will work on the Interactive Slide exergame platform

Week 5

  • Full-Body Interaction Learning Environments Jam:

In this Game Jam we will explore design methods and process of Full-Body Interaction Learning Experiences. The class will be in the form of a workshop that will be especially oriented towards exploring how to sketch through the body by meaningfully incorporating the notion of body and space in the design proces 

Practical session in the "Active Space" room of the first underground floor of Tanger building. 
We will meet in front of the elevators of floor -1 of Tanger building

Week 6

  • Interaction in Public Spaces
  • Theme Parks & Interaction (Part 1)

Recommended readings:

  • Reynolds, R. (1999) Roller Coasters, Flumes, and Flying Saucers: the Story of Ed Morgan & Carl Bacon, Ride Inventors of the Modern Amusement Parks. Jupiter, FL: Northern Lights Pub., 1999.

Week 7

  • Theme Parks & Interaction (Part 2)
  • Assignment of Design of Large-scale Interactive Experiences

Recommended readings:

  • Schell, J., and Shochet, J. (2001) Designing Interactive Theme Park Rides, IEEE Comput. Graph. Appl. 21, 4 (July 2001), 11-13. DOI=10.1109/38.933519 http://dx.doi.org/10.1109/38.933519.

Week 8

  • Exergames & Exertion Interfaces
  • Interactive Playgrounds
  • Museum Displays
  • Work session of Large-scale Interactive Experiences

Recommended readings:

  • Mueller, F., Agamanolis, S. and Picard, R. Exertion interfaces: sports over a distance for social bonding and fun. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI '03. ACM Press (2003), 561-568
  • Mueller, F., Agamanolis, S., Gibbs, M. R., and Vetere, F. 2008. Remote Impact: Shadowboxing over a Distance. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI '08. ACM, New York, NY, USA, 2291-2296
  • Bianchi-Berthouze, N. (2013) Understanding the role of body movement in player engagement. Human Computer Interaction 28(1), 42-75
  • Barnett, A., Cerin, E., and Baranowski, T. (2011)  Active videogames for youth: a systematic review, JPhys Act Health. 8(5), 724-773.
  • Peng, W., Lin, J., and Crouse, J. (2011) Is Playing Exergames Really Exercising? A Meta-Analysis of Energy Expenditure in Active Video Games, Cyberpsychology, Behavior, and Social Networking. 14(11), 681-688
  • Soute, I., (2013) Head Up Games,On the design, creation and evaluation of interactive outdoor games for children , PhD Thesis, Supervisor Prof. Panos Markopoulos, Technical University Eindhoven, Netherlands, ISBN: 978-90-386-3393-0

Week 9

  • Games & Special Needs

Week 10

  • Presentations of Large-scale Interactive Attractions

TOP

Sound Communication - 30852 Sound Communication - 30852

Professor: Jonatas Manzolli (Univ. Campinas, Brazil) &Jaume Piqué (DCOM)

 

Brief summary

This course will study methods, concepts and practice of music, composition and sonorization underlying the implicit and explicit transduction of content. Tthe main goals are to develop the student's perception and understanding of sound and its behaviour in the interpersonal, social, environmental, media and creative fields. Students will gain experience in designing and conducting reseah projects in one of these areas.

Description

From Noise to Sound:
A systems view of the evolution of signs and communication

Jaume Piqué (Communication Studies, UPF)

 All accurate descriptions of sounds will be biographical, based on personal experiences. Anything otherwise would be romantic. Therefore, all I can do (…) is to track a few of the many sounds that have been close to me in the places I know. - On a description on The Canadian Soundscape (Shafer, Voices of Tyranny)

The distinguished types of communication: contiguous, indexical, iconic, and symbolic (in that order) broad­ly reflect the evolutionary sequence of communication systems found among liv­ing organisms, and are correlated with increasing levels of organization of the nervous system, from plants through animals to humans.

The meaning of a work of art exists neither solely in the author's mind nor solely in the work itself, but results from an interaction between the two.


Sound Communication:
concepts, applications and design of sound and music systems

Guest Professor: Jônatas Manzolli (NICS – Unicamp, Brazil)

1. PROPOSAL

The objective of this course is to present a set of concepts and applications on interactive sound communication and let the students to formulate their own projects.

1.1 Concept

Music and Psychology of Anticipation

Psychoacoustics

Interactive Composition

Evolutionary Computation

Interactive Sound Systems

1.2 Applications

Interactive Performances

Soundscape Installations

Graphic Interfaces for Sound Control

New Music Interfaces & Robotics

Sonic Therapies

The distinguished types of communication: contiguous, indexical, iconic, and symbolic (in that order) broad­ly reflect the evolutionary sequence of communication systems found among liv­ing organisms, and are correlated with increasing levels of organization of the nervous system, from plants through animals to humans.(Saowsky) 
The meaning of a work of art exists neither solely in the author's mind nor solely in the work itself, but results from an interaction between the two.

Class attendance

Regular attendance to the classes is mandatory

Evaluation Criteria

3 EVALUATION

Groups of 3 students. Each group is going to choose a theme based on one Concept and one Application. The project consists of a) description and concepts and b) implementation. The grade for each project is: 30% for (a) + 70% for (b).

3.1 Description of Concepts

A 2-3 pages of written work describing the ideas supporting the project. This text must be related to concepts discussed during the classes, presented above in 1.1 and applications, presented in 1.2.

3.2 Implementation

The project implementation should be done in one of the following formats:

a) Graphic Diagrams with a set of sound examples and implementation description (2-3 pages)

b) A CD or DVD with graphics, sound examples and implementation description (2-3 pages)

c) a software implementation (IQR or/and Pure Data and/or Max) and implementation description (2-3 pages)

Course Structure

Week 1

  • From Noise to Sound (J. Piqué)

Week 2

  • From Sound to Soundscape (J. Piqué)

Week 3

  • Manzolli's Course Section Introduction

Week 4

  • Algorithmic Design for Interactive Soundscapes (J. Manzolli)

Week 5

  • Sound Design, Interface and Interactive Soundscapes (workshop 1) (J. Manzolli)

Week 6

  • Generative Music Systems: Iterations and Interactions (J. Manzolli)

Week 7

  • Sound Design, Interface and Interactive Soundscapes (workshop 2) (J. Manzolli)

Week 8

  • Evolutionary Sound Systems and Adaptive Strategies (J. Manzolli)

Week 9

  • Real world composition and Interactive Sonification (J. Manzolli)

Week 10

  • Presentations of Large-scale Interactive Experiences (J. Manzolli)

TOP

Real-Time Interaction - 30876 Real-Time Interaction - 30876

Professor: Anna Mura (SPECS), Vicky Vouloutsi (SPECS)

 

Description

The Real-time Interaction course focuses on the study of real-time interaction from several perspectives, both conceptual and technological. It is compounded of a more theoretical part with lectures, readings and discussions (4 sessions), that is common to all the students in the course (i.e. SMC and CSIM students), and two separated practical parts. In these hands-on practical labs, SMC students will focus on "Musical Interaction and Control", whereas CSIM students will study "Human Robot Interaction" (6 sessions for each group). All classes are programmed on Mondays from 14:00 to 16:30; check the course calendar for the distribution of theoretical (common) sessions and practical (separated) ones.

Course Objectives

Theoretical sessions

The theory part starts discussing the concepts of "interactivity" and "real-time interaction" showing how relative and subjective both concepts can be. 
Then a more detailed view on the foundations of Human Robot Interaction is presented.

Human Robot Interaction Lab

The 21st century is dominated by a new strategic technology: robotics. Indeed, the large-scale introduction of robots in society is already happening and the social compatibility of such robots is of fundamental importance. In order to meaningfully interact with humans, robots must develop and advanced real-world social intelligence that includes novel perceptual, behavioural, emotional, motivational and cognitive capabilities. The CSIM hands-on part of the Real Time Interaction course will introduce the students to the Human Robot Interaction domain and expand on novel robot-based perceptual, cognitive and motor architectures. The students will learn what it means for a robot to perceive humans, understand and perform actions accordingly. In particular, students will learn how to implement personality traits in a robot and experiment with "emotions" using the humanoid robot Nao. The course will start with an introduction to Human Robot interaction and its relevant field of research such as AI, robotics, natural language understanding, cognitive and social sciences etc. During the second class, projects will be assigned and groups will develop their own personality models. During the following classes the tutorial will focus on getting the students familiar with a humanoid robot and students will learn how to configure personality traits and implement emotional states based on existing experimental models/algorithms. The projects will be completed with a final demonstration during the last class of the tutorial.

Class attendance

Regular attendance to the classes is mandatory.

Evaluation Criteria

Development of project assignments, project presentation & participation in class.

Course Structure

Week 1

Introduction to real-time interaction

Week 2

Introduction to Human-Robot interaction

Week 3

Real-time interaction and NIME

Week 4

Models of emotions, hand on with the robot

Week 5

Mapping

Week 6

Evaluation and User Experience in HCI

Week 7

Personality models and allostatic control (part 1), hands on with the robot

Week 8

Personality models and allostatic control (part 2), hands on with the robot and group projects

Week 9

Group project work

Week 10

Project presentations

TOP

Adaptive Behavior - 30864 Adaptive Behavior - 30864

Professor: Anna Mura (SPECS), Riccardo Zucca (SPECS)

 

Description

The course will expose the students to key concepts, theories and experimental pardigms emerged from the study of the biological basis of adaptive behavior. In the first part of the course we will provide an overview of the basic principles of evolution, genetics and organization of the brain to finish with an introduction of the biological basis of regulation. 

In the second part of the course we will mainly focus on three different types of learning (classical and operant conditioning and probabilistic learning), exploring what is known about the neuronal substrates as well as computational and mathematical theories. Finally, we will overview the main principles of autonomous exploration and foraging in rodents and how they can be translated into biologically based cognitive technologies for autonomous exploration in the real world.

Course Objectives

The course is designed to familiarize students with the general principles of adaptive behavior that govern individual and social behavior. Students presentation and discussions will be aimed to foster critical evaluation of relevant experimental literature. 

Class attendance

Regular attendance to the classes is mandatory. During each class students will have to present and discuss a scientific paper centered on the topic. The final project consists in the writing of a report and its presentation (in individual form or in small groups) expanding on one of the themes covered during the trimester (i.e., the impact of bionics on society, genetic engineering, etc.).

Evaluation Criteria

Quality of class participation, presentation of the scientific article for class discussion, final project presentation.

Course Structure 

(may vary slightly)

Week 1

Introduction to evolution, genetics and environmental adaptation

Week 2

The nervous system of simple and complex organisms

Week 3

The human brain and its organisation: an overview

Week 4

Brain regulation of physiological states: Introduction to the concept of homeostasis in complex organisms 

Week 5

Physiology of reward

Week 6

The cerebellum and classical conditioning: 

Cerebellar physiology and neuronal substrates of conditioning

Week 7 

Theoretical and computational models of conditioning

Week 8

Probabilities in the brain

Week 9

Navigation and foraging

Week 10

Final project presentations

TOP