Multi-Instance Dynamic Ordinal Random Fields for Weakly-supervised Facial Behavior Analysis

  • Authors
  • Ruiz A; Ognjen R; Binefa X; Pantic M
  • UPF authors
  • BINEFA VALLS, XAVIER; RUIZ OVEJERO, ADRIÀ;
  • Type
  • Scholarly articles
  • Journal títle
  • IEEE Transactions on Image Processing
  • Publication year
  • 2018
  • Volume
  • 27
  • Number
  • 8
  • Pages
  • 3969-3982
  • ISSN
  • 1057-7149
  • Publication State
  • Published
  • Abstract
  • Mutliple-Instance-Learning (MIL) has become a popular modelling framework for addressing weakly-supervised problems in Computer Vsion. In MIL, a training set is formed by bags (sets of feature vectors or instances) and only labels at bag-level are provided. In this work, we address this problem when instance labels are naturally represented as ordinal variables and bags are structured as temporal sequences: Multi-Instance Dynamic-Ordinal-Regression (MI-DOR). In our approach, we assume two scenarios: (i) bag label is equal to the maximum label of its constituent instances, (ii) bag-label provides information about the evolution of the instance ordinal labels within the sequence. To address these modeling problems, we propose Multi-Instance Dynamic Ordinal Random Fields (MI-DORF). In this framework, we treat instance-labels as temporally-dependent latent variables in a graphical model. The different MIL assumptions are modelled by incorporating a high-order potential relating bag and instance-labels into the energy function. Moreover, we extend the proposed framework for Partially-Observed MI-DOR problems, where a subset of instance labels are also available during training. We show the benefits of the proposed framework for weakly-supervised facial behavior analysis. Specifically, we evaluate the model on the tasks of Facial Action Unit (DISFA dataset) and Pain Intensity (UNBC dataset) estimation. We show that the proposed framework significantly outperforms alternative approaches on these tasks.
  • Complete citation
  • Ruiz A; Ognjen R; Binefa X; Pantic M. Multi-Instance Dynamic Ordinal Random Fields for Weakly-supervised Facial Behavior Analysis. IEEE Transactions on Image Processing 2018; 27(8): 3969-3982.
Bibliometric indicators
  • 9 times cited Scopus
  • 7 times cited WOS
  • Índex Scimago de 1.809 (2018)