Publications & Deliverables
Publications
A. Kokaram, Vibhoothi, J. Zouien, F. Pitié, C. Nash, J. Bentley, and P. Coulam-Jones, “Demystifying the use of Compression in Virtual Production” in SMPTE Media Summit 2024
Vibhoothi, J. Zouien, F. Pitié, and A. Kokaram, “Unravelling the Power of Single-Pass Look-Ahead in Modern Codecs for Optimized Transcoding Deployment”in NAB 2024
A. Cartas, C. Ballester, and G. Haro. 2024. "Two Weakly Supervised Approaches for Role Classification of Soccer Players." Accepted for presentation at the 7th International ACM Workshop on Multimedia Content Analysis in Sports (MMSports '24), Melbourne, Australia.
Díaz-Juan, A., Ballester, C., & Haro, G. "SoccerHigh: a benchmark dataset for automatic soccer video summarization." In Proceedings of the 8th International ACM Workshop on Multimedia Content Analysis in Sports (pp. 121-130), 2025.
Ghafourian, M., & Sukno, F. M. (2025). "NLML-HPE: Head Pose Estimation with Limited Data via Manifold Learning." International Joint Conference on Biometrics (IJCB 2025)
Vibhoothi., Zouein, J., Pitié, F. & Kokaram, A. "Using Single-Pass Look-Ahead in Modern Codecs for Optimized Transcoding Deployment." SMPTE Motion Imaging Journal. 133, 2024
Vibhoothi, V., Pitié, F., & Kokaram, A. "LiteVPNet: A Lightweight Network for Video Encoding Control in Quality-Critical Applications." IEEE Picture Coding Symposium (PCS 2025)
Vibhoothi, V., Zouein, J., Shreejith, S., Kempf, J. B., & Kokaram, A. "An Empirical Study of Reducing AV1 Decoder Complexity and Energy Consumption via Encoder Parameter Tuning" IEEE Picture Coding Symposium (PCS 2025)
Zouein, J., Vibhoothi, V., & Kokaram, A. "AV1 Motion Vector Fidelity and Application for Efficient Optical Flow." IEEE Picture Coding Symposium (PCS 2025)
Zouein, J., Javidnia, H., Pitié, F., & Kokaram, A. "Leveraging AV1 motion vectors for Fast and Dense Feature Matching." International Conference on Intelligent Reality (ICIR 2025)
Ganbaatar, B. E., & Pitié, F. "Towards Consistent Automatic Colour Grading: A Scene-Aware Neural Network for Illuminant Estimation." In Proceedings of the 22nd ACM SIGGRAPH European Conference on Visual Media Production (pp. 1-10), 2025
Li, C., Bled, C., Fernandez, R., & Shanker, S. "ReTiDe: Real-Time Denoising for Energy-Efficient Motion Picture Processing with FPGAs." In Proceedings of the 22nd ACM SIGGRAPH European Conference on Visual Media Production (pp. 1-10), 2025
Veekanchery, H., Lyons, H., Coulam-Jones, P., & Shanker, S. "Towards Energy Monitoring in Visual Processing Pipelines." SMPTE Media Technology Summit 2025
Bled, C., & Pitié, F. "Multi Task Denoiser Training for Solving Linear Inverse Problems." In Proceedings of the 22nd ACM SIGGRAPH European Conference on Visual Media Production (pp. 1-9), 2025
Nash, C., Coulam-Jones, P. "Hidden Depths: Disguise’s Integration of Depth and Volumetric Capture Workflows for Virtual Production" In NAB Broadcast Engineering and Information Technology (BEITC) Proceedings 2025
Deliverables
The Handbook provides a comprehensive overview of the project procedures with specific examples. It also gives an overview of consortium partners, communication procedures and describes in detail how documents related to the project should be formatted.
This deliverable reports on the plans of EMERALD to manage data according to FAIR principles. The different data in a large sense managed by the consortium are discussed.
The Self-Assessment Plan sets out the ways in which the project’s operational performance will be assessed, including the measurement of progress toward achieving the Objectives. It includes a self-assessment plan for each task within each WP from 2 to 6, outlining the evaluation strategy, the success indicators and the timetable, with the level of detail appropriate at this early stage of the project.
This deliverable updates the EMERALD project’s Data Management Plan (DMP), reflecting progress since D1.2 (month 6). It details advancements in data classification, accessibility, and open science practices, with an emphasis on public dissemination. Restricted data remains under review for potential partial release. The report also revisits key strategies from the initial DMP and outlines steps toward the final version due in month 30.
This deliverable provides the final EMERALD project’s Data Management Plan (DMP), reflecting progress since D1.4 (month 15). It details the final advancements in data classification, accessibility, and open science practices, with an emphasis on public dissemination. Restricted data remains under review for potential partial release. The report also revisits key strategies from the initial DMP and its updated version D1.4.
This document provides an in-depth analysis of key variables relevant to video matting, followed by the architecture of the dataset utilised in the study. It outlines the process of capturing content for the dataset, including planning, participant recruitment, early trials, and final development alongside the obtained results. The preliminary dataset comprises 80 captures and processed videos, serving as a foundation for training video matting algorithms.
This report details Disguise’s preliminary investigations into Volumetric technology and data as it pertains to the goals of the EMERALD project. Discussion of initial prototypes developed to laid the groundwork for the remainder of the necessary development work for EMERALD, along with plans for upcoming work building on it, makes up the bulk of this report and forms the structure from which future technology will be iterated upon.
AI-driven methods for player classification and action spotting in sports videos, developed for the EMERALD project. It highlights the implementation, dataset usage, and results of both tasks.
This document provides an explanation of the reasons for the collection of a new, natural history, dataset. It describes the method followed to retrieve the media and process it into the correct format along with the process for human annotations of ground truth. It also includes some early experiments with the dataset exploring the opportunities for zero-shot classification.
This document gives some background on colour grading with regard to colour matching in particular and details the design and development of an automated colour matching algorithm that adjusts colour balance and flare of shots within the context of the scene in a motion picture. An initial evaluation to determine viability, scope and evaluation strategy is performed, showing an overall reduction in time required to achieve a match, while a more detailed evaluation is needed.
This deliverable presents the final dataset creation for automatic video matting in the EMERALD project. It outlines the planning, trials, and results from capturing and processing 100 videos, essential for algorithm training. The tool and training outcomes will be detailed in deliverable D2.11.
This report describes Disguise’s efforts to build a tool for enabling dynamic volumetric content creation for deployment in a variety of virtual production workflows. As an extension of this, there has also been an investigation into supporting methodologies for extracting the material properties of subjects for the purpose of enabling more accurate reactions to changes in lighting.
This report describes Disguise’s dataset used to evaluate the tools more fully described in deliverable D2.7. It covers the considerations made in gathering the source material, the process of using it in validating the pipeline and the known limitations of the dataset that will need to be accounted for in future work.
This document details AI-driven methods for video summarization in soccer videos, developed for the EMERALD project. It highlights the implementation, dataset usage, and results of the task.
This deliverable describes the algorithm developed in EMERALD (WP2T1) to generate metadata for wildlife video content. The system detects animals within frames, identifies their spatial and temporal presence, and classifies species with human oversight. Building on dataset work in D2.4, the tool is now ready for deployment with production teams to support evaluation and further refinement.
This deliverable presents the development and integration of a deep learning–based model for automatic video matting in virtual production. Leveraging both public datasets and a custom Brainstorm dataset, the MobileOne-based model achieves accuracy comparable to state-of-the-art methods while offering faster inference suitable for real-time use. The report outlines training strategies, fine-tuning approaches, and integration into broadcast workflows via NDI and InfinitySet, demonstrating the tool’s potential to streamline production, cut resource use, and support more sustainable content creation.
This report describes the new featureset for colour calibration of LED panels used in Virtual Production that have been integrated into Disguise’s Designer platform. Colour management is also supported by Trinity College Dublin’s new spatially consistent LUT generation system, which is also described in detail.
This deliverable reports on the development of an automatic colour balancing tool for virtual studio presenters, created within the framework of the EMERALD project. The tool combines a Virtual Video Capture and Inserting Laboratory, developed to generate large-scale synthetic datasets, with a machine learning algorithm trained to adapt the colour properties of presenters recorded over chroma to match the characteristics of virtual environments. Integrated into Brainstorm’s InfinitySet system, the solution operates in real time and has been validated with both real and synthetic data, demonstrating improved visual coherence, reduced manual workload, and alignment with EMERALD’s objectives of efficiency and sustainability in digital media production.
This report presents an experimental evaluation of two machine learning algorithms (FL and TCD) designed to pre-match footage for colour grading, with the aim of improving efficiency in media production. A controlled study with five professional colourists assessed the algorithms' impact on grading time and quality across diverse scenes. Results showed both algorithms improved upon raw footage, but the FL algorithm was notably more effective, with colourists rating its output at 70% of a human-matched baseline. While time savings were variable, the study suggests a potential 20% time reduction with the FL algorithm. The findings indicate significant potential for AI-driven tools to streamline colour grading, allowing colourists to achieve better results more quickly with fewer resources.
This deliverable describes the development of an open-source virtual production system by BBC R&D within EMERALD (WP2T4). Using fiducial marker tracking and Gaussian splatting for real-time 3D reconstruction, the system enables high-quality, lightweight rendering of virtual sets. Designed for portability, it supports low-cost productions and is being tested with the BBC Natural History Unit.
The first deliverable for EMERALD Work Package 3, which focuses on providing frameworks for assessing the consumption of common visual processing and virtual production workflows. This report covers Disguise and Trinity College Dublin’s investigations into building tools for profiling the requirements of current systems and their initial recommendations for improving efficiencies in this space.
The second task in EMERALD Work Package 3 focuses on developing frameworks for assessing the consumption of common visual processing and virtual production workflows. This report presents the API descriptions for the energy profiling tool (CPU) developed by Trinity College Dublin and an example integration for profiling CPU tasks using the NUKE workflow.
The third task in EMERALD Work Package 3 combines the GPU and CPU energy profiling tool as an integrated flow for analysing common visual processing and virtual production workflows. This report presents the tool API descriptions for the energy profiling tool (CPU + GPU) developed by Trinity College Dublin, and the installation instructions for the open source tooling.
The third task in EMERALD Work Package 3 combines the GPU and CPU energy profiling tool as an integrated flow for analysing common visual processing and virtual production workflows. This report presents the tool overview and detailed working of the energy profiling tool (CPU + GPU) developed by Trinity College Dublin and an example integration for profiling CPU & GPU tasks.
This report presents the benchmarking results from a selection of tools developed in the project, using the combined energy profiling tool developed by Trinity College Dublin. The selected tools are evaluated on our test workstation setup, and the performance results are captured using vendor tools as well as the energy profiler developed in this project. The results described in this report include the energy-performance analysis of the denoiser plugin from Trinity College Dublin (CPU and GPU), impact of AV1 parameters on the Decoder performance (Trinity College Dublin, CPU) and the AI-colour grading plugin from FilmLight.
This document is the first version of the algorithm and tool for the Real-time estimation of the physical orientation of a presenter on set. It reports on the initial steps taken in developing the head pose estimation algorithm and corresponding tool. Comprehensive details are provided regarding the background, state of the art, training and testing phase of the proposed algorithm. In addition, it encompasses the processing of training datasets and augmenting them with synthetically generated data for the missing angles. These datasets are essential assets for the subsequent phase, the training of the algorithm developed by UPF.
This document presents the second version of the algorithm and tool for real-time estimation of a presenter’s physical orientation on set. It describes the development of the head pose estimation algorithm, including background, training, testing, and dataset processing. The report also details the augmentation of training datasets with rendered data to cover missing angles, supporting the continued improvement of the UPF-developed algorithm.
This document reports on the resulting applications for the generation of 3D animation from monocular video input. It details the starting point of the applications involved, the steps taken to improve them and a precise explanation of the most relevant algorithms, with a special emphasis on the animation generation phase. Additionally, it explains the modifications on the User Interface (UI), leading to a better usage of the application as well as to accommodate new features: the batch processing of videos, multiclip pipeline, propagation window and video cropping. The applications combine AI and analytical algorithms for a fast animation generation while keeping low hardware requirements, saving time and energy on the process.
This document summarizes the development of energy-efficient streaming tools within the EMERALD project. It outlines FilmLight’s Smart Streaming Architecture for cloud-based post-production and MOG’s Green Streaming ecosystem for large-scale content distribution, together enabling more sustainable and efficient media workflows.
This document summarizes the development of a cloud-based ingest server capable of handling multiple input and output formats. It outlines how the solution was enhanced through the integration of a machine learning–based metadata enrichment tools for live sports broadcasting, enabling a more streamlined and efficient media production workflow.
This report describes a tool that uses compressed domain data for motion extraction, generating features for production workflows. Initial results suggest the motion data is usable, with further depth extraction efforts to be covered in a future report.
This document details the approach being taken for on location content analysis. This describes the limitations in terms of hardware and connectivity. It details the proposed tool that has been developed to prototype stage and demonstrated to end users along with their feedback and next steps.
This report details efforts made towards a robust ST 2110 compliant streaming solution which aims to deploy in Virtual Production environments that integrate with experimental new implementations of the AV1 codec. It describes the motivation for using both feature sets in a VP setup and the eventual aims to combine both for a streamlined, modern approach to content delivery for VP shoots.
This report presents an EMERALD demonstrator developed by Trinity College Dublin for FPGA-based acceleration of image denoising. The system implements a cGAN encoder-decoder model, compressed through quantisation and pruning, on a PCIe-connected Alveo U50 FPGA. Integrated with NUKE via the NUKE-FPGA plugin or run standalone, the demonstrator is benchmarked against an NVIDIA A4000 GPU. Results show state-of-the-art denoising performance (PSNR) on colour and grayscale images, with the FPGA achieving 4.8× higher energy efficiency than the GPU. This work has been accepted for publication at CVMP 2025.
This deliverable details the low-power algorithm developed in EMERALD (WP5 T3) for content processing and data capture. The method encodes video frames into semantic embeddings to identify anomalous or editorially relevant content, while remaining lightweight enough for use on standard laptops. Already deployed with production teams, the tool is under evaluation to refine its effectiveness in field conditions.
This document provides the rationale of the project website implementation, key concepts, design and actual implementation are briefly discussed. It also gives an introduction to the different social media channels that have been set up.
Plan for dissemination activities and exploitation of results, with links to the conferences, papers, web resources and publications produced during the project.
This document presents a comprehensive evaluation strategy for the EMERALD project, detailing the key performance indicators used to assess its impact. It highlights how the project's contributions are measured in relation to its defined goals and tasks, ensuring a thorough analysis of results and progress.
This document is the second version of the Dissemination, Communication and Exploitation Plan for the EMERALD project. The deliverable updates the first version of the plan reported on month 3 in the deliverable D6.2. It reports on activity and plans for the sharing and communication of the results of the project with the wider community and how the partners plan to exploit the result of their work.
This report summarises the dissemination and communication activities of the EMERALD project. It covers scientific publications, conferences, industry engagement and online outreach, targeting both academic and industrial audiences. The project achieved strong visibility in key media technology forums and supports the continued uptake of its results beyond the project lifetime.
This document presents the final Exploitation Plan for the EMERALD project, updating the version delivered at month 18. It outlines the exploitation strategies for the tools and assets developed by each partner, including key features, target markets, status, and intellectual property aspects, supported by canvas business models. It also includes route-to-market approaches, competitive analyses, and an overall market outlook, reflecting the results achieved over the project’s 30-month duration.
This document presents a comprehensive evaluation for the EMERALD project’s contributions, overall results and success indicators in all the project dimensions, including EMERALD algorithms and tools, efficiency and energy saving, and user responses to demonstrations
This deliverable provides a short description of how the final demonstration for the EMERALD project (in the form of a showreel for social media consumption) was assembled by all project partners. It is meant to be the written accompaniment to that video, which can be found online at https://youtu.be/EnnmQqBOQPA