Assessing metrics for video quality verification in Livepeer’s ecosystem (I)

Assessing the quality of a single asset

Above: two different lossly compressed versions of an original picture (a frame from Big Buck Bunny). Definition of a criterion on what is an acceptable quality level is not an easy task, hence the plethora of objective quality methods attempting to measure quality.

SSIM time evolution for the first 1240 frames (50 seconds) of Big Buck Bunny, encoded at 500 kbps (above) and 250 kbps (below)

Time evolution of the ratio between SSIMs encoded at 500 kbps and 250 kbps. First 1240 frames (50 seconds) of Big Buck Bunny

SSIM time evolution for the first 1240 frames (50 seconds) of Big Buck Bunny, encoded to 500 kbps with a watermark

Time evolution of the ratio between SSIMs encoded at 500 kbps and with a watermark. In some segments (ratios below 1), watermarks seem to give even better SSIM results than the ‘bona fide’ rendition at 500 kbps.

Bringing in more metrics

Time evolution of the ratio between SSIMs and PSNRs for encodings at 500 kbps and 250 kbps.

Time evolution of the ratio between SSIMs and PSNRs, encoded at 500 kbps and at 500 kbps with a watermark.

Generalizing to several assets

Pairs plot for our dataset of 140 Youtube clips. Distance metrics (cosine, euclidean and Hamming) are fairly linearly correlated, whereas SSIM and PSNR show some logarithmic / exponential relationship with each other. VMAF and MS-SSIM also have some degree of linear correlation to each other. MS-SSIM appears as the one with the most compact distribution.

Spearman correlation table for all seven metrics used in our analysis

Frame from one of the outliers. The video presents a high amount of motion and high detail hard to deal with for the codec in a reasonable compression rate.

Conclusions and further work


Livepeer has 30 repositories available. Follow their code on GitHub.
(Machine) Learning from YouTube8M dataset . Contribute to epiclabs-io/YT8M development by creating an account on…
Toward A Practical Perceptual Video Quality Metric
measuring video quality accurately at scale
Video Quality Assessment
In Eyevinn’s initiative to share our knowledge around quality we continue with addressing video quality assessment…
Human visual system model — Wikipedia
needs additional citations for verification .improve this article by adding citations to reliable sources. Unsourced…

About the authors

NLP for Computer Vision – Having breakfast with Terminator

In this video, we are talking about another tool of Machine Learning- Natural Language Processor- which could be used to enhance the accuracy and power of computer vision. Natural Language Processing or NLP is used to understand the meaning of human language by training an algorithm that looks for the relationship between words. One of these relationships could be “belongs to” or “contains”, and so we can relate several concrete terms (fruit, cereals, milk) with an abstract concept that contains all (breakfast, morning). We can use this feature to increase our accuracy in detecting concepts in a scene. We also propose this method for the use case “Contextual Advertising” to avoid wrong detections derived from the context of a video:

0:00 – Intro
0:37 – Object Detection
1:07 – NLP definition
1:16 – Context example 1: BREAKFAST
2:00 – Context expample 2: BASEBALL
2:23 – Contextual advertising USE CASE
3:00 – Recap

La inteligencia artificial aplicada a los archivos de televisión

La inteligencia artificial aplicada a los archivos de televisión

Presentación Power Point de Epic Labs en las II Jornadas de Archivo y Televisión organizadas por RTVE el 16 de abril de 2018, dentro del panel “Metadatado automático y herramientas cognitive en archivos de televisión”.



Descargate la presentación.


Machine Learning for Business as Unusual. Álvaro Gonzalez and Ignacio Peletier.

Álvaro González. Epic Labs AI Lead Engineer.
Ignacio Peletier. Epic Labs AI Engineer.

In 2004, Chris Anderson presented his article “The Long Tail” in Wired magazine, explaining how the future of digital businesses would focus on a lot of niches in front of mass markets. This is one of the most important things that allows video in the internet era, where each person can search and find content that fits perfectly to their likes, finding it either within the user generated content. And that is exactly where the qualities of artificial intelligence can help.

“It is not about doing the same things by replacing people with machines, it is about doing business as unusual, doing something that is not usually done.” Álvaro González. Epic Labs AI Lead Engineer.

There is a lot of talk these days about how AI can simplify workflows and perform tasks that are repetitive and automatic to achieve more efficiency. But what if instead of focusing on resource optimization, what we do is think about those things that are not done because there is not enough people or budget for it? It is then when machine learning, video and long tail enable new and interesting use cases that can be undertaken through artificial intelligence tools, either in a 100% automated process or requiring some kind of supervision.

In sports, we have the case of those that have a local interest which are minority, or smaller leagues. Thanks to the commoditization of content generation and delivery technologies, it is possible to organize productions operated by artificial intelligence. Focusing on football, a machine learning system can enable a local football match to be made automatically and broadcast it over a streaming channel. You can even train to do replays and include graphics. It is not a question of whether the production is as good as a handful of technicians could have made, it is about having a product where there was nothing before, which among other things, allows monetizing that long tail through dynamic advertising or a unique Sponsorship.

This technology can also be applied to recognize and cross-reference material to create content. As we continue to talk about football, this can be applied in parallel to the cameras on the air to all those that are being ingested on some server, taking advantage of material that is usually discarded because no staff is available to review and classify it. In this way, new content can be created, looking for this niche once again, as can be a summary of everything a player has done in a football match, at the choice of the user.

Álvaro González, Epic Labs AI Lead Engineer: “This also applies to any other sport, personal hobbies and much other cases. We are living in an era where the end-user is able to generate their own content if they cannot find it, and the artificial intelligence can help us to get and retain that people.  More than ever, unusual content requires unusual business.”

Stable GPU TensorFlow 1.3 environment. By Ignacio Peletier

Using GPUs for training machine learning applications can speed operations up to 40 times faster than using a common CPU. In this video a stable environment for using TensorFlow 1.3 with CUDA is presented and a simple script is given so we can assure we are using our GPU!

Cuda 8.0 →
cuDNN 6.0 → (requires sign up)
Tensorflow 1.3 →

Pip and virtualenv:

sudo apt-get install python-pip python-dev build-essential
sudo pip install --upgrade pip
sudo pip install --upgrade virtualenv

Create virtual enviroment´s directory:

mkdir my_envs && cd my_envs

Create virtualenv:

virtualenv env

Activate virtualenv:

source env/bin/activate

Installing tensorflow-gpu:

pip install tensorflow-gpu #pip≥8.1

Running the script:


keep learning with Epic Academy

Inteligencia Artificial real para Broadcast y Media BIT Experience

Video y Power Point del workshop ofrecido por Epic Labs y Antonio Tablero (FOX) sobre Inteligencia Artificial, Machine Learning, para Media y Broadcast el pasado 4 de octubre en BIT Experience 2017. (
Incluimos en esta sección el vídeo completo, un resumen de 10´con los puntos clave y el Power Point que se utilizó con toda la información.




Workshop sobre Inteligencia Artificial para Broadcast

Antonio Tablero, Head of Broadcast Engineering & Technology, FNG Europe and Africa y Epic Labs ofrecen un workshop sobre Inteligencia Artificial para Broadcast en BIT Experience.

Se celebra el miércoles 4 de octubre a las 17:00 en la sala S15+S16 del Auditorio Sur de IFEMA  bajo el nombre: Inteligencia artificial real para broadcast. Automatización de metadatos, auto-tagging y moderación de contenidos con Machine Learning.

El impacto de la Inteligencia Artificial en nuestra sociedad se prevé que sea tan importante como el descubrimiento de la electricidad, una ola que sin duda cubrirá el sector de la producción y difusión de contenidos en los próximos años. A pesar de este escenario tan contundente, poca información real hay sobre la aplicación y los beneficios que aporta esta tecnología a la industria Broadcast y Media, y sobre todo, cómo puede utilizarse hoy en día, sin necesidad de esperar al futuro.

En cuanto al contenido, se expondrá un caso de uso actual sobre moderación automática de contenidos y auto-tagging. Por otra parte, se explicarán cuáles son las bases para abordar un proyecto de Inteligencia Artificial aplicado a entornos de canales de televisión ya sean tradicionales o Web TV, plataformas de contenidos, así como el amplio ecosistema de empresas de servicios de producción y difusión.

Alfonso Peletier, Founder & CEO de Epic Labs: “Estamos ayudando a empresas del sector Media en sus procesos de innovación y transformación digital; tecnologías de Deep Learning e Inteligencia Artificial nos habilitan nuevos e interesantes casos de uso para Media y Broadcast. Es un orgullo poder aportar las investigaciones y desarrollo del Equipo de Epic Labs junto a un profesional tan reputado como Antonio Tablero en el marco de BIT Experience”.


La sesión incluye una demostración y una ronda de Q&A con expertos de Epic Labs que resolverán las dudas de los asistentes.