Assessing metrics for video quality verification in Livepeer’s ecosystem (I)


Assessing the quality of a single asset

Above: two different lossly compressed versions of an original picture (a frame from Big Buck Bunny). Definition of a criterion on what is an acceptable quality level is not an easy task, hence the plethora of objective quality methods attempting to measure quality.

SSIM time evolution for the first 1240 frames (50 seconds) of Big Buck Bunny, encoded at 500 kbps (above) and 250 kbps (below)

Time evolution of the ratio between SSIMs encoded at 500 kbps and 250 kbps. First 1240 frames (50 seconds) of Big Buck Bunny

SSIM time evolution for the first 1240 frames (50 seconds) of Big Buck Bunny, encoded to 500 kbps with a watermark

Time evolution of the ratio between SSIMs encoded at 500 kbps and with a watermark. In some segments (ratios below 1), watermarks seem to give even better SSIM results than the ‘bona fide’ rendition at 500 kbps.

Bringing in more metrics

Time evolution of the ratio between SSIMs and PSNRs for encodings at 500 kbps and 250 kbps.

Time evolution of the ratio between SSIMs and PSNRs, encoded at 500 kbps and at 500 kbps with a watermark.

Generalizing to several assets

Pairs plot for our dataset of 140 Youtube clips. Distance metrics (cosine, euclidean and Hamming) are fairly linearly correlated, whereas SSIM and PSNR show some logarithmic / exponential relationship with each other. VMAF and MS-SSIM also have some degree of linear correlation to each other. MS-SSIM appears as the one with the most compact distribution.

Spearman correlation table for all seven metrics used in our analysis

Frame from one of the outliers. The video presents a high amount of motion and high detail hard to deal with for the codec in a reasonable compression rate.

Conclusions and further work

References

Livepeer
Livepeer has 30 repositories available. Follow their code on GitHub.
github.com
epiclabs-io/YT8M
(Machine) Learning from YouTube8M dataset . Contribute to epiclabs-io/YT8M development by creating an account on…
github.com
Toward A Practical Perceptual Video Quality Metric
measuring video quality accurately at scale
medium.com
Video Quality Assessment
In Eyevinn’s initiative to share our knowledge around quality we continue with addressing video quality assessment…
medium.com
Human visual system model — Wikipedia
needs additional citations for verification .improve this article by adding citations to reliable sources. Unsourced…
en.wikipedia.org

About the authors

Dash.js V 2.6.8 released with relevant features.

Dash.js V 2.6.8 released with relevant features.

Epic Labs, a proud member of the DASH Industry Forum and Digital Production Partnership (DPP), announces the release of version 2.6.8 of dash.js that incorporates major improvements such as low latency Chunked CMAF support.

Several enhancements have been incorporated in this new version such as fixes that improve captioning and DRM handling. But above all, two implementations are extremely important, one of them is the support of persistent licenses (DRM), while the other is the introduction of Low latency chunked CMAF over HTTP, which enables 2-4 second latency for live streams keeping a smooth playback experience. Supporting this feature in a scalable manner is enabled by major CDNs such as Akamai.

The development aims to provide an alternative to older protocols like RTMP whose main advantage over HTTP streaming ones was its lower latency.

In the client side, part of the solution is based on the use of the fetch API offered by browsers, which allows reading incoming chunks as they arrive, instead of waiting for them to be completed. Dash.js analyzes the byte stream representing each chunk and pushes it to the video decoder as soon as it has enough information to start playback stably. Every player based on dash.js is able to benefit of this new feature by doing nothing but updating to the latest version.

Jesús Oliva, Epic Labs Media Engineer Lead & Founder: “This is an exciting achievement because it provides a low latency chunked solution based on open standards, protocols and source code. A real alternative to proprietary or custom solutions that require to package content in a strange manner or exceed the norm. This is a great deal for the industry.”

 

 

Dash Industry Forum Dash.js
Dash Industry Forum Dash.js

The DASH Industry Forum (DASH-IF) establishes interoperability gui- delines on the usage of the MPEG-DASH streaming standard. MPEG- DASH simpli es and converges the delivery of IP video, to provide a rich and enjoyable user experience, to help drive down costs and ultimately to enable a better content catalogue to be offered to consumers.

Epic Labs Media Engineering Services
Epic Labs Media Engineering Services

Epic Labs, proud member of the DASH Industry Forum, is a software engineering center that helps companies to innovate in Media, offering advanced video solutions and several collaboration proposals in digital transformation.

WebRTC real-time video App case study. Developing of a low-latency

Developing of a low-latency, high-scalability and real-time video App for social media

The Customer: A company that is dedicated to developing different Apps for mobile devices. Since 2006 they have been pioneers in developing different social networking platforms and games for Middle East and Africa.

 

The Background: The product is a social media network based on video, a platform for mobile devices that allows to make a personalized streaming broadcast reaching a large audience of followers and fans. Effectively, it allows to create your own television channel with massive broadcast over the Internet. Each user can utilize the App to showcase their talents or different life moments, interacting and speaking in groups with live, low-latency video and audio.
This App is consumed also by celebrities to share things they are interested in, provide news and updates and occasionally chat and interact with their fans.

 

The Challenge: The customer wanted to create an interactive broadcast solution as the cornerstone for a social media network. The experience would provide a great level of interactivity via video and chat, combined with the ability to reach several thousands of users.
The key point was that the application had to work with a latency of less than half a second to facilitate communication between the different participants regardless of where they were located in the world.

 

The Solution: Epic Labs proposed a solution based on WebRTC technology, a protocol that allows a connection of video-audio-data between browsers and native apps with a latency less than one second. Epic Labs worked on getting the benefits of WebRTC in terms of low latency combined with the use of broadcast-oriented protocols (i.e. DASH and HLS) allowing for cost-effective, high-quality and massive content delivery.

The solution allows for video composition, a layout that includes the main signal and windows with the other users that participate in each App channel, to be broadcasted through a CDN.

 

The Results: The project was executed following Epic Agile Methodology for Media. At the end, the customer got exactly the product they needed in the timing they wanted, with a perfect balance between image quality and low latency. Epic Labs also provided the necessary SDKs to integrate the solution in Android and iOS. Now, the App is a complete success in its target region.