VP9

History

VP9 is the last official iteration of the TrueMotion series of video formats that Google bought in 2010 for $134 million together with the company On2 Technologies that created it. The development of VP9 started in the second half of 2011 under the development names of Next Gen Open Video (NGOV) and VP-Next. The design goals for VP9 included reducing the bit rate by 50% compared to VP8 while maintaining the same video quality, and aiming for better compression efficiency than the MPEG High Efficiency Video Coding (HEVC) standard. In June 2013 the "profile 0" of VP9 was finalized, and two months later Google's Chrome browser was released with support for VP9 video playback. In October of that year a native VP9 decoder was added to FFmpeg, and to Libav six weeks later. Mozilla added VP9 support to Firefox in March 2014. In 2014 Google added two high bit depth profiles: profile 2 and profile 3.

In 2013 an updated version of the WebM format was published, featuring support for VP9 together with Opus audio.

In March 2013, the MPEG Licensing Administration dropped an announced assertion of disputed patent claims against VP8 and its successors after the United States Department of Justice started to investigate whether it was acting to unfairly stifle competition.

Throughout, Google has worked with hardware vendors to get VP9 support into silicon. In January 2014, Ittiam, in collaboration with ARM and Google, demonstrated its VP9 decoder for ARM Cortex devices. Using GPGPU techniques, the decoder was capable of 1080p at 30fps on an Arndale Board. In early 2015 Nvidia announced VP9 support in its Tegra X1 SoC, and VeriSilicon announced VP9 Profile 2 support in its Hantro G2v2 decoder IP.

In April 2015 Google released a significant update to its libvpx library, with version 1.4.0 adding support for 10-bit and 12-bit bit depth, 4:2:2 and 4:4:4 chroma subsampling, and VP9 multithreaded decoding/encoding.

In December 2015, Netflix published a draft proposal for including VP9 video in an MP4 container with MPEG Common Encryption.

In January 2016, Ittiam demonstrated an OpenCL based VP9 encoder. The encoder is targeting ARM Mali mobile GPUs and was demonstrated on a Samsung Galaxy S6.

VP9 support was added to Microsoft's web browser Edge in 2016.

In March 2017, Ittiam announced the completion of a project to enhance the encoding speed of libvpx. The speed improvement was said to be 50-70%, and the code "publicly available as part of libvpx".

Features

VP9 is customized for video resolutions greater than 1080p (such as UHD) and also enables lossless compression. It supports resolutions up to 65536×65536, whereas HEVC supports resolutions up to 8192×4320 pixels.

The VP9 format supports the following color spaces (and corresponding YCbCr to RGB transformation matrices): Rec. 601, Rec. 709, Rec. 2020, SMPTE-170, SMPTE-240, and sRGB.

VP9 supports many transfer functions and supports HDR video with hybrid log–gamma (HLG) or perceptual quantizer (PQ).

Efficiency

An early comparison that took varying encoding speed into account showed x265 to narrowly beat libvpx at the very highest quality (slowest encoding) whereas libvpx was superior at any other encoding speed, by SSIM.

In a subjective quality comparison conducted in 2014 featuring the reference encoders for HEVC (HM 15.0), MPEG-4 AVC/H.264 (JM 18.6), and VP9 (libvpx 1.2.0 with preliminary VP9 support), VP9, like H.264, required about two times the bitrate to reach video quality comparable to HEVC, while with synthetic imagery VP9 was close to HEVC. By contrast, another subjective comparison from 2014 concluded that at higher quality settings HEVC and VP9 were tied at a 40 to 45% bitrate advantage over H.264.

Netflix, after a large test in August 2016, concluded that libvpx was 20% less efficient than x265, but by October the same year also found that tweaking encoding parameters could "reduce or even reverse the gap between VP9 and HEVC". At NAB 2017, Netflix shared that they had switched to the EVE encoder, which according to their studies offered better two-pass rate control and was 8% more efficient than libvpx.

An offline encoder comparison between libvpx, two HEVC encoders and x264 in May 2017 by Jan Ozer of Streaming Media Magazine, with encoding parameters supplied or reviewed by each encoder vendor (Google, MulticoreWare and MainConcept respectively), and using Netflix's VMAF objective metric, concluded that "VP9 and both HEVC codecs produce very similar performance" and "Particularly at lower bitrates, both HEVC codecs and VP9 deliver substantially better performance than H.264".

Performance

An encoding speed versus efficiency comparison of the reference implementation in libvpx, x264 and x265 was made by an FFmpeg developer in September 2015: By SSIM index, libvpx was mostly superior to x264 across the range of comparable encoding speeds, but the main benefit was at the slower end of x264@veryslow (reaching a sweet spot of 30–40% bitrate improvement within twice as slow as this), whereas x265 only became competitive with libvpx around 10 times as slow as x264@veryslow. It was concluded that libvpx and x265 were both capable of the claimed 50% bitrate improvement over H.264, but only at 10–20 times the encoding time of x264. Judged by the objective quality metric VQM in early 2015, the VP9 reference encoder delivered video quality on par with the best HEVC implementations.

A decoder comparison by the same developer showed 10% faster decoding for ffvp9 than ffh264 for same-quality video, or "identical" at same bitrate. It also showed that the implementation can make a difference, concluding that "ffvp9 beats libvpx consistently by 25–50%".

Another decoder comparison indicated 10–40 percent higher CPU load than H.264 (but does not say whether this was with ffvp9 or libvpx), and that on mobile, the Ittiam demo player was about 40 percent faster than the Chrome browser at playing VP9.

Profiles

There are several variants of the VP9 format (known as "coding profiles"), which successively allow more features; profile 0 is the basic variant, requiring the least from a hardware implementation: ; profile 0: color depth: 8 bit/sample, chroma subsampling: 4:2:0 ; profile 1: color depth: 8 bit, chroma subsampling: 4:2:2, 4:2:0, 4:4:4 ; profile 2: color depth: 10–12 bit, chroma subsampling: 4:2:0 ; profile 3: color depth: 10–12 bit, chroma subsampling: 4:2:2, 4:2:0, 4:4:4

Levels

VP9 offers the following 14 levels:

Technology

VP9 is a traditional block-based transform coding format. The bitstream format is relatively simple compared to formats that offer similar bitrate efficiency like HEVC.

VP9 has many design improvements compared to VP8. Its biggest improvement is support for the use of coding units of 64×64 pixels. This is especially useful with high-resolution video. Also, the prediction of motion vectors was improved. In addition to VP8's four modes (average/"DC", "true motion", horizontal, vertical), VP9 supports six oblique directions for linear extrapolation of pixels in intra-frame prediction.

New coding tools also include:

• eighth-pixel precision for motion vectors,
• three different switchable 8-tap subpixel interpolation filters,
• improved selection of reference motion vectors,
• improved coding of offsets of motion vectors to their reference,
• improved entropy coding,
• improved and adapted (to new block sizes) loop filtering,
• the asymmetric discrete sine transform (ADST),
• larger discrete cosine transforms (DCT, 16×16 and 32×32), and
• improved segmentation of frames into areas with specific similarities (e.g. fore-/background)

In order to enable some parallel processing of frames, video frames can be split along coding unit boundaries into up to four rows of 256 to 4096 pixels wide evenly spaced tiles with each tile column coded independently. This is mandatory for video resolutions in excess of 4096 pixels. A tile header contains the tile size in bytes so decoders can skip ahead and decode each tile row in a separate thread. The image is then divided into coding units called superblocks of 64×64 pixels which are adaptively subpartitioned in a quadtree coding structure. They can be subdivided either horizontally or vertically or both; square (sub)units can be subdivided recursively down to 4×4 pixel blocks. Subunits are coded in raster scan order: left to right, top to bottom.

Starting from each key frame, decoders keep 8 frames buffered to be used as reference frames or to be shown later. Transmitted frames signal which buffer to overwrite and can optionally be decoded into one of the buffers without being shown. The encoder can send a minimal frame that just triggers one of the buffers to be displayed ("skip frame"). Each inter frame can reference up to three of the buffered frames for temporal prediction. Up to two of those reference frames can be used in each coding block to calculate a sample data prediction, using spatially displaced (motion compensation) content from a reference frame or an average of content from two reference frames ("compound prediction mode"). The (ideally small) remaining difference (delta encoding) from the computed prediction to the actual image content is transformed using a DCT or ADST (for edge blocks) and quantized.

Something like a b-frame can be coded while preserving the original frame order in the bitstream using a structure named superframes. Hidden alternate reference frames can be packed together with an ordinary inter frame and a skip frame that triggers display of previous hidden altref content from its reference frame buffer right after the accompanying p-frame.

VP9 enables lossless encoding by transmitting at the lowest quantization level (q index 0) an additional 4×4-block encoded Walsh–Hadamard transformed (WHT) residue signal.

In order to be seekable, raw VP9 bitstreams have to be encapsulated in a container format, for example Matroska (.mkv), its derived WebM format (.webm) or the Duck IVF format which is traditionally supported by libvpx, a library that originated at The Duck Corporation. VP9 is identified as V_VP9 in WebM and VP09 in MP4, adhering to respective naming conventions.

Adoption

Adobe Flash, which traditionally used VPx formats up to VP7, was never upgraded to VP8 or VP9, but instead to H.264. Therefore, VP9 penetrated web applications only with the shift from Flash to HTML5 technology, which was still somewhat immature when VP9 was introduced.

Content providers

A main user of VP9 is Google's popular video platform YouTube, which offers VP9 video at all resolutions along with Opus audio in the WebM file format, through DASH streaming.

Another early adopter was Wikipedia (specifically Wikimedia Commons, which hosts multimedia files across Wikipedia's subpages and languages). Wikipedia endorses open and royalty-free multimedia formats. As of 2016, the three accepted video formats are VP9, VP8 and Theora.

Since December 2016, Netflix has used VP9 encoding for their catalog, alongside H.264 and HEVC. As of February 2020, AV1 has been started to be adopted for mobile devices, not unlike how VP9 has started on the platform.

Google TV uses (at least in part) VP9 profile 2 with Widevine DRM.

Stadia used VP9 for video game streaming up to 4k on supported hardware like the Chromecast Ultra, mobile phones as well as web browsers.

Encoding services

A series of cloud encoding services offer VP9 encoding, including Amazon, Bitmovin, Brightcove, castLabs, JW Player, Telestream, and Wowza.

Encoding.com has offered VP9 encoding since Q4 2016, which amounted to a yearly average of 11% popularity for VP9 among its customers that year.

Web middleware

JW Player supports VP9 in its widely used software-as-a-service HTML video player.

Browser support

VP9 is implemented in these web browsers:

• Chromium and Google Chrome (usable by default since version 29 from May and August 2013, respectively)
• Opera (since version 15 from July 2013)
• Firefox (since version 28 from March 2014)
• Microsoft Edge (as of summer 2016)
• Safari (as of Safari Technology Preview Release 110, with official support added in version 14)
• Pale Moon (as of 2018)

Operating system support

Media player software support

VP9 is supported in all major open source media player software, including VLC, MPlayer/MPlayer2/MPV, Kodi, MythTV, and FFplay.

Hardware device support

Android has had VP9 software decoding since version 4.4 "KitKat". For a list of consumer electronics with hardware support, including TVs, smartphones, set top boxes and game consoles, see webmproject.org's list.

Hardware implementations

Hardware accelerated VP9 decoding support nowadays is ubiquitous as most GPUs and SoCs support it natively. Hardware encoding is present in Intel's Kaby Lake processors and above.

Video game consoles

The Sony PlayStation 5 supports capturing 1080p and 2160p footage using VP9 in a WebM container.

Software implementations

The reference implementation from Google is found in the free software programming library libvpx. It has a single-pass and a two-pass encoding mode, but the single-pass mode is considered broken and does not offer effective control over the target bitrate.

Encoding

• libvpx
• SVT-VP9 – Scalable Video Technology for VP9 – open-source encoder by Intel
• Eve – a commercial encoder

Decoding

• libvpx
• ffvp9 (FFmpeg)

FFmpeg's VP9 decoder takes advantage of a corpus of SIMD optimizations shared with other codecs to make it fast. A comparison made by an FFmpeg developer indicated that this was faster than libvpx, and compared to FFmpeg's h.264 decoder, "identical" performance for same-bitrate video, or about 10% faster for same-quality video.

Patent claims

In March 2019, Luxembourg-based Sisvel announced the formation of patent pools for VP9 and AV1. Members of the pools included JVCKenwood, NTT, Orange S.A., Philips, and Toshiba, all of whom were also licensing patents to the MPEG-LA for either the AVC, DASH, or the HEVC patent pools. A list of claimed patents was first published on 10 March 2020. This list contains over 650 patents.

Sisvel's prices are .24 Euros for display devices and .08 Euros for non-display devices using VP9, but would not seek royalties for encoded content. However, their license makes no exemption for software.

According to The WebM Project, Google does not plan to alter their current or upcoming usage plans of VP9 or AV1 even though they are aware of the patent pools, none of the licensors of the patent pools were involved in the development of VP9 or VP8, and third parties cannot be stopped from demanding licensing fees from any technology that is open-source, royalty-free, and/or free-of-charge.

Successor: from VP10 to AV1

On September 12, 2014, Google announced that development on VP10 had begun and that after the release of VP10 they planned to have an 18-month gap between releases of video formats. In August 2015, Google began to publish code for VP10.

However, Google decided to incorporate VP10 into AOMedia Video 1 (AV1). The AV1 codec was developed based on a combination of technologies from VP10, Daala (Xiph/Mozilla) and Thor (Cisco). Accordingly, Google has stated that they will not deploy VP10 internally nor officially release it, making VP9 the last of the VPx-based codecs to be released by Google.

References

References

1. "Supported media formats".
2. Esposito, Filipe. (June 24, 2020). "Apple adds WebP, HDR support, and more to Safari with iOS 14 and macOS Big Sur". 9to5Mac.
3. Peterson, Mike. (June 23, 2020). "iPhones, iPads can now stream 4K YouTube videos in iOS 14". AppleInsider.
4. "HDR Video Playback". Android.
5. Rasmus Larsen. (2016-09-07). "Android TV 7.0 supports Dolby Vision, HDR10 and HLG". flatpanelshd.
6. "VP9 Levels and Decoder Testing".
7. "ivfdec.c - webm/libvpx - Git at Google".
8. "Duck IVF - MultimediaWiki".
9. "Netflix has started streaming to Android in AV1".
10. (15 March 2020). "Google details what you need to play Stadia games in 4K on the web".
11. (2017-03-24). "MPEG-DASH VP9 for VoD and Live - Bitmovin". Bitmovin.
12. "Safari Technology Preview Release Notes". developer.apple.com.
13. (11 June 2010). "HTML5 + &".
14. "Encode and Decode Capabilities for 7th Generation Intel® Core™ Processors and Newer". Intel Corporation.
15. (14 November 2020). "MLB The Show 20 Gameplay Video - 4K HDR 60 FPS on PlayStation 5, Load Times Also Revealed".
16. Ozer, Jan. (2019-03-28). "Sisvel Announces Patent Pools for VP9 and AV1".
17. Cluff, Phil. (2019-03-28). "Did Sisvel just catch AOM with their patents down?".
18. Ozer, Jan. (2019-03-28). "No Content Royalties in Sisvel VP9/AV1 Patent Pools". Information Today Inc.
19. "Frequently Asked Questions".
20. Janko Roettgers (Gigaom), January 2, 2014: [http://gigaom.com/2014/01/02/youtube-4k-streaming-vp9/ YouTube goes 4K, Google signs up long list of hardware partners for VP9 support] {{Webarchive. link. (2020-07-23)
21. Alex Converse (Google), 19 September 2015: [//www.youtube.com/watch?v=gkz1ZvejmEc New video compression techniques under consideration for VP10] – presentation at the VideoLAN Dev Days 2015 in Paris
22. Anja Schmoll-Trautmann (CNET), April 8, 2015: [//www.cnet.de/88148309/ Youtube: Kompression mit Codec VP9 gestartet] (german)
23. "The WebM Project | VP8 Bitstream License".
24. (2016-03-31). "VP9 Bitstream & Decoding Process Specification".
25. (2013-06-07). "Add slightly more colorspace variations". Chromium (web browser).
26. (2014-11-06). "Change the use of a reserved color space entry". Chromium (web browser).
27. (2014). "Applications of Digital Image Processing XXXVII". Proceedings of the SPIE.
28. Iain Richardson, Abharana Bhat, September 5, 2014: [https://www.vcodex.com/news/how-to-stream-better-quality-video-part-3-ultra-high-definition-4k-and-next-generation-video-codecs/ How to stream better quality video: Part 3 – Ultra High Definition, 4K and next generation video codecs] {{Webarchive. link. (2016-07-18)
29. Ronald S. Bultje. (September 28, 2015). "VP9 encoding/decoding performance vs. HEVC/H.264".
30. Jan Ozer, April 2015: [http://www.streamingmedia.com/Articles/Editorial/-103577.aspx The Great UHD Codec Debate: Google's VP9 Vs. HEVC/H.265] {{Webarchive. link. (2016-07-18)
31. (22 February 2014). "The world's fastest VP9 decoder: ffvp9".
32. Jan Ozer, Juni 2016: [http://www.streamingmedia.com/Articles/Editorial/-111550.aspx VP9 Finally Comes of Age, But Is it Right for Everyone?] {{Webarchive. link. (2016-08-11)
33. Paul Wilkins. (2013-05-08). "VP9 Bitstream finalization update". WebM Project.
34. (2014-06-25). "Update on WebM/VP9". Google Developers.
35. (2014-10-03). "Remove experimental-bitstream flag for profiles>0". Chromium (web browser).
36. Romain Bouqueau, July 12, 2016: [http://www.gpac-licensing.com/2016/07/12/vp9-av1-bitstream-format/ A view on VP9 and AV1 part 1: specifications] {{Webarchive. link. (2016-08-06)
37. "VP-Next Overview and Progress Update". WebM Project.
38. Adrian Grange. "Overview of VP-Next". [[Internet Engineering Task Force]].
39. Pieter Kapsenberg. (2013-10-08). "How VP9 works, technical details & diagrams".
40. Max Sharabayko. (2013-10-22). "Next Generation Video Codecs: HEVC, VP9, Daala".
41. (2014). "Digital Video Concepts, Methods, and Metrics".
42. Christopher Montgomery. (2013-08-12). "Introducing Daala part 3: Time/Frequency Resolution Switching". Xiph.Org, Red Hat Inc..
43. (2017-11-28). "WebM Container Guidelines".
44. Jan Ozer, 12. April 2016: [http://www.streamingmedia.com/Articles/Editorial/-110383.aspx A Progress Report: The Alliance for Open Media and the AV1 Codec] {{Webarchive. link. (2021-02-25)
45. ["chrome] Revision 172738". Src.chromium.org.
46. Ed Hewitt (Ohso Ltd.), 21. Februar 2013: [//www.omgchrome.com/google-chrome-hits-25/ Google Chrome hits 25]
47. Volker Zota. (2013-06-18). "Googles Web-Videocodec VP9 auf der Zielgeraden". Heise Newsticker.
48. (2014-03-18). "Firefox Release 28.0". [[Mozilla]].
49. Peter Bright. (2016-04-18). "Windows 10 Anniversary Update: Google's WebM and VP9 codecs coming to Edge". Ars Technica.
50. "android supported media formats".
51. (11 April 2016). "Release Notes – 0.28".
52. "SoCs Supporting VP8/VP9 – wiki".
53. (2014). "Applications of Digital Image Processing XXXVII". Proceedings of the SPIE.
54. BoF meeting on the IETF85 conference in Atlanta, USA with a presentation on VP-Next. [https://www.ietf.org/audio/ietf85/ietf85-grandballroomc-20121105-1735-pm3.mp3 Audio recording] {{Webarchive. link. (2016-03-04 (MP3, ~60 MiB), [http://www.ietf.org/proceedings/85/slides/slides-85-videocodec-4.pdf Präsentationsfolien] {{Webarchive). link. (2013-07-25 (PDF, ~233 kiB))
55. "Next Gen Open Video (NGOV) Requirements". WebM Project.
56. (2013-06-11). "VP9 profile 0 release candidate". [[Chromium (web browser)]].
57. (2013-10-03). "Native VP9 decoder is now in the Git master branch". [[Launchpad (website).
58. Press release from 7 March 2013: [http://www.businesswire.com/news/home/20130307006192/en/Google-MPEG-LA-Announce-Agreement-Covering-VP8 Google and MPEG LA Announce Agreement Covering VP8 Video Format] {{Webarchive. link. (2022-01-24)
59. Thomas Catan. (2011-03-04). "Web Video Rivalry Sparks U.S. Probe". Dow Jones & Company, Inc..
60. Cheng, Jacqui. (2011-03-04). "Report: DoJ looking into possible anti-WebM moves by MPEG LA". Condé Nast Digital.
61. (2014-01-07). "Ittiam and ARM are the first to efficiently bring Google's VP9 to mobile devices". ARM Community.
62. (2014-01-07). "Ittiam's H.265 and VP9 Solutions to Have Widespread Coverage at CES 2014". ARM Community.
63. (January 2015). "NVIDIA Tegra® X1". [[Nvidia.
64. Joshua Ho, Ryan Smith (AnandTech), January 5, 2015: [http://www.anandtech.com/show/8811/nvidia-tegra-x1-preview NVIDIA Tegra X1 Preview & Architecture Analysis] {{Webarchive. link. (2015-01-05)
65. (2015-03-02). "VeriSilicon Introduces Hantro G2v2 Multi-format Decoder IP with VP9 Profile 2 to Support 10-bit Premium Internet Content". [[Business Wire]].
66. Michael Larabel. (2015-04-03). "libvpx 1.4.0 Brings Faster VP9 Encode/Decode". [[Phoronix]].
67. Jan Ozer. (May 24, 2016). "Netflix Discusses VP9-Related Development Efforts". streamingmedia.com.
68. (12 January 2016). "A High Performance, OpenCL-Based VP9 Encoder". phoronix.com.
69. Stephen Shankland. (September 12, 2014). "Google's Web-video ambitions bump into hard reality". [[CNET]].
70. Michael Larabel (Phoronix.com), 17. August 2015: [//phoronix.com/scan.php?page=news_item&px=Libvpx-VP10-Starts Google Starts Pushing Out VP10 Open-Source Code Into Libvpx]
71. (2016-04-05). "The Alliance for Open Media Welcomes New Members and Announces Availability of Open Source Video Codec Project". Alliance for Open Media.
72. Jan Ozer. (2016-04-12). "A Progress Report: The Alliance for Open Media and the AV1 Codec". StreamingMedia.com.
73. Jan Ozer. (2016-05-15). "What is VP9". StreamingMedia.com.
74. "Commons:Video".
75. "Help:Converting video".
76. (2017-03-31). "Ittiam accelerates open source VP9 encoder in partnership with Netflix and Google".
77. (2016-12-09). "[Updated – It will soon]NVIDIA SHIELD Android TV Does Not Support Google's 4K Content".
78. (2016-10-11). "Widevine Quarterly Partner Update Q3 2016".
79. Zimmerman, Steven. (15 May 2017). "Google's Royalty-Free Answer to HEVC: A Look at AV1 and the Future of Video Codecs". XDA Developers.
80. "Key benefits of Widevine's DRM solution".
81. (2016-08-31). "Encoding.com releases support for VP9".
82. (2017-03-09). "HLS still "industry standard" says encoding.com report".
83. "HEVC: Rating the contenders". Streaming Learning Center.
84. (2017-03-22). "The State of Codecs 2017". streamingmedia.com.
85. (5 May 2017). "NAB 17 Codec Roundup".
86. (17 February 2019). "SVT-VP9 Is Intel's Latest Open-Source Video Encoder Yielding High Performance VP9".
87. (10 March 2020). "Streaming video could be saddled with a new patent licensing cost".