Decoding the Landscape: Recent Developments in Video Codec Licensing
As recently as two or three years ago, you had to be a compression geek to choose and deploy a new codec for publishing your content. Today, in mid-2024, you need to be part patent attorney, part CFO, and it helps if you know a bit about compression.
The most significant changes relate to the potential for codec-related patent royalties on encoded content. Forces moving in this direction include the Avanci Video Pool, which is seeking content royalties on a range of codecs, along with two court cases—one adjudicated, and one not. I’ll start by reviewing where content royalties are and where they will likely go.
Then, I’ll discuss new European Union regulations concerning how Standard Essential Patents (SEPs) are licensed in general. While these have no direct bearing on US law, if a butterfly flaps its wings in Europe, EU regulations could impact the US, a concern that some commenters have mentioned. We'll conclude with a look at the Sisvel and RPX licensing agreement, which takes us one step closer to the seemingly ageless question of whether AV1 is truly royalty-free.
About Patent Pools
Let’s quickly review what patent pools are and the value they deliver. Patent pools are collections of patent owners related to a particular technology run by an administrator like Access Advance, Sisvel, or VIA-LA. Patent pools serve as convenient one-stop shops for companies wishing to license the underlying technology for inclusion in a product or service (called implementors). If pools didn’t exist, these implementors would need an agreement with each patent owner. Since there are often dozens or even hundreds relating to a particular technology, obtaining these license agreements would be costly and time-consuming.
You see this in Figure 1. On the left, without a pool, the four patent owners must sign 16 license agreements with the four implementors on the right, totaling 16 agreements, plus agreements between the patent owners if they are also implementors. On the right, once the pool agreement is signed, the four additional implementors need to sign one agreement each.
Figure 1. The contractual efficiencies delivered by patent pools. Image from Copperpod.
Many technologies have one or two patent pools, like VIA-LA and Access Advance for HEVC and VVC. However, not all patent owners join patent pools, and those who don’t can monetize and assert their patents as desired. Most technology standards have large patent owners—like Qualcomm in the cellular space—who don’t join a pool and license directly with large implementors.
The key point to remember is that patent pools represent only one source of patent-related claims. The other can come from the non-affiliated patent owners.
Content Licensing Practices of the Patent Pools
Now let’s review the history of codec-related patent royalties on encoded content from the various pools. At the very start, MPEG-LA was formed around a pool for MPEG-2, which received the seminal Department of Justice approval letter (go2sm.com/doj) stating that this pool didn’t violate US Antitrust laws. The pool was formed in 1997, while the Internet and streaming video were in their infancy. The only content giving rise to royalty payments were DVDs encoded with MPEG-2.
Regarding Internet content, the MPEG-LA H.264 pool (now VIA-LA) charges up to $100,000 for subscription revenue (for 1 Million + subscribers) and the lower of (a) 2% of the remuneration paid in the licensee’s first arm’s-length sale or (b) $ 0.02 per title for all H.264-encoded content longer than 12-minutes or less. There is no royalty for free internet video (go2sm.com/free), though that was on the table until 2010. Interestingly, MPEG-LA decided to forgo royalties on free internet content in perpetuity three months after Google open-sourced VP8—though, as my physician wife likes to say, correlation doesn’t mean causation.
In September 2014, the MPEG-LA HEVC pool launched without any content royalty to “make the terms as simple as possible in a ‘modern streamlined pool licensing approach.’” HEVC Advance (now Access Advanced) launched in March 2015 with a .05% royalty on attributable revenue and no cap, which triggered a hate campaign against HEVC (and later VVC), which hasn’t totally subsided.
In September 2015, the Alliance for Open Media launched with the supposedly royalty-free AV1 codec. In another correlation-is-not-causation moment, in December 2015, HEVC Advance eliminated the .05% royalty but imposed a royalty on HEVC-encoded video delivered via subscription and pay-per-view and digital media containing HEVC-encoded content. It also instituted a $40 million overall cap and a separate $5 million cap on content. In 2018, HEVC Advance dropped the royalty on subscription and pay-per-view, leaving only the royalty on physical content.
When MPEG-LA (now VIA-LA) and HEVC Advance (now Access Advance) launched their VVC pools, they mimicked their HEVC content policies; MPEG-LA charged no content-related royalties, while HEVC Advance charged only for VVC-encoded video delivered on physical media. That’s where things stood from a patent pool perspective until Avanci launched its Video pool in October 2023.
The Avanci Video Pool
By way of background, unlike some pools offering licenses for a wide range of uses for a single patent (like a decoder patent usable in various devices), Avanci creates pools of patents essential for a particular industry application. For example, while there are other general-purpose pools for the 5G cellular standard, the Avanci pool is for all cellular and other patents used specifically in connected vehicles. Their Internet of Things pools are for smart meters and EV chargers, not general IoT products. This focus means that Avanci pools almost always include patent owners who are also members of other pools.
Judging from its vehicle-related pools, Avanci is certainly credible. For example, the Avanci 5G Vehicle program has 21 licensees including Audi, Bentley, BMW, Rolls-Royce, Ford, GM, Volkswagen, Mercedes Benz, Porsche, Kia, Hyundai, and others. The pool page also lists the price, which is $32/vehicle with discounts available for early licensees.
As to the Avanci Video pool, you can glean its focus from a single FAQ entry which reads:
Who should obtain a license from Avanci Video—“Internet video streaming companies that are providing Internet video streaming services to its users using any of the five standards – H.265 (HEVC), H.266 (VVC), VP9, AV1, and MPEG-DASH—have the option to take a license from Avanci Video. Our licensing program is open to all internet video streaming companies, including subscription-based entertainment services, ad-based video sharing sites, social media and video messaging platforms, and video conferencing providers.”
You see the codecs covered in the pool, which excludes H.264, likely because the 20-year patent protection on many of the related patents is starting to expire. The FAQ identifies the 26 patent owners in the platform who own various patents in each listed technology, most of whom are included in other pools.
The FAQ does not list the royalty rate, and doesn’t list any licensees, perhaps confirming what I heard from several sources: that Avanci either doesn’t have any licensees or only has a couple of small ones. I contacted Avanci and asked to speak to a representative on this and other issues, but never heard back.
The Response from Other Pools
By virtue of the codecs involved, Avanci poached patent owners from VIA-LA, Access Advance, and Sisvel for its Video pool. Since none of these other pools currently charge for content, patent owners who wish to seek royalties on content seemingly have no incentive not to join Avanci. If you're Sisvel, Access Advance, or VIA-LA, the natural way to avoid this is to start charging royalties on content.
To explain, the fact that the Sisvel, VIA-LA, and Access Advance pools don't charge for content is strictly the decision of the patent owners, who call the shots. Patent pool administrators can advise, but the patent owners decide. As we saw with HEVC and HEVC Advance, part of the equation is maximizing revenue by doing what it takes to promote the adoption of the standard. Clearly, back in 2015, patent owners in the HEVC pool thought it best to forgo royalties on content to help drive adoption.
In 2024, HEVC will be available in virtually all Smart TVs and about 96% of mobile devices. It’s safe to say that adoption has occurred. Premium content producers must use HEVC to deliver 4K and HDR. Even AV1 doesn’t avoid the royalty obligation since it’s also covered by the Avanci pool. Though we’re not there yet, at some point in the near term, premium content owners may either decide to pay a codec-royalty on content or shut down.
This being the case, if you’re a codec patent owner and your job is to maximize revenue, why not charge a royalty on content? Some owners might object on philosophical grounds like broadcasters or other publishers who feel content royalties are inappropriate. Others may feel it unwise to charge content owners for codecs since it might poison the water for other technologies marketed to these companies, like VVC or MV-HEVC, or proprietary products or services.
Many others, however, will want to maximize their patent-related investment and claim a share of content royalties, whether by joining Avanci or convincing their existing pool-mates to start charging for royalties, in the same or a sister pool for content only. If you’re a pool administrator, you can watch them join Avanci, a competitor, or create your own offering.
I reached out to all three pool administrators on this issue. Heath Hogland, President of VIA-LA, responded. “We are exploring a content licensing program and are receiving positive market interest from licensors and licensees,” Hogland said. “We believe Via LA is well-positioned to find a balanced ecosystem solution for the market.”
The other two had no comment. Reading the tea leaves (100-foot-high flashing marquee), it seems likely that these pools will start charging for content at some point in the near term, either in the existing pool or a new pool.
Other Patent Owners and Content Royalties
As mentioned above, patent pools aren't the only potential source of claims for codec-related patent royalties. There is an adjudicated case in Germany—Broadcom vs. Netflix—where the court found Netflix “to be infringing on European patent EP 2 575 366, due to its transmission of HEVC video, with which the company delivers Ultra HD content.” A Munich court ordered Netflix to stop streaming HEVC-encoded video, but Netflix refused, amassing over $7 million in fines and the threat of imprisonment for executives or directors for the company’s “defiant” behavior. Netflix has the right to appeal, so the suit is not final, but as it stands, delivering video encoded to a particular codec standard has been ruled by a German court to infringe upon that standard.
Figure 2. The headline says it all.
Because the pleadings are in German, we don’t know what Broadcom claimed for infringement-related damages, and whether that was for Germany or worldwide. So, we don’t have a sense of what damages are claimed, and they haven't yet been awarded.
That's Germany. In the US, in October 2023, Nokia sued Amazon and HP in five jurisdictions (US, Germany, India, UK, and European Unified Patent Court (UPC) for “infringement of Nokia’s Asserted Patents ... through Amazon’s video streaming products and services, including Prime Video, Amazon.com videos, and Twitch.tv.”
According to the Complaint, these claims relate to video “encoded into formats compliant with the H.264 Advanced Video Coding Standard (“H.264”) and the H.265 High Efficiency Video Coding Standard (“H.265”)....Amazon’s unlicensed streaming products and services, including without limitation Amazon streaming videos, movies, shows, trailers, and advertising, such as those on Amazon Prime Video (including Freevee), Amazon.com, and Twitch.tv, (“Accused Products”), infringe Nokia’s Asserted Patents.”
As of this writing, Amazon hasn’t filed its answer. Until this or a similar case is decided in the US, we won’t definitively know that encoded content gives rise to royalties, and while the court would certainly consider Broadcom vs. Netflix, it’s not controlling. And given the sweeping changes coming to European patent licensing, it’s hard to tell how relevant this decision will prove in even the short term.
The EU’s SEP Regulation
In February 2024, the European Parliament voted overwhelmingly to approve major changes to the EU's standard-essential patent (SEP) licensing landscape. The European Commission’s proposal, first introduced in April 2023, has four key elements:
- A centralized SEP registry at the EU Intellectual Property Office (EUIPO) to catalog all SEPs.
- Mandatory (but non-binding in a subsequent court case) essentiality checks by the EUIPO to eliminate “false” SEPs.
- A process for determining aggregate royalty rates for SEP portfolios.
- A framework for FRAND (fair, reasonable, and non-discriminatory) licensing determinations by the EUIPO (again, non-binding).
At a high level, the regulation takes much of the work previously done by the courts and assigns it to an agency with little relevant experience and clearly inadequate staffing. Not surprisingly, the regulations have drawn significant criticism from various quarters, mostly from patent owners.
Figure 3. Sisvel lobbying against the EU SEP regulations
A January 22 Politico article claims that “the initiative is imbalanced and one-sided as it completely ignores the perspective of patent owners and is apparently motivated by the surprising aim of redistributing value from patent owners to implementers...[T]he immediate effect of such an approach would be to devalue European patents, endangering future investments in innovation and even the survival of some of the European leading innovators in key areas such as advance mobile communication."
The “Comments” article referenced earlier argues that “Crucially, other countries may be inspired by the Commission’s SEP Regulation and decide to adopt similar regulatory regimes. Regulations implemented by other countries might easily backfire and be used for protectionist purposes and as a strategic tool to devalue the royalties of innovative European SEP owners. The primary beneficiaries of the Regulation might be non-EU based implementers, to the detriment of European innovators and Europe’s technological leadership.”
Sisvel argues that the new regulations will do the following:
- “Lead to a wealth transfer from low resource SMEs (small/medium-sized enterprises) to deep pocket big corporations and from Europe to the rest of the world.
- Present a threat to European leadership in building the global connectivity market and to European technology security.
- Prove a disincentive to future investments in and collaboration on developing connectivity technology, resulting in job losses.”
Despite widespread opposition, the European Parliament voted by a large majority (454–83 with 78 abstentions) on February 28, 2024, to adopt the European Commission’s legislative proposal. Now, the final version of the legislation will be hammered out by the European Commission, the Council of the European Union (representing member states), and the European Parliament.
Of all the provisions of the regulations, the one that seemed to have merit was a “process for determining aggregate royalty rates for SEP portfolios.” Certainly, large content publishers, now facing royalties from multiple directions, would agree.
I asked Sisvel about this and heard back from Vincent Angwenyi, Senior IP Counsel, who replied, “The question is how to go about determining the maximum (aggregate) FRAND value in the first place and how to appropriately attribute it to the relevant patents. In fact, no court has been able so far to successfully use the top-down approach. It is, therefore, questionable how the SEP regulation will be able to pull it off. “
The complete answer is much longer (see the end of this article). But essentially, it boils down to “the courts haven’t been able to do it, so how can we expect an unproven agency to do it?” After reviewing all the factors involved in Angwenyi’s longer answer, it’s hard to disagree.
As these new regulations take effect, the landscape for patent licensing and enforcement within the EU is poised for significant change, affecting patent owners and codec users over the next few years. This will likely disrupt “business as usual” and could reshape industry practices. While the direct impact on the US and other jurisdictions may be less immediate, these changes will likely influence worldwide policies, whether through emerging case law, adjustments in litigation or pricing strategies, or even calls for similar legislative measures. For those with global patent practices, the new EU regulations will almost certainly change how intellectual property rights are managed internationally.
Sisvel and RPX
One of the primary allures of the AV1 codec was its “royalty-free” nature, a status which was always in question for many observers (go2sm.com/av1arrives) and came to a practical end when Sisvel announced its AV1 pool in 2019. If you’re a manufacturer of a covered consumer device that incorporates AV1, you’re free to happily continue to call it royalty-free, though your CFO will likely insist on reserving funds for the possible charges, and your chief counsel might look at you askance in the lunchroom.
Of course, if you’re a streaming publisher through 2023, all of this was academic. The Sisvel pool doesn’t impose content royalties, and content royalties weren't on the radar screen.
Well, the picture has changed. Now you have two pools claiming royalties on AV1, with the Sisvel pool including dozens of licensees. Some of these come from a significant licensing agreement with RPX, a “defensive patent aggregation service” that buys licenses to different patents to make them available to its customers, which according to its FAQ includes more than 450 clients.
The Sisvel press release announcing the agreement was sparse, stating, “RPX and Sisvel, both aggregators, represented a group of 16 technology companies and 20 patent owners, respectively, to conclude a transaction related to video codecs. The agreement provides participants with a license to patents in Sisvel’s Video Coding Licensing Platform and resulted in significant transactional efficiencies.”
I asked attorney Robert J.L. Moore about his thoughts on the deal, and he replied, “At the very least, this RPX-Sisvel deal represents a determination by 16 companies that either use or plan to use VP9 and/or AV1 technology that they need to clear the patent risk presented by Sisvel's licensors. This casts doubt on claims that VP9 and AV1 are open source.” (See my full interview with Robert at go2sm.com/moore.)
Digging a bit deeper into RPX, I noticed that Google was listed as a customer in RPX’s S1 Registration Statement and more recently identified as an RPX customer in this third-party blog post. I asked Sisvel if Google was party to the agreement, but they declined to comment, citing tight NDAs around the deal.
Does Google taking a license to AV1 sound shocking? Actually, it sounds reminiscent. As Moore succinctly recounted on LinkedIn last year, “Google called VP9 royalty-free, but Google itself took a license to the MPEG LA AVC…patent pool on behalf of VP9 implementers, acknowledging eventually that VP9, like all commercially-viable codecs, used technology developed earlier and elsewhere.”
In the hope of resolving Google’s involvement, I sent a note via LinkedIn to an attorney high up in Google’s Patents group asking for comments and haven't heard back. I sent another note to a former contact on the AV1 team who is still at Google and I haven’t heard back from him either. If anyone from Google wishes to clarify whether Google was a party to the Sisvel/RPX deal, please contact me at janozer@gmail.com, and we’ll update the online version of this article.
La Cuenta, Por Favor
It seems likely that many content publishers will be receiving demand letters from multiple sources over the next few years, and that streaming AV1 won't mitigate this. What’s the final bill going to add up to? I asked Moore about this, and he responded, “It’s hard to say at this point. I think the metrics that will be used for the content side will be radically different than per unit. You won’t be able to charge a publisher every time an app is instantiated. You’ll probably charge on the basis of things like their revenue or their monthly active users or something like that.”
The bottom line is that no one knows.
Author’s note: I worked with Sisvel as a consultant from 2019 through August 2022. Thanks to Robert JL Moore for discussing many of the issues in this article; his contribution extends far beyond the attributed quotes (but all errors and opinions are mine).
Vincent Angwenyi, Senior IP Counsel - complete answer
- The question is how to go about determining the maximum (aggregate) FRAND value in the first place and how to appropriately attribute it to the relevant patents. In fact, no court has been able so far to successfully use the top-down approach. It is therefore questionable how the SEP regulation will be able to pull it off. The proposed framework is far from convincing. Such a process in essence would require taking into consideration quite a number of factors and variables that make it practically impossible to have a one-size-fits-all approach. This could include:
- Determining all the patents that read on a standard.
- Taking into consideration that specific use-cases or products may only implement specific parts of a standard.
- Finding out all the relevant patents reading on the specific implementation of the standard and determining separate aggregate royalties for these separate use-cases.
- Keeping track of the ever-evolving market dynamics and the introduction of new use-cases to ensure that the aggregate royalties reflect the market.
- Predicting aggregate royalties where the market for products or use-cases is not yet established and there are no clear parameters for establishing value.
- Taking into account that standards often include optional parts meaning that SEPs reading on these parts are also optional.
- There is also the concern that a top-down approach usually devolves into a patent counting exercise that assigns an arbitrary value to the patents reading on a standard, thus grossly undervaluing or overvaluing certain patents.
- The proposed framework also appears to give free reign to any number of SEP holders to notify aggregate royalties. This is having in mind that there is no one-size-fits-all methodology for determining aggregate royalties. The system would essentially be populated with meaningless information.
- Looking specifically at the proposed regulation, it is not clear how it will deal with the fact that determining a global (aggregate) FRAND value would require assigning value to patents that are outside of the jurisdiction of the proposed framework.
Related Articles
All-in-One (AIO) live production devices leverage the latest mobile CPU and GPU power and integrated capability, with real hardware connections for multiple HDMI ports, ethernet, headphones, audio input, and more—all from one manufacturer so there's just one update cycle to track. Magewell recently came out with the 3.0 update to their Director Mini AIO production tablet (Figure 1), and they are pushing the envelope with what these little powerhouses can do.
05 Nov 2024
While VMAF has been a valuable tool for assessing video quality, its limitations highlight the need for complementary metrics or the development of new assessment methods. These new metrics should address VMAF's shortcomings, especially in the context of modern video encoding scenarios that incorporate AI and other advanced techniques. For the best outcomes, a combination of VMAF and other metrics, both full-reference and no-reference, might be necessary to achieve a comprehensive and accurate assessment of video quality in various applications.
05 Nov 2024
In this wide-ranging interview, Streaming Media's Jan Ozer and intellectual property attorney Robert Moore of Moore IP solutions delve into a number of topics related to video codec intellectual property issues and patent monetisation, two constantly moving targets.
07 May 2024
To promote the understanding of AI codec development, Deep Render's CTO and Co-founder, Arsalan Zafar, has launched an educational resource titled "Foundations of AI-Based Codec Development: An Introductory Course." In a recent interview with Streaming Media Magazine, Zafar provided insights into the course's content, target audience, and expected outcomes.
30 Apr 2024
What is the new Avanci Video codec patent pool, who is involved with it, what codecs does it cover, and what does it mean for OTTs and other streaming content companies in terms of paying for codec usage going forward? Should the streaming industry be bracing for a sea change when it comes to codec royalties? In this interview with Streaming Learning Center's Jan Ozer, Volpe Koenig IP attorney discusses these implications and more based on what we know so far.
24 Oct 2023