This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new!

Feature: Graphics & Multimedia

Sun's OMS Video codec project is a means to an end

By Nathan Willis on August 22, 2008 (9:00:00 PM)

Share    Print    Comments   

Sun Microsystems is setting out to create an open source, royalty-free video codec. Given the considerable head start of well-known, royalty-free video codecs like Dirac and Theora, you might ask why the world needs another. The answer, according to Sun, is the process the company will use to develop it -- starting with a full-on, careful examination of the patent situation.

I spoke with Gerard Fernando and Rob Glidden about the project, which was unveiled in April. Fernando is a senior staff engineer at Sun, and Glidden is the company's global alliance manager for TV and media.

Fernando says the video codec idea dates back to 2005, when the Open Media Commons (OMC) initiative was launched by Sun CEO Jonathan Schwartz. It just took a back seat to OMC's first large-scale project, the DReaM digital rights management (DRM) system.

DRM rubs many in the open source and free software communities the wrong way, Glidden admits. But for DRM detractors, he says, the important thing about DReaM is the process that OMC used to develop it. Sun spent a full year conducting a thorough analysis of the patents and associated claims in DRM, and built DReaM on top of security systems that were either unpatented or whose patents had expired. You can read the fruits of Sun's patent examination for yourself on the OMC Web site. Regardless of how you feel about DRM, in a field dominated by patent-holding commercial interests and standards bodies that promote member products to collect royalty checks, DReaM is both open source and -- more importantly -- composed entirely of patent-unencumbered, royalty-free technology.

Submarine patents, industry FUD

By systematically examining the patent landscape, Glidden says, OMC can document the intellectual property rights asserted by other players, including valid patents to be worked around, and bad patents that need to be challenged. That strikes at the heart of one of the chief criticisms of codecs created through an open source development model, the so-called "submarine patent threat." Glidden calls that threat FUD, a scare tactic used by those collecting royalties on their own codecs to discourage the development of others.

"It's a call to inaction," he says. "When you are surrounded by a fleet of battleships all taking aim at you, the possibility of a submarine is not really a problem." The battleships in question are the patent-holding participants in royalty-charging standards groups like the ISO and the ITU. MPEG-LA exists solely for the purpose of selling licenses to MPEG codec patents, Glidden says, so of course it is going to spread the notion that competing technologies are fraught with risk and unspecified infringements. That is why OMC's development process begins by proactively challenging the allegation.

The "how can you be sure you don't infringe on someone else's patent?" question has a straightforward answer, Glidden says. You research the field, taking a conservative approach, and build up a taxonomy of the relevant work. "The fact is that these standards have known patent pools. They are listed in the documents, on the Web sites, even in the press releases."

H.261 baseline

OMC has already begun its patent evaluation for OMS Video, and has preliminary ideas sketched out for the structure of the codec. You may have heard it reported that OMS Video will be based on the H.261 codec, whose first wave of patents has already expired. Critics object that H.261 is irrelevant because of its age, arguing either that OMS Video will be shackled to archaic technologies, or that OMS Video will contain so little of H.261 that its expired patents make no difference.

That OMS Video will be based on H.261 is true only in the most basic sense; as Glidden explains it, H.261 is the starting point for the OMS codec because it is the grandparent of all discrete cosine transform (DCT)-based codecs -- a family that includes the majority of the patent-encumbered codecs popular today. But by citing H.261 as the basis for the new codec, OMC makes a statement about its lineage: demonstrably unencumbered by patents.

Glidden and Fernando both insist that "based on H.261" is oversimplifying too much anyway. Fernando explores the structure of the codec in a PDF presentation that you can download for free. As he puts it, what we often refer to as the "codec" is actually better understood as a suite of tools performing different functions -- spatial prediction, motion region coding, arithmetic coding, and so on. A patent holder might make a claim on one or more of those specific functions, not on the codec as a whole.

That distinction is important both for tracking the patent issues and for understanding the amount of new work that remains for the OMS Video project. If you search the ITU's Web site, you will find seven specific patents listed as parts of H.261. The block diagram of the OMS Video codec in Fernando's presentation (which is also used in an OMC blog post about the project) references just two components from the original H.261: arithmetic coding and 1/4 pixel motion estimation.

Taken as a whole, the few expired patents implemented by H.261 will not constitute much of the final OMS Video. But the H.261 heritage will make it harder for opponents to argue that the new codec infringes on more recent codecs, since those recent codecs are derived from the demonstrably historic H.261 as well.

Free perspective

Christopher Montgomery of The Xiph Foundation is well-versed in codec creation, and no stranger to patent infringement FUD either. He agrees with Sun's basic claim that the vast majority of modern video codecs share the same mathematical lineage, but is not persuaded by its arguments regarding the value of basing new work on H.261. "If you're limited to an executive summary flowchart of no more than 10 boxes that fit on one page, you'd be hard-pressed to tell the differences between any of these codecs. The differences are all in the details (of course, the details are very important).

"The claim that building on top of H.261 frees you from patent concerns and that's a reason to use H.261 over Theora is dubious, as Theora is based on a codec of approximately the same age. The truth is that as soon as you start trying to move forward (using any of the discoveries published since then) you immediately run into patent concerns again."

Montgomery also referred me to a blog post he wrote in response to the HTML 5 Video element debate. There he recounts his personal experience with years of patent FUD aimed at the Xiph codecs, Vorbis and Theora, noting "part of the problem is that there is no legal test for 'unencumbered.' Nor do you have to be infringing to get sued. We will always have a halting problem here. And yet, both Vorbis and Theora are 10 years old with no claims against them. Would another 10 be sufficient? Twenty? Vague threats that come to nothing (nothing that is except eliminating the monopoly interest's competition) are the story of my professional life. Patent law is being used to extort outright, and most of the interests represented here are willingly buying into it. This isn't even getting to the trolls. And we sit around and wonder why this industry is so dysfunctional."

Standards and beliefs

Glidden also points to the W3C's withdrawn support for Vorbis and Theora in HTML 5 as evidence that the standards bodies -- like the ITU, ISO, and MPEG-LA -- are falling victim to FUD from vendors of competing codecs. When participants in the process stand to profit through patent royalties, he concludes, the process cannot be trusted to make objective technical decisions. It is lamentable to see in the W3C, he says, because it had long enjoyed a reputation for fair and pragmatic decision-making. By caving to FUD over codec patents, it is acting just like everyone else.

Sun's work on OMS Video is an attempt to challenge the patent-crazed belief system that dominates the codec industry, Glidden says. "We don't accept the stock answers on FUD, such as 'How can you ever be sure you aren't violating a patent?' You can be sure by doing a taxonomy; you dig into the process and ask questions."

So what impact will Sun's work have on the prevailing attitudes towards software patents? Only time will tell. Glidden and Fernando could not predict when the OMS Video patent audit would produce its first fruits or when a general plan would be available for the project.

I see evidence of the FUD campaign that Glidden describes whenever I read the discussion threads attached to news story about codecs: Fear that developers (or even users) will get sued for patent infringement. Uncertainty about what patents exist and whether it is even possible to know. Doubt that new ideas or patent-unencumbered ideas could ever match the patented, for-profit products.

Regardless of whether OMS Video bests H.264 technically or ever makes a dent in the MPEG-LA's market share, this initiative from Sun is important to watch because the development process is a counterattack against the it's-not-worth-trying mindset. That mindset is costing consumers millions of dollars a year in licensing fees, and it has a chilling effect on new work.

Share    Print    Comments   


on Sun's OMS Video codec project is a means to an end

Note: Comments are owned by the poster. We are not responsible for their content.

A comment on the patent FUD around Theora

Posted by: Anonymous [ip:] on August 23, 2008 12:01 PM
Sun's effort on making sure not to use patent encumbered technology in their new codec by publishing all the patent research they are doing around the technology development is very much to applaud. This has also been done for Vorbis. For Theora, it is assumed that the research was done by the doners of the Theora technology On2Technologies, but this research was not published. It seems Mozilla is out to rectify that situation: states that they have investigated the legal situation around Ogg Theora before accepting it into the new Firefox as the default supported video codec. We all hope to see that research published soon.


Next is Flash

Posted by: Anonymous [ip:] on August 23, 2008 03:02 PM
I heard firefox is working to embed flash in the browser so it will work on any OS.

What is next for video?



Re: Next is Flash

Posted by: Anonymous [ip:] on August 24, 2008 08:21 AM
I haven't heard anything about that, but they are implementing native Ogg Vorbis and Theora playback in Firefox, on all platforms:


Sun's OMS Video codec project is a means to an end

Posted by: Anonymous [ip:] on August 25, 2008 03:02 PM
Ah, nice article, thanks!

However, your first sentence in the last paragraph is crucial. Existing codecs like Theora and Dirac are outperformed by standards like H.264 as of today, and I still want to see a codec which is technologically state of the art. Sun should focus on that.

On the other hand, good old JPEG is so 80s now (ok, 90s), but it is the de facto standard in modern browsers still. Maybe a basic but working video codec will be enough such that people will stick to it.


Re: Sun's OMS Video codec project is a means to an end

Posted by: Nathan Willis on August 25, 2008 04:01 PM
Sun should focus on that.

They are. Why did it sound like they weren't?

On a related note, I'd like to take a shot at the term "outperformed" -- it is broad and vague, and stating that one codec "outperforms" another codec omits multiple levels of important details that ought to be mandatory in any head-to-head comparison: size and framerate of source material, bitrate, processing power required to encode and processing power to decode, to name a few. "Outperform" without examining technical details is almost entirely subjective. H.264 certainly doesn't outperform H.261 on a PII -- H.264 may be capable of handling HD far better under appropriate conditions, but it demands far more horsepower to even run.

Please note that I'm not mentioning this in response to your comment directly; just introducing it as a new topic triggered by use of the term. I'm not suggesting you were sweeping detail under the rug by any means.



Re: Sun's OMS Video codec project is a means to an end

Posted by: Anonymous [ip:] on September 15, 2008 05:00 PM
Actually, Dirac can outperform H.264 at similar bitrates


Just make a MPEG-1 Royalty Free Profile.

Posted by: Anonymous [ip:] on August 26, 2008 01:38 PM
How about just make a royalty free profile of MPEG-1? Take out MPEG-1 Audio layer 3, and maybe a few other minor features of of MPEG-1, and you probably can make a video codec that Quicktime and Microsoft Media Player can already decode, and can be implemented royalty free. MPEG-1 Video is pretty close to H.261 anyway.


This story has been archived. Comments can no longer be posted.

Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya