Fun fact: in the interests of forward compatibility, Firefox appears to have used liboggplay for its video pipeline.
This means that if they ever want to use anything other than Theora, they will have to completely rewrite the entire video system. Apparently the problem with doing something sane like using Gstreamer is that... it's LGPL. Which apparently isn't free enough for them.
They also force YV12 -> RGB conversion in software on playback and do two sizeof(videoframe) callocs per frame, leading to the tremendous irony of having video playback vastly slower than Flash.
Apparently the problem with doing something sane like using Gstreamer is that... it's LGPL. Which apparently isn't free enough for them.
This is not correct. The reason GStreamer is not used in Firefox is because the GStreamer backend wasn't finished in time for FF 3.5. Rather than push back the release date, they released an implementation that works for many videos. The GStreamer backend is still under active development, and will eventually be merged to trunk.
leading to the tremendous irony of having video playback vastly slower than Flash.
Using the most recent version of Flash and Firefox, I can play high-definition Theora easily on my laptop, but even medium-resolution H.264 in Flash struggles to achieve real-time decoding. A quick, unscientific sampling of other FF 3.5 users online has indicated to me that many others experience similar performance in both cases.
I don't think you're correct about the gstreamer backend.
Some Mozilla-based projects use Gstreamer (Songbird for one), and I believe they're having discussions about a future collaboration for accessing user's video cameras in the browser but Mozilla don't want to support any old codecs.
Their mission as a non-profit is to support the open web, and promoting royalty-free codecs fits with that mission. They can make a difference by actively choosing not to do stuff, just like at one point in the past they refused to support ActiveX despite having code available. It would have made Firefox a more useful browser at the cost of making the web less open.
I'm not sure how that article turned into a treatise on how mean Microsoft is, but I stopped reading after the first third of it. Someone let me know if it picked up after that. :)
Some more sophisticated analysis from the author in the comment thread:
"I did read the Ars article, which similarly voices the "free codec is good, international standards body specifications are evil" thing everyone has been repeating.
The bottom line is that x264 is GPL so the whole "Linux users can't use H.264 video" is simply wrong, and in the big picture Mozilla and Opera don't really matter. Once Chrome starts gaining traction, do you really think Google will see any need to dump $50 million on the remains of Netscape's ashes? And what purpose does Opera serve other than to make the EU feel like it has some say in the browser market?"
Sounds like someone pissed in his cheerios. Bloody fanboys.
It isn't about the implementation's license, it's about patents. H.264 is patented while Theora is not. This makes it illegal for anybody to distribute H.264 decoding software, whether open source or not, until they've paid for the patent license.
FooBarWidget: "This makes it illegal for anybody to distribute H.264 decoding software, whether open source or not, until they've paid for the patent license."
Only within the jurisdictions in which the patent is valid and in force (renewal fees paid).
Amazing; the very first sentence in the article is factually incorrect:
Pundits are roasting Apple over a scuffle raised by Mozilla and Opera to define the free Ogg Theora video codec as the official way to present video on the web in the new HTML 5 specification.
There was never any attempt to define Theora as the "official way to present video". It was merely provided as a baseline, a lowest-common-denominator format that could be relied upon by content providers.
Ogg Theora is based upon VP3 video technology originally proprietary to On2, and subsequently abandoned after it became obsolete.
Theora is not obsolete; it is superior to most popular codecs, such as XVid, WMV, and QuickTime. In standard-definition encoding, it's even superior to modern proprietary codecs such as H.264:
Apple has also voiced concerns that Ogg Theora may be encumbered by unknown patents. That risk isn't significant to Mozilla and Opera, both of whom could simply abandon the format for something else [...] Apple also has a bankroll to attract patent trolls that Mozilla and Opera both lack.
This hand-waving ignores that Google will also be supporting Theora, and undoubtedly has deeper pockets than Apple. Besides, a risk of unknown patents applies to any video codec. There's no technical or legal reason to exclude Theora support.
Theora is not obsolete; it is superior to most popular codecs, such as XVid, WMV, and QuickTime. In standard-definition encoding, it's even superior to modern proprietary codecs such as H.264:
Are you seriously this misinformed?
1. Quicktime isn't a "codec", it's a video player. There is no "Quicktime codec".
2. H.264 isn't proprietary.
3. Theora is roughly between MPEG-2 and MPEG-4 SP in potential efficiency; it's nowhere near VC-1 or H.264. Right now it's worse than MPEG-2 because the encoder is so bad, but Thusnelda is working on fixing that. The eventual goal is to become on par with FLV1 and similar formats; reaching more modern levels of compression will require a spec revision.
The purpose of Theora is to provide a patent-free video format, not to be the best out there in compression, since the two goals are basically contradictory. Spreading lies like these are a great way to completely ruin any chance of Theora's adoption, because expectations will be set so high that the Theora developers will never be able to realistically reach them. This is a disservice to Xiph in every way.
Despite the fact that the Theora specification is terrible and even their own developers admit it, they have one thing going their way: the quality of an encoder is far more important than the format itself. In other words, a really good MPEG-2 encoder can give better output than a really bad H.264 encoder, for example. A classic example of this is Quicktime's H.264 encoder, which is so bad that it is widely used across the industry as a marketing tool by competing encoder developers to demonstrate how good theirs is by comparison.
As such, it is possible for Theora to beat some of the lower-end H.264 encoders if they really work hard at it, even if H.264 as a whole is vastly better than Theora. This is why Thusnelda is a big deal for Theora.
This is also why it's such a problem that people consider "H.264" or "MPEG-2" or "JPEG" to be single entities; the compression quality can vary massively between different encoders. That's why I always cringe when someone posts, for example, an "H.264 vs Theora" comparison; they're really doing an "X H.264 encoder vs Thusnelda" comparison and generalizing it far beyond its meaning.
What codec is used for .MOV files, then? As far as I know they can only be en/decoded by Quicktime.
H.264 isn't proprietary.
Cool -- so if I start distributing H.264 videos without paying the appropriate licensing fees, there wouldn't be any trouble, right? And there's an encoder/decoder that I can install legally?
Last I checked, the answer to both of those is "no".
Theora is roughly between MPEG-2 and MPEG-4 SP in potential efficiency; it's nowhere near VC-1 or H.264. Right now it's worse than MPEG-2 because the encoder is so bad, but Thusnelda is working on fixing that. The eventual goal is to become on par with FLV1 and similar formats; reaching more modern levels of compression will require a spec revision.
Obviously you've not used Theora for months or years. It is significantly better than MPEG-4 SP, and on par with H.264 for standard-def video.
What codec is used for .MOV files, then? As far as I know they can only be en/decoded by Quicktime.
MOV files have used dozens of different video and audio formats over the years. Currently, the most common is H.264 video and AAC audio, and they can in fact be played by nearly everything, mainly because the MOV format is basically the same as MP4. You might have to remux the file though, as there are some subtle differences between MOV and MP4. Yes, this is yet another reason to hate Apple.
Cool -- so if I start distributing H.264 videos without paying the appropriate licensing fees, there wouldn't be any trouble, right? And there's an encoder/decoder that I can install legally?
"Proprietary" has a very specific meaning in the multimedia format world; it revers to formats like Real Video, VP6, and so forth which are completely closed. In other words, there is no public specification, nobody can implement an encoder or decoder without reverse-engineering it. If you want something to describe H.264, say "patent-encumbered" or similar; don't misuse words with a specific meaning.
Obviously you've not used Theora for months or years. It is significantly better than MPEG-4 SP, and on par with H.264 for standard-def video.
I've read the spec. Here's the basic summary:
1. Prediction. DC/AC coefficient prediction is somewhat similar to MPEG-4 Part 2. Intra prediction is similar, as is motion compensation, which is basically the same as MPEG-2/4 halfpel except that the center position averages between the diagonals instead of all four neighbors, which is slightly lower quality but much faster (a smart decision, H.264 does the same for qpel). 4MV mode is supported in the same way as in MPEG-4 SP.
2. Transform is basically the same as MPEG-2/4, though it's bit-exact like VC-1's and H.264's in order to avoid decoders disagreeing on the correct output. It's slightly slower though because of how On2 originally designed it.
3. Entropy coder is bizarre but not too terrible; it should probably give results similar to MPEG-4 SP's entropy coder, though I can't be entirely sure just because of how weird it is. Speed-wise the entropy coder is a real problem, especially for hardware implementations. The reason for this is because every single data element is coded in a massive bitplane, requiring that the entire frame be decoded before a single inverse transform is done. This requires a cache larger than the decoded size of the frame itself, as DCT coefficients are stored in 16-bit precision as opposed to the 8-bit precision of the actual pixels. It's probably fine compression-wise though.
That's for residual coefficients though; the primary problem in terms of compression with Theora is the motion vector coding...
3a. Motion vectors can be coded in three modes: "use the previous vector", "use the previous previous vector", or "write an entirely new vector". The last mode requires writing 12 bits of raw data with no compression; X and Y vector components. The problem with this is twofold. First of all, it means you can't have motion vectors longer than 16 pixels, which is crippling for high resolutions. Second, it means the motion vector coding is actually worse than H.261 (the original standardized video format from 1988); Theora probably loses at least 10-30% compression solely because of this. It would in fact simplify the specification to fix this, which is why many have been pushing to revise the spec and release a "Theora 2".
I agree that the complaining about Theora's quality is blown of of proportion by people who don't know any better. Theora is definitely better than low-definition FLV files as streamed by YouTube and better than MPEG-2. But can you really claim it is better than H.264? I don't see many differences in the Theora vs H.264 images on the comparison page, but I can clearly see the differences in the actual videos. H.264 is better than Theora, although not by too much.
I'm more curious about how Theora compares to XviD/DivX, which is (was?) the standard in the video encoding community. But so far I haven't seen any comparisons.
Theora is slightly better than XviD, which is much better than DivX. They are still the standard as far as I know, though being supplanted by H.264 encoders for high-end use.
"In standard-definition encoding, it's even superior to modern proprietary codecs such as H.264"
That's arguably true in some senses, but not in the way most people will take it and it's not really backed up by your link.
Theora only needs one qualification, being in the clear with respect to patents. If Apple's lawyers don't find anything (and it's an interesting question as to whether Apple would be pleased if they did) then there's still a chance Apple will support it.
LZW patents only expired in 2003. Before that, GIF was not only patented, but the patents were actively (and often obnoxiously) enforced by Unisys. The situation was similar to that of MP3 with Frauenhofer in the late 1990s.
LZW encoding was patented, decoding was never patent encumbered. So web browsers never had to worry, since all they were doing was decompressing. For the H.264 case, decoding is also encumbered, so it's not the same situation.
Comments
Fun fact: in the interests of forward compatibility, Firefox appears to have used liboggplay for its video pipeline.
This means that if they ever want to use anything other than Theora, they will have to completely rewrite the entire video system. Apparently the problem with doing something sane like using Gstreamer is that... it's LGPL. Which apparently isn't free enough for them.
They also force YV12 -> RGB conversion in software on playback and do two sizeof(videoframe) callocs per frame, leading to the tremendous irony of having video playback vastly slower than Flash.
Apparently the problem with doing something sane like using Gstreamer is that... it's LGPL. Which apparently isn't free enough for them.
This is not correct. The reason GStreamer is not used in Firefox is because the GStreamer backend wasn't finished in time for FF 3.5. Rather than push back the release date, they released an implementation that works for many videos. The GStreamer backend is still under active development, and will eventually be merged to trunk.
leading to the tremendous irony of having video playback vastly slower than Flash.
Using the most recent version of Flash and Firefox, I can play high-definition Theora easily on my laptop, but even medium-resolution H.264 in Flash struggles to achieve real-time decoding. A quick, unscientific sampling of other FF 3.5 users online has indicated to me that many others experience similar performance in both cases.
I don't think you're correct about the gstreamer backend.
Some Mozilla-based projects use Gstreamer (Songbird for one), and I believe they're having discussions about a future collaboration for accessing user's video cameras in the browser but Mozilla don't want to support any old codecs.
Their mission as a non-profit is to support the open web, and promoting royalty-free codecs fits with that mission. They can make a difference by actively choosing not to do stuff, just like at one point in the past they refused to support ActiveX despite having code available. It would have made Firefox a more useful browser at the cost of making the web less open.
See https://bugzilla.mozilla.org/show_bug.cgi?id=422540 for the progress of the GStreamer <video> backend.
I was confused by this recent comment:
http://lists.xiph.org/pipermail/theora/2009-June/002362.html
"We can't ship gstreamer. We don't ship under the LGPL.
That being said, we can link to it on Linux. We would just need something for the Mac and Windows as well."
Though the mentions of Fennec and the fact that some mobile folk have adopted Gstreamer for DSP accelerated decoding might make it useful there too.
Looks like Fennec on Nokia 810 is the first target:
http://iloapp.mikek.dk/blog/developer?Home&post=44
I'm not sure how that article turned into a treatise on how mean Microsoft is, but I stopped reading after the first third of it. Someone let me know if it picked up after that. :)
Not to shill for my employer or anything, but Ryan Paul wrote a really good overview of the whole HTML5 video thing the other day: http://arstechnica.com/open-source/news/2009/07/decoding-the...
You didn't miss anything, it is average Apple-can't-do-wrong fanboi article.
Some more sophisticated analysis from the author in the comment thread:
"I did read the Ars article, which similarly voices the "free codec is good, international standards body specifications are evil" thing everyone has been repeating.
The bottom line is that x264 is GPL so the whole "Linux users can't use H.264 video" is simply wrong, and in the big picture Mozilla and Opera don't really matter. Once Chrome starts gaining traction, do you really think Google will see any need to dump $50 million on the remains of Netscape's ashes? And what purpose does Opera serve other than to make the EU feel like it has some say in the browser market?"
Sounds like someone pissed in his cheerios. Bloody fanboys.
It isn't about the implementation's license, it's about patents. H.264 is patented while Theora is not. This makes it illegal for anybody to distribute H.264 decoding software, whether open source or not, until they've paid for the patent license.
FooBarWidget: "This makes it illegal for anybody to distribute H.264 decoding software, whether open source or not, until they've paid for the patent license."
Only within the jurisdictions in which the patent is valid and in force (renewal fees paid).
Patents limit distribution of the 'machine', and are intended to make the 'plans' describing how to build it a matter of public record.
Distributing the source code does not infringe the patent.
Amazing; the very first sentence in the article is factually incorrect:
Pundits are roasting Apple over a scuffle raised by Mozilla and Opera to define the free Ogg Theora video codec as the official way to present video on the web in the new HTML 5 specification.
There was never any attempt to define Theora as the "official way to present video". It was merely provided as a baseline, a lowest-common-denominator format that could be relied upon by content providers.
Ogg Theora is based upon VP3 video technology originally proprietary to On2, and subsequently abandoned after it became obsolete.
Theora is not obsolete; it is superior to most popular codecs, such as XVid, WMV, and QuickTime. In standard-definition encoding, it's even superior to modern proprietary codecs such as H.264:
http://people.xiph.org/~greg/video/ytcompare/comparison.html
http://people.xiph.org/~maikmerten/youtube/
Apple has also voiced concerns that Ogg Theora may be encumbered by unknown patents. That risk isn't significant to Mozilla and Opera, both of whom could simply abandon the format for something else [...] Apple also has a bankroll to attract patent trolls that Mozilla and Opera both lack.
This hand-waving ignores that Google will also be supporting Theora, and undoubtedly has deeper pockets than Apple. Besides, a risk of unknown patents applies to any video codec. There's no technical or legal reason to exclude Theora support.
Theora is not obsolete; it is superior to most popular codecs, such as XVid, WMV, and QuickTime. In standard-definition encoding, it's even superior to modern proprietary codecs such as H.264:
Are you seriously this misinformed?
1. Quicktime isn't a "codec", it's a video player. There is no "Quicktime codec".
2. H.264 isn't proprietary.
3. Theora is roughly between MPEG-2 and MPEG-4 SP in potential efficiency; it's nowhere near VC-1 or H.264. Right now it's worse than MPEG-2 because the encoder is so bad, but Thusnelda is working on fixing that. The eventual goal is to become on par with FLV1 and similar formats; reaching more modern levels of compression will require a spec revision.
The purpose of Theora is to provide a patent-free video format, not to be the best out there in compression, since the two goals are basically contradictory. Spreading lies like these are a great way to completely ruin any chance of Theora's adoption, because expectations will be set so high that the Theora developers will never be able to realistically reach them. This is a disservice to Xiph in every way.
Despite the fact that the Theora specification is terrible and even their own developers admit it, they have one thing going their way: the quality of an encoder is far more important than the format itself. In other words, a really good MPEG-2 encoder can give better output than a really bad H.264 encoder, for example. A classic example of this is Quicktime's H.264 encoder, which is so bad that it is widely used across the industry as a marketing tool by competing encoder developers to demonstrate how good theirs is by comparison.
As such, it is possible for Theora to beat some of the lower-end H.264 encoders if they really work hard at it, even if H.264 as a whole is vastly better than Theora. This is why Thusnelda is a big deal for Theora.
This is also why it's such a problem that people consider "H.264" or "MPEG-2" or "JPEG" to be single entities; the compression quality can vary massively between different encoders. That's why I always cringe when someone posts, for example, an "H.264 vs Theora" comparison; they're really doing an "X H.264 encoder vs Thusnelda" comparison and generalizing it far beyond its meaning.
Quicktime isn't a "codec", it's a video player.
What codec is used for .MOV files, then? As far as I know they can only be en/decoded by Quicktime.
H.264 isn't proprietary.
Cool -- so if I start distributing H.264 videos without paying the appropriate licensing fees, there wouldn't be any trouble, right? And there's an encoder/decoder that I can install legally?
Last I checked, the answer to both of those is "no".
Theora is roughly between MPEG-2 and MPEG-4 SP in potential efficiency; it's nowhere near VC-1 or H.264. Right now it's worse than MPEG-2 because the encoder is so bad, but Thusnelda is working on fixing that. The eventual goal is to become on par with FLV1 and similar formats; reaching more modern levels of compression will require a spec revision.
Obviously you've not used Theora for months or years. It is significantly better than MPEG-4 SP, and on par with H.264 for standard-def video.
What codec is used for .MOV files, then? As far as I know they can only be en/decoded by Quicktime.
MOV files have used dozens of different video and audio formats over the years. Currently, the most common is H.264 video and AAC audio, and they can in fact be played by nearly everything, mainly because the MOV format is basically the same as MP4. You might have to remux the file though, as there are some subtle differences between MOV and MP4. Yes, this is yet another reason to hate Apple.
Cool -- so if I start distributing H.264 videos without paying the appropriate licensing fees, there wouldn't be any trouble, right? And there's an encoder/decoder that I can install legally?
"Proprietary" has a very specific meaning in the multimedia format world; it revers to formats like Real Video, VP6, and so forth which are completely closed. In other words, there is no public specification, nobody can implement an encoder or decoder without reverse-engineering it. If you want something to describe H.264, say "patent-encumbered" or similar; don't misuse words with a specific meaning.
Obviously you've not used Theora for months or years. It is significantly better than MPEG-4 SP, and on par with H.264 for standard-def video.
I've read the spec. Here's the basic summary:
1. Prediction. DC/AC coefficient prediction is somewhat similar to MPEG-4 Part 2. Intra prediction is similar, as is motion compensation, which is basically the same as MPEG-2/4 halfpel except that the center position averages between the diagonals instead of all four neighbors, which is slightly lower quality but much faster (a smart decision, H.264 does the same for qpel). 4MV mode is supported in the same way as in MPEG-4 SP.
2. Transform is basically the same as MPEG-2/4, though it's bit-exact like VC-1's and H.264's in order to avoid decoders disagreeing on the correct output. It's slightly slower though because of how On2 originally designed it.
3. Entropy coder is bizarre but not too terrible; it should probably give results similar to MPEG-4 SP's entropy coder, though I can't be entirely sure just because of how weird it is. Speed-wise the entropy coder is a real problem, especially for hardware implementations. The reason for this is because every single data element is coded in a massive bitplane, requiring that the entire frame be decoded before a single inverse transform is done. This requires a cache larger than the decoded size of the frame itself, as DCT coefficients are stored in 16-bit precision as opposed to the 8-bit precision of the actual pixels. It's probably fine compression-wise though.
That's for residual coefficients though; the primary problem in terms of compression with Theora is the motion vector coding...
3a. Motion vectors can be coded in three modes: "use the previous vector", "use the previous previous vector", or "write an entirely new vector". The last mode requires writing 12 bits of raw data with no compression; X and Y vector components. The problem with this is twofold. First of all, it means you can't have motion vectors longer than 16 pixels, which is crippling for high resolutions. Second, it means the motion vector coding is actually worse than H.261 (the original standardized video format from 1988); Theora probably loses at least 10-30% compression solely because of this. It would in fact simplify the specification to fix this, which is why many have been pushing to revise the spec and release a "Theora 2".
"What codec is used for .MOV files, then? As far as I know they can only be en/decoded by Quicktime."
.mov is a container format. It can potentially contain any of many different actual types of encoded media.
I agree that the complaining about Theora's quality is blown of of proportion by people who don't know any better. Theora is definitely better than low-definition FLV files as streamed by YouTube and better than MPEG-2. But can you really claim it is better than H.264? I don't see many differences in the Theora vs H.264 images on the comparison page, but I can clearly see the differences in the actual videos. H.264 is better than Theora, although not by too much.
I'm more curious about how Theora compares to XviD/DivX, which is (was?) the standard in the video encoding community. But so far I haven't seen any comparisons.
I don't know how to help if you can't tell the difference between the examples at < http://people.xiph.org/~greg/video/ytcompare/comparison.html >. The H.264 examples are significantly blurrier than the Theora versions.
Theora is slightly better than XviD, which is much better than DivX. They are still the standard as far as I know, though being supplanted by H.264 encoders for high-end use.
"In standard-definition encoding, it's even superior to modern proprietary codecs such as H.264"
That's arguably true in some senses, but not in the way most people will take it and it's not really backed up by your link.
Theora only needs one qualification, being in the clear with respect to patents. If Apple's lawyers don't find anything (and it's an interesting question as to whether Apple would be pleased if they did) then there's still a chance Apple will support it.
JPEG, GIF, PNG are used as long as nobody threatens us with patent lawsuits.
Just make H.264 free for the world to use (without fine print legalese) and we will adopt it and support it inmediately.
Meantime, theora is the only free and viable option.
Actually, GIF was patented for a very, very long time:
http://en.wikipedia.org/wiki/Graphics_Interchange_Format#Uni...
LZW patents only expired in 2003. Before that, GIF was not only patented, but the patents were actively (and often obnoxiously) enforced by Unisys. The situation was similar to that of MP3 with Frauenhofer in the late 1990s.
LZW encoding was patented, decoding was never patent encumbered. So web browsers never had to worry, since all they were doing was decompressing. For the H.264 case, decoding is also encumbered, so it's not the same situation.