High-quality lossy files are fine for listening, but lossless is required if you want to sample the audio to create a remix or DJ set from it, because you're inevitably going to have to encode the output artifact of that—and you don't want to be chaining lossy encoding steps, as the loss-function of most psychoacoustic encodings tends to be exponential when applied repeatedly.
re: wavpack, from wikipedia:
WavPack also incorporates a "hybrid" mode which still provides the features of lossless compression, but it creates two files: a relatively small, high-quality, lossy file (.wv) that can be used by itself; and a "correction" file (.wvc) that, when combined with the lossy file, provides full lossless restoration. This allows the use of lossy and lossless codecs together.
This hybrid encoding format lets you keep files for both purposes (lossless for audio production and archiving, pre-encoded to HQ lossy for audio consumption) in less space than would be required to keep e.g. an ALAC file + a cached lossy M4A encoding of it, the way iTunes does when you specify "optimize tracks' space-usage for portable devices."
Such a hybrid design is ideal for e.g. the Internet Archive, where they want to keep lossless archival representations of files for historians to study, and to create encodings from; but also want to serve usable representations of those artifacts to users who care more about transfer-speed and bandwidth cost than perfect fidelity.
This has been tried - the problem is that FLAC works by removing redundancies and correlation in the audio signal. When you subtract an Opus encoding from the original, the resulting difference file is the difference, which is much less correlated noise - often it actually compresses worse than the original FLAC. WavPack gets around this by making its lossy format a subset of the lossless data, but as a downside the lossy compression is far worse - about 3x worse than Opus.
Other attempts include Dolby TrueHD and DTS HD - but both of those ended up larger than just separate FLAC and Opus files.
Keep in mind that Opus does not define a bit-exact decoder. That allows it to have both floating-point and fixed-point implementations, as well as the flexibility to use different internal resamplers, MDCT implementations, etc. That flexibility helps to allow efficient implementations across a broad variety of devices.
However, as a result, the difference between an Opus decode and the original will only get you lossless reconstruction if you decode with the same build of Opus on the same machine. Also, since Opus uses IIR filters that ring to infinity, even on the same machine you only get bit-exact results if you decode each file from the beginning. It won't work if you seek.
Really, if you're concerned about the disk space required for separate FLAC and Opus encodes, delete the Opus version. FLAC decoding is extremely fast (5x faster than Wavpack), and Opus encoding is pretty efficient, too (on the same order as Wavpack decoding).
It's not published yet, though I might. It's pretty simple, just grabs a static JSON manifest (which is generated by the shell script I run when I add music to the collection, which adds replaygain tags and transcodes) which includes basic metadata and the ReplayGain tags so that I can set that manually in the browser. Uses the standard rule for continuous play and replaygain: play whatever was in the view when you played the first song then shuffle play, when the song you're playing is followed by the next song or follows the previous song on the same album, apply album gain, else apply track gain.
I have yet to add dynamic features like cross-player play count tracking, and server-side play counts for the webapp. I think these would be a lot more work than I put in.
I just use separate files, honestly. Hybrid could save me a small amount of disk space, which is itself quite cheap. Allows me to have a central repository for files of every bitrate for each device class. Smartphone gets the 128k, I stream the 96k, 128k, or FLAC (depending on connection and browser constraints) from my private listening webapp, and I keep the 128k Opus on my laptop.
FLAC and Opus are each more widely supported than Wavpack, Chrome can play both as of the last stable release. My loudness scanner, cmus, MPD, and others all recognize each fine, including ReplayGain and album art, but I'm not sure I'd have the same experience with Wavpack.
I have about 600GiB FLAC music from bandcamp, random publishers, and CDs. I then have, separately, the same directory structure but encoded as 96k and 128k Opus. Still all fits just fine on a mere TiB disk. In a couple years the difference between 96k and 128 in terms of space won't matter and I'll drop 96.
And yeah, it's very nice to have the FLACs when I want to timewarp, transcribe, analyze, or remix. I usually do that at home though, so I can splurge on the bandwidth.
WRT the Internet Archive, Wavpack hybrid decoding is probably too slow and too rare to bother including in any major browser. Even if they did, if lossy listening is a priority, a dedicated lossy codec will save them money in short order on bandwidth. Indeed, this is exactly how IA does things, they even have wav in addition to flac and two or three lossy versions of (almost?) every recording.
Not a user of Wavpack myself, but I think the idea would be to have a sort of "pressure-based fidelity retention" setup: where you'd put the lossy-encoded versions of files onto your portable player device, but as long as your portable device has the space, it would also retain the lossless encodings as supplemental files. As soon as your portable starts running out of space, it treats the lossless encodings as a cache folder, evicting things from it to free up space.
Then, you could actually do mixing in apps like DJay straight on your iPad, on a whim, sometimes—without having planned for it in advance—and make it straight through to mastering and uploading the result... unless you filled up your whole iPad with random videos/apps/etc. in the meantime, in which case the lossless versions would be gone, you'd have to work with the lossy copies, and then, when you get home, restore the lossless copies (or transfer the project to your PC) to master the encoding.
So, not so much when you want the lossless version as canonical (most times when you have a lot of space), but rather when you want the lossy version to be canonical, and the lossless version to be there as a sort of "optional file to enable mastering using this track" file, that's sometimes available and sometimes not.
Comments
High-quality lossy files are fine for listening, but lossless is required if you want to sample the audio to create a remix or DJ set from it, because you're inevitably going to have to encode the output artifact of that—and you don't want to be chaining lossy encoding steps, as the loss-function of most psychoacoustic encodings tends to be exponential when applied repeatedly.
re: wavpack, from wikipedia:
This hybrid encoding format lets you keep files for both purposes (lossless for audio production and archiving, pre-encoded to HQ lossy for audio consumption) in less space than would be required to keep e.g. an ALAC file + a cached lossy M4A encoding of it, the way iTunes does when you specify "optimize tracks' space-usage for portable devices."
Such a hybrid design is ideal for e.g. the Internet Archive, where they want to keep lossless archival representations of files for historians to study, and to create encodings from; but also want to serve usable representations of those artifacts to users who care more about transfer-speed and bandwidth cost than perfect fidelity.
This has been tried - the problem is that FLAC works by removing redundancies and correlation in the audio signal. When you subtract an Opus encoding from the original, the resulting difference file is the difference, which is much less correlated noise - often it actually compresses worse than the original FLAC. WavPack gets around this by making its lossy format a subset of the lossless data, but as a downside the lossy compression is far worse - about 3x worse than Opus.
Other attempts include Dolby TrueHD and DTS HD - but both of those ended up larger than just separate FLAC and Opus files.
I made some very crude tests a few minutes ago and got about 15% size increase in total size (opus + diff.flac vs flac --best)
Keep in mind that Opus does not define a bit-exact decoder. That allows it to have both floating-point and fixed-point implementations, as well as the flexibility to use different internal resamplers, MDCT implementations, etc. That flexibility helps to allow efficient implementations across a broad variety of devices.
However, as a result, the difference between an Opus decode and the original will only get you lossless reconstruction if you decode with the same build of Opus on the same machine. Also, since Opus uses IIR filters that ring to infinity, even on the same machine you only get bit-exact results if you decode each file from the beginning. It won't work if you seek.
Really, if you're concerned about the disk space required for separate FLAC and Opus encodes, delete the Opus version. FLAC decoding is extremely fast (5x faster than Wavpack), and Opus encoding is pretty efficient, too (on the same order as Wavpack decoding).
What do you use for your private listening webapp? mpd?
I'm guessing you meant to reply to my reply.
It's not published yet, though I might. It's pretty simple, just grabs a static JSON manifest (which is generated by the shell script I run when I add music to the collection, which adds replaygain tags and transcodes) which includes basic metadata and the ReplayGain tags so that I can set that manually in the browser. Uses the standard rule for continuous play and replaygain: play whatever was in the view when you played the first song then shuffle play, when the song you're playing is followed by the next song or follows the previous song on the same album, apply album gain, else apply track gain.
I have yet to add dynamic features like cross-player play count tracking, and server-side play counts for the webapp. I think these would be a lot more work than I put in.
My Github is https://github.com/xorgy if you want to follow/check in to see if I publish it.
I just use separate files, honestly. Hybrid could save me a small amount of disk space, which is itself quite cheap. Allows me to have a central repository for files of every bitrate for each device class. Smartphone gets the 128k, I stream the 96k, 128k, or FLAC (depending on connection and browser constraints) from my private listening webapp, and I keep the 128k Opus on my laptop. FLAC and Opus are each more widely supported than Wavpack, Chrome can play both as of the last stable release. My loudness scanner, cmus, MPD, and others all recognize each fine, including ReplayGain and album art, but I'm not sure I'd have the same experience with Wavpack.
I have about 600GiB FLAC music from bandcamp, random publishers, and CDs. I then have, separately, the same directory structure but encoded as 96k and 128k Opus. Still all fits just fine on a mere TiB disk. In a couple years the difference between 96k and 128 in terms of space won't matter and I'll drop 96.
And yeah, it's very nice to have the FLACs when I want to timewarp, transcribe, analyze, or remix. I usually do that at home though, so I can splurge on the bandwidth.
WRT the Internet Archive, Wavpack hybrid decoding is probably too slow and too rare to bother including in any major browser. Even if they did, if lossy listening is a priority, a dedicated lossy codec will save them money in short order on bandwidth. Indeed, this is exactly how IA does things, they even have wav in addition to flac and two or three lossy versions of (almost?) every recording.
Not a user of Wavpack myself, but I think the idea would be to have a sort of "pressure-based fidelity retention" setup: where you'd put the lossy-encoded versions of files onto your portable player device, but as long as your portable device has the space, it would also retain the lossless encodings as supplemental files. As soon as your portable starts running out of space, it treats the lossless encodings as a cache folder, evicting things from it to free up space.
Then, you could actually do mixing in apps like DJay straight on your iPad, on a whim, sometimes—without having planned for it in advance—and make it straight through to mastering and uploading the result... unless you filled up your whole iPad with random videos/apps/etc. in the meantime, in which case the lossless versions would be gone, you'd have to work with the lossy copies, and then, when you get home, restore the lossless copies (or transfer the project to your PC) to master the encoding.
So, not so much when you want the lossless version as canonical (most times when you have a lot of space), but rather when you want the lossy version to be canonical, and the lossless version to be there as a sort of "optional file to enable mastering using this track" file, that's sometimes available and sometimes not.