Why aren't we see many video hosting companies adopt peertube or P2P?
Case Study: Bit Torrent [0]
Bit torrent is a brilliant idea - allows everyone who has part(s) of a file to contribute to the pool of availability so that any given central server/mirror isn't overwhelmed
And it has its place (eg in file sharing)
But for streaming? Not so much
Say you're getting the "next chunk" (whatever 'chunksize' is in this context) from me, and I go offline (it's the end of my day, need to reboot for updates, any of myriad reasons). Where does the next bit of the video come from in a way that is seamless to the viewer?
That is the fundamental problem of shared/p2p streaming protocols - everytime the host of the current/next blob o' data goes offline, you need to waste time finding a replacement
Even if the replacement can be found "quickly", how do you ensure they don't go offline in the middle of streaming? How do you ensure "enough" copies of every chunk are distributed that it, effectively, 'doesn't matter' how many go offline [at once], it will still stream?
You get around that with long buffers. Like...several minutes long, rather than several seconds.
You still have a good chance of buffer underruns when you start a video, but that would likely get mitigated by the fact that the data for the start of the video would likely end up highly duplicated on the network.
The bigger issue is seeking. Jumping to the middle of a video could take many seconds before it played. The UX would be awful.
You get around that with long buffers. Like...several minutes long, rather than several seconds.
That sounds horrible - now you've got to have something buffering [potentially] 100s of MBs (depending on quality) over lousy residential upload speeds?
It would take at least three seeds at Comcastic upload speeds (~10Mbps) to handle a single client at 25Mbps with a small buffer. You'd need a multiple of seeds proportionate to the ratio of stream rate to buffer size. Not ever seeder will have every bit of content, or even every bit of the top 1% of content for a large corpus. Because a large portion of clients will be leechers unable to reseed (mobile, shitty residential routers, etc) the network would need a huge number of seeders with even modest popularity.
everytime the host of the current/next blob o' data goes offline, you need to waste time finding a replacement
Ther is an algorithm in place where the rarest is downloaded. Also a priority list of older peers is implemented. Some protocol also request the chunk from muliple peers (kademlia).
Even if the replacement can be found "quickly", how do you ensure they don't go offline in the middle of streaming? How do you ensure "enough" copies of every chunk are distributed that it, effectively, 'doesn't matter' how many go offline [at once], it will still stream?
You don't need to _ensure_ all that stuff, in the strictest sense of the word. Network hiccups happen, nodes go down. As long as the frequency of interruptions scales inversely with the number of nodes in the network, once you're on a big enough network everything will work smoothly.
Wouldn't the "next chunk" problem be mitigated greatly by just maintaining larger buffer sizes? I know most client devices (such as Smart TVs and HDMI Dongles) don't have the memory capacity for it, but a 10 second long buffer would effectively mask a lot of those issues.
That being said, most residential internet packages have low upload speeds, which is an issue
A 1Mbps stream (high quality 480p or potato quality 720p) would need a 10Mbps burst to fill a 10 second buffer or wait some time before playback. It would need somewhere north of 1Mbps to keep the buffer full accounting for dropped packets, jitter, and seeds dropping out. If the stream drops below 1Mbps for longer than ten seconds then playback stops.
Comments
Case Study: Bit Torrent [0]
Bit torrent is a brilliant idea - allows everyone who has part(s) of a file to contribute to the pool of availability so that any given central server/mirror isn't overwhelmed
And it has its place (eg in file sharing)
But for streaming? Not so much
Say you're getting the "next chunk" (whatever 'chunksize' is in this context) from me, and I go offline (it's the end of my day, need to reboot for updates, any of myriad reasons). Where does the next bit of the video come from in a way that is seamless to the viewer?
That is the fundamental problem of shared/p2p streaming protocols - everytime the host of the current/next blob o' data goes offline, you need to waste time finding a replacement
Even if the replacement can be found "quickly", how do you ensure they don't go offline in the middle of streaming? How do you ensure "enough" copies of every chunk are distributed that it, effectively, 'doesn't matter' how many go offline [at once], it will still stream?
--------------
[0] https://en.wikipedia.org/wiki/BitTorrent
You get around that with long buffers. Like...several minutes long, rather than several seconds.
You still have a good chance of buffer underruns when you start a video, but that would likely get mitigated by the fact that the data for the start of the video would likely end up highly duplicated on the network.
The bigger issue is seeking. Jumping to the middle of a video could take many seconds before it played. The UX would be awful.
That sounds horrible - now you've got to have something buffering [potentially] 100s of MBs (depending on quality) over lousy residential upload speeds?
Netflix in 4K is up to 25 Mbps, which would put a 5 minute buffer at about 1 GB.
That's perfectly reasonable on any desktop/laptop, and likely fine on any decent phone or tablet.
Still doesn't solve the instant seeking problem, of course.
It would take at least three seeds at Comcastic upload speeds (~10Mbps) to handle a single client at 25Mbps with a small buffer. You'd need a multiple of seeds proportionate to the ratio of stream rate to buffer size. Not ever seeder will have every bit of content, or even every bit of the top 1% of content for a large corpus. Because a large portion of clients will be leechers unable to reseed (mobile, shitty residential routers, etc) the network would need a huge number of seeders with even modest popularity.
Ther is an algorithm in place where the rarest is downloaded. Also a priority list of older peers is implemented. Some protocol also request the chunk from muliple peers (kademlia).
You don't need to _ensure_ all that stuff, in the strictest sense of the word. Network hiccups happen, nodes go down. As long as the frequency of interruptions scales inversely with the number of nodes in the network, once you're on a big enough network everything will work smoothly.
Wouldn't the "next chunk" problem be mitigated greatly by just maintaining larger buffer sizes? I know most client devices (such as Smart TVs and HDMI Dongles) don't have the memory capacity for it, but a 10 second long buffer would effectively mask a lot of those issues.
That being said, most residential internet packages have low upload speeds, which is an issue
A 1Mbps stream (high quality 480p or potato quality 720p) would need a 10Mbps burst to fill a 10 second buffer or wait some time before playback. It would need somewhere north of 1Mbps to keep the buffer full accounting for dropped packets, jitter, and seeds dropping out. If the stream drops below 1Mbps for longer than ten seconds then playback stops.
It all makes for a shitty user experience.