An idea I've had, the answer is most probably no, but I found it interesting to think about, others may find it trivial:
Is there a number that (1) contains most subsequences and where (2) finding a certain subsequence is computationally efficient? Also, (3) finding a subsequence given a starting and end position should be efficient. If so, we have a efficient mean to transfer data. Just send the index positions. And we can store any data with just those two index positions. Then again, those index numbers are likely to be very very large. Perhaps we can in turn transform the index positions to smaller index positions by finding their positions in the sequence. Then we need a third number to signify the number of recursive uses of the storage.
I don't know about (1), but (2) could be achieved by scanning through the first n digits and building a lookup tree. If we restrict ourselves to Pi then (3) could be achieved by either just keeping all the digits in memory or using the Bailey–Borwein–Plouffe formula [1], which allows the nth digit to be determined without needing to calculate the preceeding digits.
Your're essentially describing a compression mechanism. As you might expect, this isn't a new idea [2] and it generally fails because (as you said) the index quickly gets large as the size of the block of data increases - basically destroying any advantage for non-trivial block sizes. There is some discussion of the space trade-offs in [3], and I wouldn't be surprised if the speed of this approach is poor when compared to conventional techniques.
Another problem is that it is still not known whether number like Pi actually contains every possible finite sequence of digits [3], so not all input blocks can be compressed. In practice, this shouldn't be a problem for small blocks though.
(2) I'm too lazy to do the calculation right now, but it should not be too difficult to calculate the sum of the lengths of all numbers smaller than n, and then you've got the index of n.
(3) In any feasible encoding, the expected size of the start index of n will be much larger than the size of n. And every recursive step will just blow up the sizes even more.
If you slightly modify it to 0. 0 1 2 3 4 5 6 7 8 9 00 01 02 03 04 05... then the number n padded with zeros to k digits can be found starting at position 10 * floor(10^(k - 1) / 9) + k * n.
I proposed a similar idea to a mathematician friend but I couldn't quite understand his dismissal as I am not a mathematician. My idea was to run algorithms trying to find the most convenient equations that generate number sequences that correspond to raw video piece by piece. It would take massive CPU power but if it is a popular video it might be worth it for the bandwidth savings.
Comments
An idea I've had, the answer is most probably no, but I found it interesting to think about, others may find it trivial:
Is there a number that (1) contains most subsequences and where (2) finding a certain subsequence is computationally efficient? Also, (3) finding a subsequence given a starting and end position should be efficient. If so, we have a efficient mean to transfer data. Just send the index positions. And we can store any data with just those two index positions. Then again, those index numbers are likely to be very very large. Perhaps we can in turn transform the index positions to smaller index positions by finding their positions in the sequence. Then we need a third number to signify the number of recursive uses of the storage.
I don't know about (1), but (2) could be achieved by scanning through the first n digits and building a lookup tree. If we restrict ourselves to Pi then (3) could be achieved by either just keeping all the digits in memory or using the Bailey–Borwein–Plouffe formula [1], which allows the nth digit to be determined without needing to calculate the preceeding digits.
Your're essentially describing a compression mechanism. As you might expect, this isn't a new idea [2] and it generally fails because (as you said) the index quickly gets large as the size of the block of data increases - basically destroying any advantage for non-trivial block sizes. There is some discussion of the space trade-offs in [3], and I wouldn't be surprised if the speed of this approach is poor when compared to conventional techniques.
Another problem is that it is still not known whether number like Pi actually contains every possible finite sequence of digits [3], so not all input blocks can be compressed. In practice, this shouldn't be a problem for small blocks though.
[1] http://en.wikipedia.org/wiki/Bailey-Borwein-Plouffe_formula
[2] https://www.google.co.uk/search?q=pi+compression
[3] http://penduin.blogspot.co.uk/2006/10/pi-compression.html
[4] http://www.askamathematician.com/2009/11/since-pi-is-infinit...
(1) Yes, 0.1234567891011121314151617181920212223242526....
(2) I'm too lazy to do the calculation right now, but it should not be too difficult to calculate the sum of the lengths of all numbers smaller than n, and then you've got the index of n.
(3) In any feasible encoding, the expected size of the start index of n will be much larger than the size of n. And every recursive step will just blow up the sizes even more.
If you slightly modify it to 0. 0 1 2 3 4 5 6 7 8 9 00 01 02 03 04 05... then the number n padded with zeros to k digits can be found starting at position 10 * floor(10^(k - 1) / 9) + k * n.
It's in no way efficient (and is obviously a joke -though one with compile-able and usable code), but piFS [1] kinda does what you want...
[1] https://github.com/philipl/pifs
I proposed a similar idea to a mathematician friend but I couldn't quite understand his dismissal as I am not a mathematician. My idea was to run algorithms trying to find the most convenient equations that generate number sequences that correspond to raw video piece by piece. It would take massive CPU power but if it is a popular video it might be worth it for the bandwidth savings.