All I'm saying is that you don't take into account all paths for each node. Just for ex 100 random walks starting at each node. And that results in an embedding that is 'good enough'.
Of course it is better to throw more hardware at the issue. But at a certain point the added value of being more precise or adding more hardware becomes moot, because you gain 0.1%
That is what I meant by 'it scales'. You can solve 'reasonably complex issues' with 'reasonably cheap hardware'
Comments
really ? so u keep subsampling as the data becomes larger and larger.
instead of ...well...throwing more hardware that seems to be easier and easier these days.
P.S. not trolling. im genuinely wondering if there is a better way to split the problem heuristically
All I'm saying is that you don't take into account all paths for each node. Just for ex 100 random walks starting at each node. And that results in an embedding that is 'good enough'.
Of course it is better to throw more hardware at the issue. But at a certain point the added value of being more precise or adding more hardware becomes moot, because you gain 0.1%
That is what I meant by 'it scales'. You can solve 'reasonably complex issues' with 'reasonably cheap hardware'