Interesting! I'd done something similar in Typescript to learn more about BT, and then redid it in rust to learn rust (https://github.com/ckcr4lyf/kiryuu).
However I decided to just use redis as the DB. It sounds like your entire DB is in memory? Any interesting design decisions you made and/or problems faced in doing so?
(My redis solution isn't great since it does not randomize peers in subsequent announces afaik)
in my case using the in-memory ETS has been the best decision, it lets me read&write the peer's data concurrently each on its own process so contention and latency are minimal. the only sequential part is when a new swarm is initially created but that doesn't happen a lot so its fine.
there's sadly no native support for taking random rows directly from the tables, so for now i grab the whole swarm and then take a random subset (https://github.com/Dahrkael/ExTracker/blob/master/lib/ex_tra...)
I don't remember if there's a way to see how many slots an ets table has, but if you're ok with imperfect distribution, you could maybe pick a slot at random and use ets:slot/2 to get all the items in that slot, then select from those.
You might be able to get the slot count from eta:table_info(Table, stats), although that's not intended for production use, so the format may change without notice.
Comments
Interesting! I'd done something similar in Typescript to learn more about BT, and then redid it in rust to learn rust (https://github.com/ckcr4lyf/kiryuu).
However I decided to just use redis as the DB. It sounds like your entire DB is in memory? Any interesting design decisions you made and/or problems faced in doing so?
(My redis solution isn't great since it does not randomize peers in subsequent announces afaik)
in my case using the in-memory ETS has been the best decision, it lets me read&write the peer's data concurrently each on its own process so contention and latency are minimal. the only sequential part is when a new swarm is initially created but that doesn't happen a lot so its fine. there's sadly no native support for taking random rows directly from the tables, so for now i grab the whole swarm and then take a random subset (https://github.com/Dahrkael/ExTracker/blob/master/lib/ex_tra...)
I don't remember if there's a way to see how many slots an ets table has, but if you're ok with imperfect distribution, you could maybe pick a slot at random and use ets:slot/2 to get all the items in that slot, then select from those.
You might be able to get the slot count from eta:table_info(Table, stats), although that's not intended for production use, so the format may change without notice.