This is core to how pikvm / tinypilot work as well!