I've thought about doing something similar for google's voice compression lyra https://github.com/google/lyra