This presentation hints at one possible implementation of this, specifically for VoIP/RTP.

http://www.cluecon.com/presentation/building-conferenceivr-p...

I would love to see actual working code for IP/SIP/RTP in node.js using packet sockets, if anybody knows of an open source project.