I think you have a couple of things mixed up. A stun server is different from a signaling server. Jitsi is an SFU, which is neither. The behavior you describe with combining streams is what we call an MCU. An SFU is simpler and just forwards streams. Most of the logic of an SFU is in controlling RTCP protocols, which are typically aimed at one-to-one use cases. An SFU has to make this work for a one-to-many use case.
Comments
I think you have a couple of things mixed up. A stun server is different from a signaling server. Jitsi is an SFU, which is neither. The behavior you describe with combining streams is what we call an MCU. An SFU is simpler and just forwards streams. Most of the logic of an SFU is in controlling RTCP protocols, which are typically aimed at one-to-one use cases. An SFU has to make this work for a one-to-many use case.