The way PJMEDIA works at the moment is, for each destination party (e.g. remote INVITE party), we have one media "session". For each "m" line in the SDP, PJMEDIA creates one media "stream". If the stream is a bi-directional audio, then for each stream, two media "channels" will be created, so one media channel for each direction.

The two channels in one stream share one instance of "codec". A codec is simple struct that provides encode() and decode() functions. 

The media channels will end up in the appropriate "sound stream". The decoder channel (i.e. RTP receiver) will end up in sound player stream, and the encoder channel (i.e. RTP sender) gets the audio frames from sound recorder stream.

Both sound player and recorder devices (or streams) are active objects (they have their own threads). The media channel only needs to register callback function to be called when audio frames are available (or should be supplied) from/to the sound devices. This approach works very well with DirectSound, or with PortAudio's sound framework.

But with the introduction of jitter buffer, another thread needs to be created for the decoder channel. The thread reads RTP from socket on a periodic basis, and put the frame (still encoded) to jitter buffer. When the sound player callback is called (by sound device), it looks for frame in the jitter buffer (instead of reading RTP socket), decode the frame, and return the PCM frame to the sound player.

Now getting back to the topic why I think this could work as it is for your application.