Convert G722 stream into raw pcm s16le stream

[msehnout]

Hello,

I'm trying to build a telco app and I need to convert audio codecs used in telco to raw pcm s16le for processing using voice-activity-detection, speech-to-text and other technologies. I managed to build a converter using libav from ALaw to PCM like this:

# working solution
    def convert_audio_chunk_alaw_to_pcm16(chunk: bytes | None) -> Iterator[bytes]:
        decoder: av.AudioCodecContext = av.codec.CodecContext.create("pcm_alaw", "r")  # pyright: ignore
        decoder.sample_rate = 8000
        decoder.layout = "mono"
        resampler = av.AudioResampler(format="s16", layout="mono", rate=16000)

        if chunk:
            packet = av.packet.Packet(chunk)  # pyright: ignore
        else:
            packet = None

        for frame in decoder.decode(packet):
            resampled: list[av.AudioFrame] = resampler.resample(frame)

            for sample in resampled:
                yield bytes(sample.planes[0])

I'm not sure why, but I had to strip few bytes of the end of each packet:

# working solution
return bytes(output_buffer[:-128]) if output_buffer else None

But it works and the output audio is clear and VAD/STT works without any issue.

Now I'm trying to build the same for G722 codec, but I haven't been able to get the code to work. My first attempt was simply replacing the decoder in the code above, but that did not work. So I tried to use LLM to generate the code for me, but that also resulted in non working solution:

# non-working solution
 container = av.open(io.BytesIO(chunk), format="g722")
 stream = container.streams.audio[0]
 resampler = AudioResampler(format='s16', layout='mono', rate=16000)

 for packet in container.demux(stream):
     for frame in packet.decode():
         resampled_frames = resampler.resample(frame)
         for resampled in resampled_frames:
             pcm = resampled.to_ndarray().astype(np.int16).tobytes()
             yield pcm

Any ideas how to get this to work?

I also tried running ffmpeg as a subprocess and simply sending audio to STDIN and getting it back from STDOUT like this

ffmpeg -f g722 -ac 1 -i - -f s16le -ar 16000 -ac 1 -

But that caused delay and I haven't been able to figure out where the delay comes from.

Thanks for any ideas 🙏

[abdullayevziyodulla]

Two details matter for this streaming case: keep decoder/resampler state across chunks, and export the actual audio samples rather than the whole padded plane buffer.

For raw G.722 payloads, a persistent decoder can look like:

import av

decoder = av.CodecContext.create("g722", "r")
decoder.sample_rate = 16000
decoder.layout = "mono"
resampler = av.AudioResampler(format="s16", layout="mono", rate=16000)

def convert_chunk(chunk):
    # chunk=None only when the stream ends, to flush the decoder.
    packet = None if chunk is None else av.Packet(chunk)
    for frame in decoder.decode(packet):
        for out in resampler.resample(frame):
            yield out.to_ndarray().astype("<i2", copy=False).tobytes()
    if chunk is None:
        for out in resampler.resample(None):
            yield out.to_ndarray().astype("<i2", copy=False).tobytes()

Create these objects once per incoming stream, not on every chunk. If receiving RTP, remove its transport header first; the decoder needs G.722 payload data. G.722 audio is 16 kHz even though its RTP timestamp clock is conventionally 8 kHz, so those two rates must not be conflated.

bytes(frame.planes[0]) can include alignment padding. Exporting the ndarray avoids removing an arbitrary 128-byte suffix, which could remove real samples when frame sizes change.

I tested this decoder/resampler pattern on a synthetic one-second G.722 stream with PyAV 19.0.1: it produced exactly 32,000 bytes of mono s16le at 16 kHz. See the AudioFrame/AudioResampler APIs.

[msehnout]

Thanks for your answer 🙏 . I will give it a try. In the meantime ALaw STT providers got much better so it might not be needed, but still a good thing to have the option.