How to infer diarization pipeline with multi-gpus or cpus for faster processing?

[shuvohishab]

Hi there,
Currently I'm using pyannote/speaker-diarization-community-1 locally for diarization. I'm curious to know how to infer the process faster? On GPU side, it only covers 2/3 GB of space among 80GB VRAM size and the process is tooo slow. Processes only 2 audios/sec.

How to make the process faster?

[arcticloud]

A single file can't be spread across multiple GPUs — pyannote runs one file on one device. The low VRAM use (2-3 GB out of 80) is the real clue: you're not compute-bound, you're running the internal sliding windows at a small batch size, so the GPU sits mostly idle. Two concrete speedups:

1. Raise the batch sizes so each GPU call does more work:

   import torch
   pipeline.to(torch.device("cuda"))
   pipeline.segmentation_batch_size = 32
   pipeline.embedding_batch_size = 32     # push to 64/128 while VRAM allows

   On an 80 GB card you can go much higher; this alone usually gives a big throughput jump because you finally use that idle VRAM.

2. For many files, go data-parallel, not model-parallel: launch one process per GPU (CUDA_VISIBLE_DEVICES=0, =1, …) and shard your file list across them. That scales almost linearly with GPU count, whereas trying to split one file does not.

If throughput still matters after that, the newer community pipeline is faster than the older 3.1, and make sure you're passing an in-memory waveform ({"waveform": ..., "sample_rate": ...}) rather than a path so decoding isn't repeated. But start with the batch sizes — that's the fix for "2 GB used, very slow".