NVIDIA Nemotron 3 Diarization: why correctly separating speakers matters in a medical consultation

In speech recognition, almost all the attention usually goes to one metric: how many words the system transcribed correctly.

In a clinical conversation, however, there is another question that is just as important:

who said those words?

Mistaking a patient’s sentence for a professional’s instruction can completely change how a transcription is interpreted.

That is the problem speaker diarisation tries to solve: splitting a recording into segments and determining which contributions belong to each person.

On 23 September 2026, NVIDIA released Nemotron 3 Diarization, an open-weights model aimed precisely at this problem.

What NVIDIA has released

According to the model’s technical sheet, Nemotron 3 Diarization has 99.2 million parameters and can handle up to eight speakers, both on recorded audio and in streaming scenarios.

NVIDIA also indicates a minimum-latency configuration of 0.32 seconds, a feature that is especially relevant for applications that need to process conversations while they are taking place.

On the DIHARD III benchmark, the company reports a diarisation error rate — DER — of 12.73% in offline mode.

With the minimum-latency configuration, the reported result is 13.55%.

The previous four-speaker model used in the comparison reached a DER of 19.09%.

What diarisation really means

An automatic speech recognition system can produce a correct transcription and still be insufficient for documenting a conversation.

Let us imagine a simple excerpt:

— I’ve had pain since Monday. — Have you taken any medication? — Ibuprofen.

The transcription contains all the words.

But without knowing the turns, the system loses a fundamental part of the structure of the conversation.

In a real consultation the problem is considerably more complex. There are interruptions, silences, rapid changes of turn, overlapping voices and conversations that can last many minutes.

That is why diarisation is not a cosmetic addition to transcription. It is a layer that helps reconstruct the conversation.

The important piece still missing: clinical Spanish

The results published by NVIDIA are promising, but they do not automatically allow us to conclude that behaviour will be equivalent in a healthcare consultation in Spain.

According to the information in the model’s sheet, the training data include English, Mandarin, Hindi, Kannada, Telugu and Bengali.

Moreover, DIHARD III is a demanding diarisation benchmark, but it does not specifically represent clinical conversations in Spanish.

That difference matters.

The combination of language, acoustics, consultation length, number of participants, interruptions and natural ways of speaking can substantially change a system’s performance.

Therefore, before incorporating any model into a clinical architecture, it must be evaluated on the context in which it is actually intended to be used.

From benchmark to a real consultation

This gap between a good public metric and a system’s behaviour in real situations is precisely one of the questions we are studying at Myukura.

During our participation in the 8th MareNostrum Hackathon at the Barcelona Supercomputing Center we worked on clinical speech recognition in Spanish and on the difficulties that appear when these systems face medical vocabulary, different accents and lengthy conversations.

Diarisation is part of the same general problem.

Getting a good isolated transcription is not enough. The goal is to preserve the structure needed to work on that information afterwards.

Why open models matter

Nemotron 3 Diarization is released under the OpenMDW v1.1 licence with publicly available weights.

The ability to study and run models of this type outside completely closed services is especially interesting for lines of research related to local processing.

It allows researchers and technical teams to evaluate the model’s behaviour, measure its hardware requirements and study architectures in which certain processing stages do not necessarily depend on sending all the information to a remote service.

We follow publications like this as a reference for our research on local processing, within Myukura Edge.

As members of NVIDIA Inception, we follow this kind of publication particularly closely, but on a condition we consider fundamental: a technical sheet or a benchmark is a starting point, not a validation for a specific clinical environment.

The next question must always be the same:

how does it work with our data, our language and our use case?