// TECHCRUNCH — INTELLIGENZA ARTIFICIALE
Block’s new voice model sounds eerily human
Jack Dorsey’s Block released a new voice model out of the company’s research lab that runs locally and sounds strangely human with its expressions like laughs, pauses, and cries. The company said the model, called Dessa, is trained fully on human speech and a single NVIDIA GPU rack.
Dessa is a full-duplex model, which means it can talk and listen at the same time, just like the GPT-Live model OpenAI released earlier this year. Typically, speech models are paired with a large language model for knowledge retrieval. Block said that Dessa’s knowledge stems from the conversational data it has.
We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence. https://t.co/r0qFLdvd6c
Block said that the current models can sound unnatural and annoying because they have text in the pipeline that drives speech generation. A lot of models like ElevenLabs v4 or Fish Audio Drama 3 use voice tags to drive emotion in speech generation. Dorsey’s company claims that emotion is built into its model. It’s not clear if that emotion is being steered by some kind of tags.
“Interacting with current voice AI can just plain suck. It turns your voice into text and throws away an incredible amount of information text alone can’t carry. The most powerful AIs in the world can’t hear how you feel. We think that’s ridiculous, leads to a worse experience, and is potentially dangerous,” the company said in a blog post.
The company’s researchers posted multiple examples of conversations with Dessa where the agent had different voices, cadences, and emotions. For instance, in the conversation, the bot said it was tired from work, and at that point it sounded tired.
We made the first AI model you can have a realistic conversation with that was trained entirely on speech!Dessa is a full-duplex speech-to-speech model that you can run on consumer GPUs.This is a GPT3 moment for speech. pic.twitter.com/XVD8sSyJn8
In the example below, the bot was talking about the person making it comfortable while sounding emotional and almost breaking down.
Dessa=full-duplex speech model vs Human=me, https://t.co/1eI6QFHo1w pic.twitter.com/zlZfxH2BA5
Another one shows a Block team member playing a game with the voice assistant.