Block released Dessa, a speech model trained only on human conversation

A voice assistant that keeps your sarcasm is a different product from one that transcribes you first. Block, the company behind Square and Cash App, posted on Thursday about a speech model it calls Dessa. Block's small AI research lab says it trained Dessa only on recordings of human conversation. The project page is at block.xyz/dessa. The post reached 116 points on Hacker News and sat near the top of the front page on 23 October.

What Block says the model is

Block says Dessa is a true speech-to-speech model. Audio goes in, audio comes out. The model is full-duplex, so it listens and talks at the same time. The lab says the training set was human conversation only, with no text and no pretrained models. Block calls Dessa "the world's first" AI you can hold a realistic live conversation with. It was trained only from human speech.

Block says inference runs faster than real time, with no awkward wait. The company says the model runs locally on an Apple MacBook Pro M4 Max or better. Block adds that the model was trained on one NVIDIA GB200 NVL72. The post includes recorded demos. In them, Dessa laughs mid-sentence, backchannels with "yeah" or "ooh", and raises its voice when asked to speak up.

What the lab wants from this

Block's post names three motivations: curiosity, better understanding between humans and AI, and safety. The safety point is that current voice AI turns speech into text and loses the emotion in a user's voice. Block says a voice AI that misses pain or distress is a risk. Dessa is a step toward fixing that.

The lab is small. Its post says it reports directly to Jack Dorsey, Block's cofounder and chairman. The team was founded by former members of Dessa, a Toronto deep learning startup that Square acquired in 2020. The project lead named on the page is Ragavan Thurairatnam.

Access is behind a waitlist. Block says it is giving early access to a small group of AI safety researchers first. A model that learned to speak from sound alone is a new kind of model, the post says. The company wants outside researchers to find the risks it has not thought of yet.

The speech-to-text middle step goes away

For a team building a voice interface today, the usual pipeline is: speech-to-text, then a language model, then text-to-speech. That pipeline is why most voice agents feel slow and flat. A real speech-to-speech model that runs on a laptop would remove two of those steps and the loss between them. Dessa is not available yet. A team cannot plan a release on it this quarter. But it changes what a voice interface could do once such a model is available. For a voice-first product, this is the kind of model to plan the design around.

Block has not published a benchmark against existing voice models. Block has not said when the waitlist will open to developers.

Source

Dessa: the Conversation Model trained only on speech, Block. Discussion on Hacker News (116 points on 23 October). Reporting by Emma Roth, The Verge.