ElBruno.Speech.Pipeline

ElBruno.Speech

Local-first speech runtime for .NET — VAD → STT → LLM → TTS 🎙️

A reusable, local-first speech runtime for .NET 8 and .NET 10. Built on Microsoft.Extensions.AI provider boundaries — works with local models (Whisper, VibeVoice, Qwen-TTS, Ollama) and cloud providers (Azure OpenAI, OpenAI) without changing orchestration code.

Audio input
    ↓
Audio normalization · resampling · framing · buffering
    ↓
Voice Activity Detection (Silero VAD)
    ↓
Turn detection · utterance assembly
    ↓
ISpeechToTextClient
    ↓
IChatClient
    ↓
Streaming text segmentation
    ↓
ITextToSpeechClient
    ↓
Audio output

What's New

Packages

Package	NuGet	Downloads	Description
ElBruno.Speech.Abstractions			Audio types, VAD contracts, session interfaces, error types
ElBruno.Speech.Audio			WAV I/O, PCM conversion, resampling, framing, ring buffers
ElBruno.Speech.Vad.Silero			Silero VAD via ONNX Runtime — streaming voice activity detection
ElBruno.Speech.Pipeline			VAD → STT → LLM → TTS orchestration, bounded channels, barge-in
ElBruno.Speech.AspNetCore			WebSocket endpoint, session registry, health checks
ElBruno.Speech.NAudio			Windows microphone and speaker via NAudio
ElBruno.Speech.OpenTelemetry			Activities, metrics, Aspire-compatible instrumentation
ElBruno.Speech.Cli			elbrunospeech dotnet tool
ElBruno.Speech.BlazorComponents			Blazor Server RCL — 6 ready-to-use speech pipeline UI components

Installation

dotnet add package ElBruno.Speech.Pipeline
dotnet add package ElBruno.Speech.Vad.Silero

Install the CLI tool globally:

dotnet tool install -g ElBruno.Speech.Cli
elbrunospeech --help

Quick Start

// Register providers
services.AddWhisper(o => o.Model = KnownWhisperModels.WhisperBaseEn);
services.AddLocalLLMs(o => o.Model = KnownModels.Phi35MiniInstruct);
services.AddVibeVoiceTTS(o => o.SampleRate = 24_000);
services.AddSileroVad(o => o.MinimumSilenceDuration = TimeSpan.FromMilliseconds(500));

// Build the pipeline
services.AddSpeechPipeline(builder =>
{
    builder
        .UseVoiceActivityDetector<SileroVoiceActivityDetector>()
        .UseSpeechToText(sp => sp.GetRequiredService<ISpeechToTextClient>())
        .UseChatClient(sp => sp.GetRequiredService<IChatClient>())
        .UseTextToSpeech(sp => sp.GetRequiredService<ITextToSpeechClient>())
        .UseSentenceChunking(o =>
        {
            o.MinimumCharacters = 24;
            o.MaximumCharacters = 220;
            o.FlushTimeout = TimeSpan.FromMilliseconds(350);
        })
        .UseBargeIn(o => o.CancelOnSpeechStart = true);
});

// Use it
var pipeline = sp.GetRequiredService<ISpeechPipeline>();
await using var session = await pipeline.CreateSessionAsync();

await session.WriteAudioAsync(frame);
await foreach (var update in session.GetUpdatesAsync())
{
    // SpeechStartedUpdate, FinalTranscriptUpdate, AssistantAudioChunkUpdate, ...
}

Experimental multilingual Parakeet STT

An opt-in local Parakeet TDT v3 provider is available from source in ElBruno.Speech.Stt.Parakeet, using the verified elbruno ONNX model bundle. It supports Spanish and other model languages without changing Whisper defaults. Native token-start metadata is preserved; word/segment intervals and language ID are explicitly unavailable. See usage, requirements, and evaluation limits.

CLI Tool

elbrunospeech devices                        # list audio input/output devices
elbrunospeech transcribe recording.wav       # transcribe a WAV file
elbrunospeech vad recording.wav             # run voice activity detection
elbrunospeech talk "Hello world" out.wav    # synthesize text to WAV

Observability

builder.Services.AddOpenTelemetry()
    .AddSpeechPipelineTelemetry();           // meter: ElBruno.Speech, source: ElBruno.Speech
// Aspire automatically configures OTLP export via OTEL_EXPORTER_OTLP_ENDPOINT

Samples

Related Repositories

• ElBruno.Whisper — ISpeechToTextClient via ONNX Whisper
• ElBruno.LocalLLMs — IChatClient for local LLMs
• ElBruno.VibeVoiceTTS — ITextToSpeechClient via VibeVoice
• ElBruno.QwenTTS — ITextToSpeechClient via Qwen3-TTS
• ElBruno.HuggingFace.Downloader — model downloads

Requirements

• .NET 8.0 or .NET 10.0
• Windows, Linux, or macOS
• NAudio package requires Windows (microphone/speaker I/O)

Building from Source

git clone https://github.com/elbruno/ElBruno.Speech.git
cd ElBruno.Speech
dotnet restore
dotnet build
dotnet test --filter "Category!=Integration"

Working with Copilot and Squad

Normal Copilot sessions can work on this repository without loading Squad. Squad is optional: select the Squad custom agent in your client or run copilot --agent squad to use its team orchestration and safety rules. The coordinator integrity check runs only in that custom agent.

If a normal session reports that the Squad coordinator is missing or truncated, update your checkout to include the instruction-scope fix and start a new Copilot session so it reloads .github/copilot-instructions.md. Existing sessions can retain the old instructions. If you select Squad and its loaded instructions are incomplete, stop orchestration and restart with the complete custom agent; do not add a canary to bypass the check.

📄 License

MIT — see LICENSE.

👋 About the Author

Hi! I'm ElBruno 🧡, a passionate developer and content creator exploring AI, .NET, and modern development practices.

Made with ❤️ by ElBruno

If you like this project, consider following my work across platforms:

• 📻 Podcast: No Tienen Nombre — Spanish-language episodes on AI, development, and tech culture
• 💻 Blog: ElBruno.com — Deep dives on embeddings, RAG, .NET, and local AI
• 📺 YouTube: youtube.com/elbruno — Demos, tutorials, and live coding
• 🔗 LinkedIn: @elbruno — Professional updates and insights
• 𝕏 Twitter: @elbruno — Quick tips, releases, and tech news

🙏 Acknowledgments

• Microsoft.Extensions.AI — ISpeechToTextClient, IChatClient, ITextToSpeechClient interfaces
• Silero VAD — voice activity detection model
• ONNX Runtime — local model inference
• NAudio — Windows audio I/O