Shiny.AiConversation

Shiny.AiConversation

A centralized AI service library for .NET MAUI apps that orchestrates chat, speech recognition, wake word detection, text-to-speech, and persistent message history into a single IAiConversationService interface.

Features

• Chat Integration — Send text or voice messages to any AI backend via Microsoft.Extensions.AI
• Wake Word Detection — Hands-free activation with continuous keyword listening
• Speech-to-Text / Text-to-Speech — Full voice loop powered by Shiny.Speech
• Acknowledgement Modes — None, AudioBlip (sound effects), LessWordy (concise TTS), or Full (complete TTS)
• Context Providers — Pluggable IContextProvider interface for supplying system prompts and AI tools per request. A built-in DefaultContextProvider handles time-based prompts, acknowledgement-aware voice prompts, and DI-registered AITool instances.
• Persistent Chat History — Pluggable IMessageStore for storing and querying past conversations
• AI History Lookup Tool — Optional AITool that lets the AI search past conversations on its own
• State Management — Observable AiState (Idle / Listening / Thinking / Responding) with events
• Sound Effects — Configurable sound stream factories for each state transition
• Conversation Continuation — AI responses ending with a question automatically keep the microphone open for a reply
• Voice Interruption — Configurable quiet words (e.g., "stop", "cancel") immediately silence TTS and break out of the conversation. Any other speech during TTS interrupts and continues the conversation with the new utterance.
• Speech Options — Configurable SpeechToTextOptions and TextToSpeechOptions (culture, silence timeout, voice, speech rate, etc.)

TODO

• Sessions - ability to start different AI sessions based on time passed
• Acknowledgement sounds is present, but we need acknowledgement "Hi User" or "What can I help you with?"
  • Manually activation could just be a sound? Maybe only the wake word should have a greeting, and manual text input doesn't need it?
• Speech to text - Wait for anything? then 2 seconds of silence

Installation

dotnet add package Shiny.AiConversation

Quick Start

1. Register the service

using Shiny.AiConversation;

var builder = MauiApp.CreateBuilder();
builder
    .UseMauiApp<App>()
    .ConfigureFonts(fonts => { ... });

// Register an IChatClient in DI (from any Microsoft.Extensions.AI-compatible provider)
builder.Services.AddChatClient(new OpenAIClient("your-api-key").GetChatClient("gpt-4o").AsIChatClient());

builder.Services.AddShinyAiConversation(opts =>
{
    // Optional — enable persistent history + AI lookup tool
    opts.SetMessageStore<MyMessageStore>(addAiLookupTool: true);
});

var app = builder.Build();

// Configure after build
var ai = app.Services.GetRequiredService<IAiConversationService>();
ai.SoundResolver = name => FileSystem.OpenAppPackageFileAsync(name);
ai.OkSound = "ok.mp3";
ai.ThinkSound = "think.mp3";
ai.RespondingSound = "responding.mp3";
ai.ErrorSound = "error.mp3";
ai.CancelSound = "cancel.mp3";

return app;

2. Chat Client Setup

By default, the library resolves IChatClient from DI. Simply register your chat client with the container:

builder.Services.AddChatClient(new OpenAIClient("your-api-key").GetChatClient("gpt-4o").AsIChatClient());

Shiny.AiConversation.OpenAi

A ready-made static OpenAI provider. Works with any OpenAI-compatible endpoint (OpenAI, Azure OpenAI, Ollama, etc.).

dotnet add package Shiny.AiConversation.OpenAi

builder.Services.AddShinyAiConversation(opts =>
{
    opts.AddStaticOpenAIChatClient(
        apiToken: "your-api-key",
        endpointUri: "https://api.openai.com/v1",
        modelName: "gpt-4o"
    );
});

Shiny.AiConversation.Maui.GithubCopilot

A MAUI-specific provider that authenticates via the GitHub device code flow and uses the Copilot API. Tokens are stored in SecureStorage. Authentication is fully self-contained — the library shows a popup with the device code, copies it to the clipboard, and opens the browser for the user to authorize.

dotnet add package Shiny.AiConversation.Maui.GithubCopilot

builder.Services.AddShinyAiConversation(opts =>
{
    opts.AddGithubCopilotChatClient();
});

No additional setup is needed — the provider handles the entire OAuth flow, token exchange, caching, and re-authentication on expiry.

Custom Provider

For other backends, implement IChatClientProvider:

using Microsoft.Extensions.AI;
using Shiny.AiConversation;

public class MyChatClientProvider : IChatClientProvider
{
    public async Task<IChatClient> GetChatClient(CancellationToken cancelToken = default)
    {
        // Handle auth, token refresh, etc.
        return BuildChatClient();
    }
}

// Register it:
builder.Services.AddShinyAiConversation(opts =>
{
    opts.SetChatClientProvider<MyChatClientProvider>();
});

3. Implement IMessageStore (optional)

Provide persistent storage for chat history. Without this, GetChatHistory and ClearChatHistory will throw.

using Shiny.AiConversation;

public class MyMessageStore : IMessageStore
{
    public Task Store(string? userTriggeringMessage, ChatResponse response, CancellationToken cancellationToken)
    {
        // Persist the complete AI response
    }

    public Task Clear(DateTimeOffset? beforeDate = null)
    {
        // Clear all or messages before the given date
    }

    public Task<IReadOnlyList<AiChatMessage>> Query(
        string? messageContains = null,
        DateTimeOffset? fromDate = null,
        DateTimeOffset? toDate = null,
        int? limit = null,
        CancellationToken cancellationToken = default)
    {
        // Query with optional filters, return ordered by timestamp
    }
}

4. Use the service

public class ChatViewModel(IAiConversationService aiService) : ObservableObject
{
    public async Task SendMessage(string text)
    {
        await aiService.TalkTo(text, CancellationToken.None);
    }

    public async Task UseMicrophone()
    {
        var access = await aiService.RequestAccess();
        if (access != AccessState.Available)
            return;

        await aiService.ListenAndTalk(CancellationToken.None);
    }

    public async Task StartWakeWord()
    {
        await aiService.StartWakeWord("Hey Assistant");
    }
}

API Overview

IAiConversationService

Acknowledgement Modes

ChatLookupAITool

When SetMessageStore<T>(addAiLookupTool: true) is used, the library registers an AITool named lookup_chat_history. This allows the AI to autonomously search past conversations when the user asks about previous discussions — no extra code required.

Architecture

┌─────────────────────────────────────────────────┐
│                   IAiConversationService                     │
│  (orchestrates chat, speech, sounds, history)    │
├──────────┬──────────────┬───────────────────────┤
│          │              │                       │
│  IChatClientProvider    │    IMessageStore       │
│  (default: resolves    │    (persistence)       │
│   IChatClient from DI) │                        │
│          │              │         │              │
│    IChatClient     ISpeechToText  │   ChatLookupAITool
│  (M.E.AI)          ITextToSpeech │   (optional AITool)
│                    (Shiny.Speech) │              │
│                         │         │              │
│                    IAudioPlayer   │              │
│                   (Shiny.Speech)  │              │
└─────────────────────────────────────────────────┘

Dependencies

License

MIT