Skip to main content
N
0.35.10.36.0
LMSupply.Llama package icon

LMSupply.Llama

v0.35.2
net10.0

llama.cpp server management for LMSupply. Provides llama-server downloading, GPU detection, and server pooling for GGUF model inference.

iyulab-rnd
NuGetWebsiteMITCompare
Published
Jul 17, 2026
3mo ago
Total Downloads
28.1K
28,053

Install

dotnet add package LMSupply.Llama --version 0.35.2
Download NuGet FileBrowse File Contents
ProductVersions
.NET
net10.0

README

LMSupply.Llama

Shared llama-server management for LMSupply libraries.

Features

  • On-demand binary downloading: Native binaries are downloaded on first use
  • GPU auto-detection: Automatic selection of CUDA, Vulkan, Metal, or CPU backend
  • Fallback chain: CUDA 13 → CUDA 12 → Vulkan → CPU (platform-dependent)
  • Caching: Downloaded binaries are cached locally for reuse
  • Server pooling: Efficient server instance reuse
  • Mode-aware pooling: Separate servers for generation, embedding, and reranking

Supported Backends

Backend Platform GPU
CUDA 12/13 Windows, Linux NVIDIA
Vulkan Windows, Linux AMD, Intel, NVIDIA
Metal macOS Apple Silicon
Hip/ROCm Windows, Linux AMD
CPU All AVX2/AVX512 optimized

Architecture

LMSupply.Llama uses llama-server as a unified backend:

LMSupply.Llama/Server/
├── LlamaServerDownloader.cs   - Downloads from GitHub releases
├── LlamaServerPool.cs         - Server instance pooling
├── LlamaServerProcess.cs      - Process lifecycle management
├── LlamaServerClient.cs       - HTTP API client (embeddings, completions, rerank)
└── LlamaServerStateManager.cs - Health and state tracking

Server Modes

Mode Flag Use Case
Generation (default) Text generation
Embedding --embedding Vector embeddings
Reranking --embedding --pooling rank Document reranking

Usage

This package is used internally by:

Package Server Mode Use Case
LMSupply.Generator Generation GGUF text generation
LMSupply.Embedder Embedding GGUF embeddings
LMSupply.Reranker Reranking GGUF reranking

Changes from Previous Version

No structural changes between v0.35.1 and v0.35.2.

Adoption Assessment

Assess this package against its supported frameworks, license, maintenance signals and dependency graph before making it a project standard.

Compatibility metadata lists 1 target framework; confirm the exact target used by your application.

Related Guides & Workflows

Learn how to evaluate, integrate, or audit this package in your .NET project.

Audit NuGet dependencies in a .NET projectReview NuGet licenses before shipping