konradkokosa
NuGet publisher profile
Packages (13)
Mobius.ILasm
Mobius.ILasm is a Common Intermediate Language (CIL) assembler available as a library, based on Mono assembler. It is used for testing in Mobius - .NET runtime written entirely in C#
Strings.Abbreviations
A set of predefined static methods for popular abbreviations. While this is (barely) useful by itself, it mostly serves as: Minimum Viable Product (or example) of creating your own library that is automatically tested/versioned and published as a NuGet package and playground for fresh open source contributors.
DotLLM.Engine
Inference engine for dotLLM — KV-cache (paged / quantized), scheduler, composable samplers, constrained decoding (JSON / regex / grammar), speculative decoding, and prefix caching.
DotLLM.Core
Core abstractions for dotLLM — tensor types, backend interfaces, model config, sampling, attention strategies, and diagnostics hooks.
DotLLM.Cpu
CPU backend for dotLLM — SIMD-optimized tensor kernels (TensorPrimitives + hand-tuned intrinsics) for quantized matmul, RMSNorm, RoPE, softmax, and attention.
DotLLM.Models
Model loaders and transformer architectures for dotLLM — memory-mapped GGUF/SafeTensors, parameterized TransformerBlock supporting Llama, Mistral, Phi, Qwen, and DeepSeek.
DotLLM.Cli
Command-line tool for dotLLM — run, chat, serve, and manage GGUF models. Install with: dotnet tool install -g DotLLM.Cli
DotLLM.Tokenizers
Tokenizers and chat templates for dotLLM — BPE, SentencePiece, HuggingFace tokenizer.json, and a Jinja2-subset chat template interpreter.
DotLLM.HuggingFace
HuggingFace Hub integration for dotLLM — model search, metadata queries, and GGUF download/caching under ~/.dotllm/models/.
DotLLM.Cuda
CUDA GPU backend for dotLLM — PTX kernels loaded via the CUDA Driver API for attention, dequant, matmul, RoPE, softmax, and RMSNorm. Supports CPU/GPU hybrid offloading and KV-cache quantization.
DotLLM.Telemetry
Telemetry and observability for dotLLM — System.Diagnostics.Metrics counters, Activity-based request tracing, and OpenTelemetry-friendly instrumentation.
DotLLM.Server
OpenAI-compatible HTTP API server for dotLLM — /v1/chat/completions, /v1/completions, streaming, tool calling, and a built-in chat UI. Reference this package to host the API inside your own ASP.NET Core app.
DotLLM.Diagnostics
Interpretability and diagnostics for dotLLM — zero-cost inference hooks for activation capture, logit lens, logprobs, and SAE integration.