Microsoft.ML.Tokenizers.Data.Cl100kBase
v3.0.0-preview.26513.4Latest Pre-release
net8.0.NETStandard2.0
The Microsoft.ML.Tokenizers.Data.Cl100kBase class includes the Tiktoken tokenizer data file cl100k_base.tiktoken, which is utilized by models such as GPT-4.
Published
Oct 16, 2026
3d ago
Total Downloads
3.4M
3,371,594
Install
dotnet add package Microsoft.ML.Tokenizers.Data.Cl100kBase --version 3.0.0-preview.26513.4
| Product | Versions |
|---|---|
| .NET | |
| .NET Standard |
README
Changes from Previous Version
v3.0.0-preview.26457.2v3.0.0-preview.26513.4
Added Frameworks
net8.0
Added Dependencies
Microsoft.ML.Tokenizers(net8.0)
Adoption Assessment
A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.
This version is marked prerelease. Validate its API and upgrade path before production use.Compatibility metadata lists 2 target frameworks; confirm the exact target used by your application.
Related Guides & Workflows
Learn how to evaluate, integrate, or audit this package in your .NET project.