Microsoft.ML.Tokenizers.Data.P50kBase
v3.0.0-preview.26513.4Latest Pre-release
net8.0.NETStandard2.0
The Microsoft.ML.Tokenizers.Data.P50kBase includes the Tiktoken tokenizer data file p50k_base.tiktoken, which is utilized by models such as text-davinci-002
Published
Oct 16, 2026
2d ago
Total Downloads
897.1K
897,071
Install
dotnet add package Microsoft.ML.Tokenizers.Data.P50kBase --version 3.0.0-preview.26513.4
| Product | Versions |
|---|---|
| .NET | |
| .NET Standard |
README
Changes from Previous Version
v3.0.0-preview.26457.2v3.0.0-preview.26513.4
Added Frameworks
net8.0
Added Dependencies
Microsoft.ML.Tokenizers(net8.0)
Adoption Assessment
A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.
This version is marked prerelease. Validate its API and upgrade path before production use.Compatibility metadata lists 2 target frameworks; confirm the exact target used by your application.
Related Guides & Workflows
Learn how to evaluate, integrate, or audit this package in your .NET project.