Microsoft.ML.Tokenizers.Data.P50kBase
v3.0.0Latest Stable
net8.0.NETStandard2.0
The Microsoft.ML.Tokenizers.Data.P50kBase includes the Tiktoken tokenizer data file p50k_base.tiktoken, which is utilized by models such as text-davinci-002
Published
Nov 11, 2026
5d ago
Total Downloads
1.0M
1,005,786
Install
dotnet add package Microsoft.ML.Tokenizers.Data.P50kBase --version 3.0.0
| Product | Versions |
|---|---|
| .NET | |
| .NET Standard |
README
Changes from Previous Version
No structural changes between v3.0.0-preview.26513.4 and v3.0.0.
Adoption Assessment
A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.
Compatibility metadata lists 2 target frameworks; confirm the exact target used by your application.
Related Guides & Workflows
Learn how to evaluate, integrate, or audit this package in your .NET project.