net8.0.NETStandard2.0
The Microsoft.ML.Tokenizers.Data.P50kBase includes the Tiktoken tokenizer data file p50k_base.tiktoken, which is utilized by models such as text-davinci-002
Published
Oct 15, 2026
1mo ago
Total Downloads
1.0M
1,005,786
Install
dotnet add package Microsoft.ML.Tokenizers.Data.P50kBase --version 2.0.1
| Product | Versions |
|---|---|
| .NET | |
| .NET Standard |
README
Changes from Previous Version
v2.0.0v2.0.1
Added Frameworks
net8.0
Added Dependencies
Microsoft.ML.Tokenizers(net8.0)
Google.Protobuf(net8.0)
Adoption Assessment
A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.
Compatibility metadata lists 2 target frameworks; confirm the exact target used by your application.The published metadata declares 13 direct dependency entries across framework groups. Inspect the resolved dependency graph for your target framework.
Related Guides & Workflows
Learn how to evaluate, integrate, or audit this package in your .NET project.