Microsoft.ML.Tokenizers.Data.R50kBase
v2.0.1Latest Stable
net8.0.NETStandard2.0
The Microsoft.ML.Tokenizers.Data.R50kBase includes the Tiktoken tokenizer data file r50k_base.tiktoken, which is utilized by models such as text-davinci-001
Published
Oct 15, 2026
4d ago
Total Downloads
33.3K
33,303
Install
dotnet add package Microsoft.ML.Tokenizers.Data.R50kBase --version 2.0.1
| Product | Versions |
|---|---|
| .NET | |
| .NET Standard |
README
Changes from Previous Version
v2.0.0v2.0.1
Added Frameworks
net8.0
Added Dependencies
Microsoft.ML.Tokenizers(net8.0)
Google.Protobuf(net8.0)
Adoption Assessment
A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.
Compatibility metadata lists 2 target frameworks; confirm the exact target used by your application.The published metadata declares 13 direct dependency entries across framework groups. Inspect the resolved dependency graph for your target framework.
Related Guides & Workflows
Learn how to evaluate, integrate, or audit this package in your .NET project.