.NETStandard2.0
The Microsoft.ML.Tokenizers.Data.R50kBase includes the Tiktoken tokenizer data file r50k_base.tiktoken, which is utilized by models such as text-davinci-001
Published
Feb 28, 2025
1y ago
Total Downloads
33.3K
33,303
Install
dotnet add package Microsoft.ML.Tokenizers.Data.R50kBase --version 2.0.0-preview.1.25127.4
| Product | Versions |
|---|---|
| .NET Standard |
README
Changes from Previous Version
No structural changes between v2.0.0-preview.1.25125.4 and v2.0.0-preview.1.25127.4.
Adoption Assessment
A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.
This version is marked prerelease. Validate its API and upgrade path before production use.Compatibility metadata lists 1 target framework; confirm the exact target used by your application.
Related Guides & Workflows
Learn how to evaluate, integrate, or audit this package in your .NET project.