Microsoft.ML.Tokenizers.Data.Cl100kBase
The Microsoft.ML.Tokenizers.Data.Cl100kBase class includes the Tiktoken tokenizer data file cl100k_base.tiktoken, which is utilized by models such as GPT-4.
Install
dotnet add package Microsoft.ML.Tokenizers.Data.Cl100kBase --version 1.0.3
Install-Package Microsoft.ML.Tokenizers.Data.Cl100kBase -Version 1.0.3
<PackageReference Include="Microsoft.ML.Tokenizers.Data.Cl100kBase" Version="1.0.3" />
Frameworks
Dependencies
Adoption guide
A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.
Compatibility metadata lists 1 target framework; confirm the exact target used by your application.
How to read this guidance
Sources: published NuGet v3 registry metadata and explicit NuBrowse editorial guides. Refresh: package metadata is refreshed from NuGet when the page loads. Limit: this is not a security, legal or compatibility guarantee; validate the selected version in your project.
Useful next steps
Move from package metadata to a concrete selection or review workflow.