Microsoft.ML.Tokenizers.Data.Cl100kBase

v2.0.0

The Microsoft.ML.Tokenizers.Data.Cl100kBase class includes the Tiktoken tokenizer data file cl100k_base.tiktoken, which is utilized by models such as GPT-4.

dotnetframework Microsoft MLNET

ML.NET ML Machine Learning

NuGet Stats Changelog

Total Downloads

2.5M

2,547,965

Published

Nov 11, 2025

Latest stable version

Install

.NET CLI

dotnet add package Microsoft.ML.Tokenizers.Data.Cl100kBase --version 2.0.0

Package Manager Console

Install-Package Microsoft.ML.Tokenizers.Data.Cl100kBase -Version 2.0.0

PackageReference

<PackageReference Include="Microsoft.ML.Tokenizers.Data.Cl100kBase" Version="2.0.0" />

Frameworks

.NETStandard2.0

Dependencies

Microsoft.ML.Tokenizers

.NETStandard2.0

[2.0.0, )

Google.Protobuf

.NETStandard2.0

[3.30.2, )

Microsoft.Bcl.AsyncInterfaces

.NETStandard2.0

[9.0.4, )

Microsoft.Bcl.HashCode

.NETStandard2.0

[6.0.0, )

Microsoft.Bcl.Memory

.NETStandard2.0

[9.0.4, )

System.Buffers

.NETStandard2.0

[4.6.1, )

System.IO.Pipelines

.NETStandard2.0

[9.0.4, )

System.Memory

.NETStandard2.0

[4.6.3, )

System.Runtime.CompilerServices.Unsafe

.NETStandard2.0

[6.1.2, )

System.Text.Encodings.Web

.NETStandard2.0

[9.0.4, )

System.Text.Json

.NETStandard2.0

[9.0.4, )

Adoption guide

A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.

Compatibility metadata lists 1 target framework; confirm the exact target used by your application.

How to read this guidance

Sources: published NuGet v3 registry metadata and explicit NuBrowse editorial guides. Refresh: package metadata is refreshed from NuGet when the page loads. Limit: this is not a security, legal or compatibility guarantee; validate the selected version in your project.

Useful next steps

Move from package metadata to a concrete selection or review workflow.

Audit NuGet dependencies in a .NET project Review NuGet licenses before shipping