net8.0.NETStandard2.0
Microsoft.ML.Tokenizers contains the implmentation of the tokenization used in the NLP transforms.
Published
Oct 15, 2026
1mo ago
Total Downloads
20.0M
20,032,545
Install
dotnet add package Microsoft.ML.Tokenizers --version 2.0.1
| Product | Versions |
|---|---|
| .NET | |
| .NET Standard |
README
Changes from Previous Version
No structural changes between v2.0.0 and v2.0.1.
Adoption Assessment
A verified publisher signal is available. Assess fit from the package's supported frameworks, license and dependency graph rather than popularity alone.
Compatibility metadata lists 2 target frameworks; confirm the exact target used by your application.
Related Guides & Workflows
Learn how to evaluate, integrate, or audit this package in your .NET project.