gweidart/rs-bpe

A ridiculously fast Python BPE (Byte Pair Encoder) implementation written in Rust

⭐ 41 ⑂ 4 Python MIT · 2025-03-19推送

https://pypi.org/project/rs-bpe/

huggingface llm openai pypi-package python rust tiktoken tokenizers bpe bpe-tokenizer byte-pair-encoding byte-pair-tokenizer

41

Watchers

0

贡献者

0

Commits

0

Releases

3

Open Issues

2025-03-19

最近推送

原文中文

暂无 README