byte-pair-tokenizer

1 个项目 · ⭐ 41

gweidart/rs-bpe

A ridiculously fast Python BPE (Byte Pair Encoder) implementation written in Rust

Python ⭐ 41 ⑂ 4 MIT · 2025-03-19推送

huggingface llm

2025-03-19