Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
yhavinga
/
dutch-tokenizer-arena
like
8
Running
App
Files
Files
Community
1
Fetching metadata from the HF Docker repository...
b514440
dutch-tokenizer-arena
/
vocab
/
gpt_nexo_20b
Ctrl+K
Ctrl+K
3 contributors
History:
2 commits
xu-song
update
d10ecd7
over 1 year ago
tokenizer
update
over 1 year ago
tokenzier_hf
update
over 1 year ago
20B_tokenizer.json
Safe
2.47 MB
update
over 1 year ago
20B_tokenizer.txt
Safe
4.04 MB
update
over 1 year ago
20B_tokenizer.zh.json
Safe
2.11 MB
update
over 1 year ago
README.md
Safe
1.56 kB
update
over 1 year ago
__init__.py
Safe
717 Bytes
update
over 1 year ago
convert_vocab_to_txt.py
Safe
459 Bytes
update
over 1 year ago
test_gpt_neox_20b.py
Safe
2.96 kB
update
over 1 year ago
test_hf_gpt_neox.py
Safe
487 Bytes
update
over 1 year ago
test_oov.py
Safe
505 Bytes
update
over 1 year ago
test_special_token.py
Safe
340 Bytes
update
over 1 year ago
test_tokenizer.py
Safe
1.94 kB
update
over 1 year ago
test_tokenizer_HF.py
Safe
1.3 kB
update
over 1 year ago
test_zh_coding_len.py
Safe
447 Bytes
update
over 1 year ago
vocab.zh.txt
Safe
9.34 kB
update
over 1 year ago
