GHOSTSONA / umt5-base /tokenizer.json
ghostai1's picture
Upload 13 files
6fae2ec verified
raw
history blame
364 Bytes
{
"version": "1.0",
"model": "UMT5 for PhantomStep",
"description": "Tokenizer vocabulary for PhantomStep by GhostAI",
"vocab": {
"<s>": 0,
"</s>": 1,
"<unk>": 2,
"<pad>": 3,
"[PhantomStep]": 250112,
"...": "..."
},
"added_tokens": [
{
"id": 250112,
"content": "[PhantomStep]",
"special": true
}
]
}