File size: 1,882 Bytes
ef00501 bf40d72 8039e0f 63d3654 8039e0f 95a22f2 8039e0f 95a22f2 63d3654 3dea359 3bb9dd5 3dea359 bca548f 3dea359 4fb0731 3dea359 8039e0f b2d7b35 8039e0f bf40d72 2567e8d fce1e64 6d76429 7c9e1ea 6bd33ac c003236 6bd33ac |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
---
license: apache-2.0
datasets:
- BAAI/COIG-PC
- ehartford/dolphin
- emozilla/booksum-summary-analysis_llama-8192
- OpenLeecher/GPT4-10k
- 0x70DA/stackoverflow-chat-data
- togethercomputer/Long-Data-Collections
---
## RWKV 7B world focus on reading comprehension
This is a experimental model based on RWKV 7B world.
why this model is special? ===>
remove eod, add special token, change vocabs.
this model is used to QA in large texts, do some in context learning with knowledge indexed database.
## trainning details
train with this kind of new format,
```<s>User: <sys>xxxx\n\n</sys>xxxxx\n\n</s><s>Assistant: xxxxx\n\n</s><s>User: xxxx\n\n</s><s>Assistant: \n\n</s>```
so ,use User Assistant as your prefix names.
and when inference in RWKV runner, just use the following format is fine.
User: xxxx\n\nAssistant: xxxx\n\n，in which are the test cases used.
--------------------------------------------
to use this model with RWKV runner,some effort needed, copy back-python folder to a new one ,which is in the same folder with rwkv-runner.exe(or the file to run) , then pastin rwkv_vocab_v20230424.txt into rwkv_pip folder to replace the vocabs file
../py310/python main.py in this new folder, then use RWKV runner setting API to 127.0.0.0.1:8000, and go to 127.0.0.1:8000/docs to switch model using this one
try different temp and topp , 1.2 0.5 may works.
temp 1.2 topp 0.6
|