Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ Jellyfish-13B is a large language model with 13 billion parameters, designed spe
|
|
13 |
|
14 |
We fine-tuned [Open-Orca/OpenOrca-Platypus2-13B](https://huggingface.co/Open-Orca/OpenOrca-Platypus2-13B) using the datasets related to data preprocessing tasks.
|
15 |
Its performance is competitive, standing up well against prior state-of-the-art algorithms and LLMs such as OpenAI GPT 3.5 and GPT 4 ([evaluated by our previous work](https://arxiv.org/abs/2308.16361)).
|
16 |
-
|
17 |
|
18 |
| Task | Dataset | Non-LLM SoTA<sup>1</sup> | GPT-3.5<sup>2</sup> | GPT-4<sup>2</sup> | Jellyfish-13B| Jellyfish-13B-Resoning |
|
19 |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- |
|
@@ -41,14 +41,13 @@ _For GPT-3.5, GPT-4 we used the few-shot approach, while for Jellyfish and Jelly
|
|
41 |
[Large Language Models as Data Preprocessors](https://arxiv.org/abs/2308.16361)
|
42 |
|
43 |
|
44 |
-
We release two versions of Jellyfish:
|
45 |
-
As the names suggest, Jellyfish-13B
|
46 |
-
In contrast, Jellyfish-13B-Reasoning
|
47 |
-
generated by GPT-4.
|
48 |
|
49 |
The two versions are designed for different application scenarios.
|
50 |
Jellyfish-13B is suitable for integration into larger data management systems due to its simple and clear responses that can be easily transformed into code.
|
51 |
-
On the other hand, Jellyfish-13B-Reasoning is more user-oriented, with responses that provide them with in-depth data insights without the necessity for advanced coding skills or an intricate grasp of statistics
|
52 |
|
53 |
**Jellyfish paper will be coming soon!**
|
54 |
|
@@ -71,7 +70,7 @@ On the other hand, Jellyfish-13B-Reasoning is more user-oriented, with responses
|
|
71 |
## Training Details
|
72 |
|
73 |
### Training Data
|
74 |
-
We utilized the training and validation sets from the paper [Can Foundation Models Wrangle Your Data?](https://arxiv.org/abs/2205.09911) to fine-tune Jellyfish
|
75 |
The original datasets is [HazyResearch/fm_data_tasks](https://github.com/HazyResearch/fm_data_tasks).
|
76 |
We revised this data and constructed an instruction tuning dataset suitable for fine-tuning LLM, mirroring the style of [OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca).
|
77 |
|
|
|
13 |
|
14 |
We fine-tuned [Open-Orca/OpenOrca-Platypus2-13B](https://huggingface.co/Open-Orca/OpenOrca-Platypus2-13B) using the datasets related to data preprocessing tasks.
|
15 |
Its performance is competitive, standing up well against prior state-of-the-art algorithms and LLMs such as OpenAI GPT 3.5 and GPT 4 ([evaluated by our previous work](https://arxiv.org/abs/2308.16361)).
|
16 |
+
Keep in mind that Jellyfish is only a 13B model, allowing for cost-effective local execution while maintaining data security.
|
17 |
|
18 |
| Task | Dataset | Non-LLM SoTA<sup>1</sup> | GPT-3.5<sup>2</sup> | GPT-4<sup>2</sup> | Jellyfish-13B| Jellyfish-13B-Resoning |
|
19 |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- |
|
|
|
41 |
[Large Language Models as Data Preprocessors](https://arxiv.org/abs/2308.16361)
|
42 |
|
43 |
|
44 |
+
We release two distinct versions of Jellyfish: Jellyfish-13B (the main branch) and Jellyfish-13B-Reasoning.
|
45 |
+
As the names suggest, Jellyfish-13B is tailored to deliver precise, straightforward answers.
|
46 |
+
In contrast, Jellyfish-13B-Reasoning, is fine-tuned with data that includes reasoning and sequential thought processes for handling data preprocessing tasks, distilling knowledge from GPT-4.
|
|
|
47 |
|
48 |
The two versions are designed for different application scenarios.
|
49 |
Jellyfish-13B is suitable for integration into larger data management systems due to its simple and clear responses that can be easily transformed into code.
|
50 |
+
On the other hand, Jellyfish-13B-Reasoning is more user-oriented, with responses that provide them with in-depth data insights without the necessity for advanced coding skills or an intricate grasp of statistics.
|
51 |
|
52 |
**Jellyfish paper will be coming soon!**
|
53 |
|
|
|
70 |
## Training Details
|
71 |
|
72 |
### Training Data
|
73 |
+
We utilized the training and validation sets from the paper [Can Foundation Models Wrangle Your Data?](https://arxiv.org/abs/2205.09911) to fine-tune Jellyfish.
|
74 |
The original datasets is [HazyResearch/fm_data_tasks](https://github.com/HazyResearch/fm_data_tasks).
|
75 |
We revised this data and constructed an instruction tuning dataset suitable for fine-tuning LLM, mirroring the style of [OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca).
|
76 |
|