FlowReasoner: Reinforcing Query-Level Meta-Agents
Paper
•
2504.15257
•
Published
•
43
Sea Language Team
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
sail-lm's activity
dreamerdeo
authored
a
paper
5 days ago
dreamerdeo
authored
a
paper
9 days ago
dreamerdeo
authored
2
papers
11 days ago
SivilTaram
authored
7
papers
about 1 month ago
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
Paper
•
2411.07763
•
Published
•
2
When Attention Sink Emerges in Language Models: An Empirical View
Paper
•
2410.10781
•
Published
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
Paper
•
2502.12982
•
Published
•
17
Predictive Data Selection: The Data That Predicts Is the Data That Teaches
Paper
•
2503.00808
•
Published
•
57
Scaling up Masked Diffusion Models on Text
Paper
•
2410.18514
•
Published
SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
Paper
•
2503.18892
•
Published
•
30
SkyLadder: Better and Faster Pretraining via Context Window Scheduling
Paper
•
2503.15450
•
Published
•
11
SivilTaram
authored
a
paper
2 months ago
dreamerdeo
posted
an
update
2 months ago
Post
2833
🚀 Excited to share our technical report on the Southeast Asian multilingual model Sailor2 and its latest updates!
Our 49-page report details Sailor2's development journey, including multilingual data cleaning, small model data mixture simulations, multi-stage continual pre-training, multi-stage post-training, and multi-cultural multi-lingual evaluations. Sailor2 aims to streamline the multilingual model pre-training process efficiently for the community.
🧭 We highlight Sailor2's impressive performance in low-resource language translation scenarios and its cultural understanding advantages in Southeast Asia, promoting practical applications for regional languages.
Model updates include:
💡 More precise outputs: Reduced redundancy in model outputs through refined post-training data and optimization techniques.
🌈 Handling longer texts: Expanded to handle up to 128K context length in Southeast Asian languages through long-text training.
⚡️ Faster inference: Achieved 2.5x faster inference speed with speculative decoding.
🌪️ More model sizes: Introduced new sizes of 3B and 14B through model pruning.
🌟 All models are Apache-licensed for commercial use; development tools (code, resources) are open-source.
📚 Technical report: Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs (2502.12982)
🤖️ Models: sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b
💬 Demo: sail/Sailor2-20B-Chat
📣 Sailor2 community:
sailor2
Our 49-page report details Sailor2's development journey, including multilingual data cleaning, small model data mixture simulations, multi-stage continual pre-training, multi-stage post-training, and multi-cultural multi-lingual evaluations. Sailor2 aims to streamline the multilingual model pre-training process efficiently for the community.
🧭 We highlight Sailor2's impressive performance in low-resource language translation scenarios and its cultural understanding advantages in Southeast Asia, promoting practical applications for regional languages.
Model updates include:
💡 More precise outputs: Reduced redundancy in model outputs through refined post-training data and optimization techniques.
🌈 Handling longer texts: Expanded to handle up to 128K context length in Southeast Asian languages through long-text training.
⚡️ Faster inference: Achieved 2.5x faster inference speed with speculative decoding.
🌪️ More model sizes: Introduced new sizes of 3B and 14B through model pruning.
🌟 All models are Apache-licensed for commercial use; development tools (code, resources) are open-source.
📚 Technical report: Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs (2502.12982)
🤖️ Models: sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b
💬 Demo: sail/Sailor2-20B-Chat
📣 Sailor2 community:
sailor2
SivilTaram
authored
3
papers
5 months ago
When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
Paper
•
2411.13476
•
Published
•
16
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
Paper
•
2407.10956
•
Published
•
7
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
Paper
•
2411.04905
•
Published
•
125
SivilTaram
authored
2
papers
7 months ago
Xalphinions
authored
a
paper
7 months ago
SivilTaram
authored
a
paper
9 months ago