|
--- |
|
license: apache-2.0 |
|
language: en |
|
tags: |
|
- llama |
|
- instruction-residual |
|
- parameter-efficient |
|
- safetensors |
|
- transformers |
|
base_model: |
|
- meta-llama/Llama-3.1-8B |
|
- meta-llama/Llama-3.1-8B-Instruct |
|
--- |
|
|
|
# Llama-3.1-8b-Instruct-Residual |
|
|
|
**Full-rank instruction residual for Llama-3.1-8B** |
|
|
|
This repository provides the **full-rank instruction residual** \(Δθ = θ_{instruct} - θ_{base}\) between the instruction-tuned Llama-3.1-8B-Instruct model and its corresponding base Llama-3.1-8B model. By adding this residual to a fresh base checkpoint, you can restore instruction-following capabilities **without** running a full fine-tuning cycle. |
|
|
|
## How it was created |
|
|
|
We follow the *instruction residual* approach introduced by Jindal et al. (2024): |
|
|
|
> “In this section, we describe the instruction residual approach to simply regain the instruction following capabilities. We compute the instruction residual between an instruction following LLM \(θ_{i,d_1,v_1}\) and its corresponding base model \(θ_{b,d_1}\) in the parametric space as |
|
> \[ |
|
> Θ_{r,v_1} = θ_{i,d_1,v_1} - θ_{b,d_1}. |
|
> \] |
|
> This tensor subtraction extracts the instruction-specific information, which can then be added to any base model.” |
|
|
|
The full paper is available at: https://arxiv.org/abs/2410.10739 |
|
|
|
## Files |
|
|
|
- `pytorch_model.safetensors` — full-rank FP16 residual weights (~16 GB). |
|
- `config.json` — configuration matching the Llama-3.1-8B architecture. |
|
- `README.md` — this model card. |
|
|
|
## Usage |
|
|
|
Below is a minimal example showing how to apply the residual to a base model: |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM |
|
from safetensors.torch import load_file |
|
import torch |
|
|
|
# 1) Load base |
|
model = AutoModelForCausalLM.from_pretrained( |
|
"meta-llama/Llama-3.1-8B", |
|
torch_dtype=torch.float16, |
|
device_map="auto", |
|
) |
|
|
|
# 2) Load residual |
|
residual_sd = load_file("pytorch_model.safetensors", device="cpu") |
|
|
|
# 3) Apply residual |
|
for name, delta in residual_sd.items(): |
|
param = dict(model.named_parameters())[name] |
|
param.data += delta.to(param.device).to(param.dtype) |
|
|
|
# 4) Save or push |
|
model.save_pretrained("llama-3.1-8b-base-plus-instruct") |
|
``` |
|
|
|
For full scripts, see the `examples/` folder. |
|
|
|
## Intended Use & Limitations |
|
|
|
- **Intended Use**: Add instruction-following capabilities to Llama-3.1-8B base models. |
|
- **Limitations**: |
|
- Residual must match the exact base checkpoint. |
|
- Stored in FP16 (~16 GB); dequantization needed if working in 4-bit. |
|
- Applying to mismatched architectures will produce invalid weights. |
|
|
|
## License |
|
|
|
This residual is released under the **Apache License 2.0**. See the `LICENSE` file for details. |
|
|
|
## References |
|
As mentioned before this method was introduced by **Jindal et al., 2024**, arXiv:2410.10739.: |
|
|
|
```bibtex |
|
@misc{jindal2024balancingcontinuouspretraininginstruction, |
|
title={Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs}, |
|
author={Ishan Jindal and Chandana Badrinath and Pranjal Bharti and Lakkidi Vinay and Sachin Dev Sharma}, |
|
year={2024}, |
|
eprint={2410.10739}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL}, |
|
url={https://arxiv.org/abs/2410.10739}, |
|
} |
|
``` |
|
|
|
|
