🧠 LexiQ – Legal Document QA using LoRA-Finetuned Mistral-7B

Authors: Nishad Kookana, Ishita, Milan Agarwal, and Sushant Bhatia
License: Apache 2.0
Base model: mistralai/Mistral-7B-Instruct-v0.2

LexiQ is a domain-specific Legal Question-Answering model built on top of Mistral 7B. It is fine-tuned using LoRA on two legal datasets: CUAD (Contract Understanding Atticus Dataset) and SARA (Summarization of Appellate Court Rulings). The model supports question answering from legal documents with Retrieval-Augmented Generation (RAG).

🔍 Use Cases

✅ Direct Use:

Legal document understanding
Question answering over case files, legal summaries, contracts
Legal education, research, and policy drafting

❌ Out-of-Scope Use:

Not suitable for non-English documents
Not trained on international law — Indian and US law concepts dominate
Not intended for legal advice in real-world litigation

🧪 Model Details

Model Type: Causal Language Model (LLM) with LoRA fine-tuning
Languages: English
Quantization: 4-bit (bnb nf4) via bitsandbytes
Training Precision: Mixed FP16
Training Datasets: CUAD, SARA
Tokenizer: AutoTokenizer (fast)
Training Framework: Transformers + PEFT + Accelerate

📦 How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

tokenizer = AutoTokenizer.from_pretrained("NishKook/legal-qa-lora")
base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(base_model, "NishKook/legal-qa-lora", device_map="auto")

question = "What are the four elements of negligence?"
context = "Negligence requires a duty, a breach of that duty, causation, and damages."

prompt = f"### Question:\n{question}\n\n### Context:\n{context}\n\n### Answer:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

NishKook
/

legal-qa-lora