unfair_tos

Task Description: Given a clause from a terms-of-service contract, determine the category the clause belongs to.
Task Type: 8-way classification
Document Type: contract clause
Number of Samples: 3822
Input Length Range: 7-371 tokens
Evaluation Metrics: accuracy (maximize), balanced_accuracy (maximize), f1_macro (maximize), f1_micro (maximize), valid_predictions_ratio (maximize)
Tags: contract law, interpretation
Paper: LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
Dataset Download: https://hazyresearch.stanford.edu/legalbench/

7 submissions

Rank	Model	accuracy	balanced_accuracy	f1_macro	f1_micro	valid_predictions_ratio	Date	Results
1	meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo	0.822	0.816	0.581	0.822	1.000	2025-07-25	View
2	google/gemma-2-27b-it	0.780	0.825	0.565	0.780	0.999	2025-07-24	View
3	gpt-4o-mini	0.726	0.833	0.454	0.726	0.997	2025-07-02	View
4	claude-3-5-haiku-20241022	0.690	0.872	0.446	0.690	0.999	2025-08-01	View
5	meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo	0.639	0.839	0.577	0.639	0.998	2025-08-04	View
6	gpt-4.1-nano	0.547	0.686	0.374	0.547	0.996	2025-07-03	View
7	claude-3-haiku-20240307	0.004	0.111	0.001	0.004	0.999	2025-07-25	View