maud_buyer_consent_requirement_(ordinary_course)
- Task Description: The model must determine if there are any limitations on the Buyer’s right to condition, withhold, or delay their consent regarding the acquired company’s ordinary business operations based on an excerpt from a merger agreement.
- Task Type: Binary classification
- Document Type: merger agreement
- Number of Samples: 182
- Input Length Range: 33-591 tokens
- Evaluation Metrics: accuracy (maximize), balanced_accuracy (maximize), f1_macro (maximize), f1_micro (maximize), valid_predictions_ratio (maximize)
- Tags: corporate law, interpretation
- Paper: LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- Dataset Download: https://hazyresearch.stanford.edu/legalbench/
7 submissions
| Rank | Model | accuracy | balanced_accuracy | f1_macro | f1_micro | valid_predictions_ratio | Date | Results |
|---|---|---|---|---|---|---|---|---|
| 1 | meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo | 0.912 | 0.556 | 0.577 | 0.912 | 1.000 | 2025-07-25 | View |
| 2 | meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo | 0.906 | 0.528 | 0.528 | 0.906 | 1.000 | 2025-08-03 | View |
| 3 | claude-3-5-haiku-20241022 | 0.901 | 0.500 | 0.474 | 0.901 | 1.000 | 2025-08-01 | View |
| 4 | gpt-4o-mini | 0.901 | 0.500 | 0.474 | 0.901 | 1.000 | 2025-07-02 | View |
| 5 | google/gemma-2-27b-it | 0.901 | 0.500 | 0.474 | 0.901 | 1.000 | 2025-07-24 | View |
| 6 | gpt-4.1-nano | 0.901 | 0.500 | 0.474 | 0.901 | 1.000 | 2025-07-03 | View |
| 7 | claude-3-haiku-20240307 | 0.901 | 0.500 | 0.474 | 0.901 | 1.000 | 2025-07-28 | View |