housing_qa_statute_comprehension
- Task Description: Answer questions about eviction law according to state law in 2021, given the relevant statute text.
- Task Type: Binary classification
- Document Type: legal question
- Number of Samples: 6853
- Input Length Range: 125-7141 tokens
- Evaluation Metrics: accuracy (maximize), f1_macro (maximize)
- Tags: eviction law, interpretation
- Paper: A Reasoning-Focused Legal Retrieval Benchmark
- Dataset Download: https://huggingface.co/datasets/reglab/housing_qa
9 submissions
| Rank | Model | accuracy | f1_macro | Date | Results |
|---|---|---|---|---|---|
| 1 | claude-sonnet-4-20250514 | 0.799 | 0.785 | 2025-08-04 | View |
| 2 | claude-opus-4-20250514 | 0.788 | 0.780 | 2025-08-04 | View |
| 3 | o3-mini | 0.775 | 0.756 | 2025-08-04 | View |
| 4 | gpt-5-2025-08-07 | 0.773 | 0.766 | 2025-08-08 | View |
| 5 | o4-mini | 0.762 | 0.751 | 2025-08-04 | View |
| 6 | o3 | 0.760 | 0.754 | 2025-08-05 | View |
| 7 | claude-3-5-haiku-20241022 | 0.727 | 0.715 | 2025-08-04 | View |
| 8 | gpt-4o-mini-2024-07-18 | 0.720 | 0.714 | 2025-08-04 | View |
| 9 | openai/gpt-oss-120b | 0.542 | 0.528 | 2025-08-07 | View |