AIforIP · Benchmark Collection
IPIntelligenceArena brings together research-backed benchmarks for evaluating large language models across intellectual property knowledge, practice, and patent examination.
01 / Overview
IPIntelligenceArena organizes the team’s public benchmarks along a progression of IP capabilities. IPEval tests bilingual knowledge and consultation, IPBench expands evaluation to legal and technical practice tasks, and PatRe moves into multi-round patent examination, where evidence, procedural context, and document generation must work together.
Together, they show how evaluation is moving from knowing IP rules to completing context-dependent professional work.
02 / Benchmarks
Each benchmark is presented with its original paper, code, data, and project resources.
A Full-Stage Office Action and Rebuttal Generation Benchmark for Patent Examination
PatRe models patent examination as an iterative process of examiner analysis and applicant response. It evaluates Office Action and rebuttal generation under direct, oracle-reference, and retrieval-simulated settings using recent USPTO cases.
Benchmarking Large Language Models on Intellectual Property Knowledge and Practice
IPBench introduces a four-level task taxonomy grounded in Depth of Knowledge. It covers technical and legal tasks across information processing, reasoning, discriminant evaluation, and creative generation in real-world IP practice.
A Bilingual Intellectual Property Agency Consultation Evaluation Benchmark for Large Language Models
IPEval evaluates how well language models understand and apply intellectual property knowledge in agency and consultation settings. Its bilingual questions span the creation, application, protection, and management of intellectual property.
03 / Scope
The collection spans complementary evaluation units, task formats, and professional contexts.
04 / Papers
Read the original work for task construction, evaluation protocols, experimental settings, findings, and limitations.
Qiyao Wang, Xinyi Chen, Longze Chen, Hongbo Wang, Hamid Alinejad-Rokny, Yuan Lin, and Min Yang.
arXiv:2605.03571 ↗Qiyao Wang et al. · ACL 2026.
arXiv:2504.15524 ↗Qiyao Wang, Jianguo Huang, Shule Lu, Yuan Lin, Kan Xu, Liang Yang, and Hongfei Lin.
arXiv:2406.12386 ↗05 / About
IPIntelligenceArena is AIforIP’s public benchmark collection for evaluating large language models across intellectual property knowledge, practice, and patent examination.
Benchmark descriptions and figures on this site are based on the corresponding papers and public repositories. Please consult the original resources for complete methodological and licensing details.
Visit AIforIP on GitHub ↗