AI for Lease Abstraction: Pulling Rent, Term and Renewal Clauses Without Getting Dates Wrong

Start with the extraction unit

Lease abstraction is often described as AI “reading” a lease and returning dates. The more useful unit is smaller: a labelled question applied to a particular passage, with the source span preserved—or an explicit absence.

CUAD, the expert-annotated contract-extraction dataset, first segments a contract into paragraphs typically ranging from one to five sentences. For every label category and paragraph, it asks the system to highlight any relevant text. The answer is the highlighted span and the character position where it begins, or the empty string when nothing in that paragraph should be highlighted.

That makes the extraction unit three connected parts:

1. The field label, such as Effective Date.
2. The paragraph in which the alleged supporting text appears.
3. The exact span supporting the field, or a no-answer result when the clause is absent.

CUAD’s lease-relevant labels illustrate how a broad concept such as “term” becomes structured data. Effective Date asks on what date the contract is effective. Expiration Date asks when the initial term expires. Renewal Term asks for the term after the initial term, including automatic extensions and unilateral extensions with prior notice. Notice to Terminate Renewal asks for the notice period required to terminate the renewal.

A rent field can be audited using the same provenance principle: identify the rent label, candidate paragraph and source span, or return no result if the passage does not support the field. However, the cited CUAD definitions do not include a rent-amount label. They support a method for examining rent extraction, not a measured rent-extraction score or error rate.

A date hit is not automatically the right field value

CUAD defines a true positive as a ground-truth text segment with a matching prediction. A false positive is a prediction that does not match any ground-truth segment. A false negative is a ground-truth segment without a matching prediction.

The match is determined using the Jaccard similarity coefficient:

J(A,B) = |A ∩ B| / |A ∪ B|

Here, A and B are sets of words in the annotation and extracted prediction. In other words, the benchmark’s “hit” is a span-overlap result. It is not a separate certification that a displayed date is the right date for the lease field.

That distinction matters because two questions are being conflated. “The system extracted a date” describes a non-empty output. “The system extracted the right date” additionally requires the supporting span to answer the correct labelled question. A date-like value does not establish that relationship by itself.

For example, the Expiration Date field must be traced to text answering when the initial term expires. The Effective Date field must be traced to the contract’s effectiveness date. The Notice to Terminate Renewal field calls for a notice period, not merely any date appearing near the renewal language. CUAD’s span metric measures whether predicted words overlap the relevant annotation; the field-level meaning still has to be checked against that annotation.

This is also why provenance should survive into the abstraction interface. A reviewer should be able to open the cited span beside the field rather than receive an unexplained calendar value.

The silent-clause failure: a populated field can be a false positive

The concrete failure is not always a badly formatted date. It is a populated field where the relevant clause is silent.

CUAD expressly represents this case with the empty string: if nothing should be highlighted for a label, the correct answer is no span. Under its scoring definitions, a prediction that does not match any ground-truth segment is a false positive. A non-empty renewal or notice value in that situation is therefore not a successful extraction; it is an unsupported output.

CUAD converts its data into the SQuAD 2.0 question-answering format. SQuAD 2.0 describes an unanswerable question as one where “the correct answer is not stated in the context” and says that systems must determine when no answer exists. Its dataset includes more than 50,000 adversarially written unanswerable questions designed to resemble answerable ones.

The same principle applies directly to lease fields. If the source does not state the requested date or period, the system should return an absent value, not manufacture a plausible one. The important operational distinction is:

• Correct abstention: the clause is silent, so the field is empty.
• False positive: the system supplies a value despite the absence of a matching ground-truth span.

Formatting that output as a valid-looking date does not cure the failure.

Precision and recall have a reviewer’s time cost

Precision and recall expose different weaknesses. Precision reflects predictions that match ground truth; recall reflects how much of the ground truth receives matching predictions. CUAD reports the expected trade-off: as recall increases, precision decreases. It specifically notes that precision at 90% recall is lower than precision at 80% recall.

The paper then gives a concrete illustration: at about 30% precision, “a lawyer would need to read through about 2 irrelevant clauses for every 1 relevant clause selected as important by the model.” This is CUAD’s worked example at a chosen recall level. It is not an AI Lawyer Bench measurement, a vendor result or a current lease-tool error rate.

The review-time implication is direct. False-positive extractions put irrelevant clauses in front of the lawyer, who must inspect text that does not answer the field. False negatives are different: the relevant ground-truth segment has no matching prediction. They represent omissions rather than irrelevant reading. A useful evaluation therefore cannot report recall while ignoring the precision cost of achieving it.

Length is a further reason not to transfer a short-clause result unchanged. LegalBench turns CUAD into per-clause-type binary classification. In its reported comparison, 38 short-clause CUAD items produced balanced accuracy of at least 88% for the large models evaluated, while GPT-4 fell to 47.8% on the longer MAUD contracts. These are LegalBench task results, not lease-date accuracy rates. LegalBench also states that it targeted questions with objectively correct answers and did not focus on long documents.

What to verify, field by field

A field-level review should answer five linked questions:

• Effective Date: Does the cited span state the date on which the contract is effective?
• Expiration Date: Does it state when the initial term expires, rather than merely containing another contract date?
• Renewal Term: Does it state the term following the initial term? If an extension mechanism matters to the field, does the span actually support the classification used?
• Notice to Terminate Renewal: Does the source give a notice period, and has that period been kept distinct from a calendar date?
• Rent: What exact span supports the amount or rent field, and has an inferred or bare value been presented as though it were extracted? The available evidence supplies no CUAD rent-amount benchmark from which to claim accuracy.

For every field, the lawyer should also confirm the paragraph, the label and the absence question. If the interface supplies no supporting span, the output has not passed the abstention test. If it supplies a span, the displayed value must still match what that span answers.

The available primary evidence does not publish per-tool lease date-extraction error rates. It therefore cannot support a ranking or a claim that a named product “gets dates wrong.” What it does support is a stricter acceptance test: every rent, term, renewal or notice field should be traceable to a labelled source span—or explicitly marked absent. A plausible value is only the beginning of verification, not the end.