INDUSTRY FIELD GUIDE
Law firm contract redline and approval evidence for AI
Legal AI researchers need tasks with a reviewable outcome.
NAICS 541110 · Offices of Lawyers
Legal AI researchers need tasks with a reviewable outcome. Scale Labs published PRBench with expert-authored legal and finance tasks and rubrics. OpenAI and Ironclad describe a research collaboration on contracting workflows. These examples show two kinds of demand for legal expertise, benchmark design and workflow evaluation. Scale PRBench · OpenAI and Ironclad collaboration
Draft an original fictional services agreement with two redlines. Approve one only for a named contract type in the exercise. Ask the model to cite which wording it can use and flag the other as pending. Have a lawyer score text accuracy, effective dates and escalation separately.
Use independently written facts and clauses. Exclude client files, privileged communications, work product and proprietary forms. ABA guidance discusses confidentiality and other duties when lawyers use generative AI. ABA Formal Opinion 512
Describe an original legal evaluation sample
Original Fiund exercise
An approved redline has a contract-type and effective-date scope
An original fictional services clause uses neutral administrative labels. The exercise retrieves approvals rather than drafting advice for a real client or determining legal enforceability.
| Record | Source | What it records |
|---|---|---|
| LFA1 | Practice baseline clause | Administrative report due day 10 for contract types A and B. |
| LFA2 | Approved redline R1 | Changes the date to day 15 for type A only, effective on practice day 5. |
| LFA3 | Proposed redline R2 | Would change both types to day 20; no approval. Review request asks about type B on day 6. |
Question for the model
Which supplied date is supported for type B, and why do the type-A approval or broader draft not establish day 15 or 20?
Show the reference answer and checks
LFA1 remains the supplied type-B baseline of day 10. LFA2 authorizes day 15 for type A only, so its effective day does not broaden contract-type scope. LFA3 is unapproved and supplies no type-B change. Cite the baseline and the limited approval, flag the pending proposal and leave actual clause use or interpretation to authorized legal review rather than borrowing an approved redline from another contract category.
- Extending a contract-type-specific approval to every agreement.
- Treating an unapproved redline as usable language because it appears in the latest draft.
Fields to preserve
- Clause and contract type
- Defines the requested text scope.
- Redline approval and effective date
- Limits the authorized change.
- Pending proposal and reviewer handoff
- Preserves the unresolved alternative.
Test another case
Ask about type A before day 5. Require the original date despite the later approved redline's presence in the packet.
Use independently authored clauses and facts. Exclude clients, privileged communications, work product and proprietary forms.
Discuss a contribution
Describe the sources you control, the task they could support and what has already been checked. Use the existing-data assessment for an archive or the collection brief for a new recording project. The contributor agreement defines Fiund’s licensing authority and compensation. A guide or fictional exercise does not establish an accepted collection or a sale.