Generative artificial intelligence (AI) is rapidly becoming part of the legal landscape, and the rules governing its use are developing just as quickly. Recent decisions reveal two distinct—and increasingly important—areas of risk for attorneys, employers, and litigants:
- Whether communications with AI tools and AI-assisted materials are protected from discovery; and
- The potential consequences when AI-generated hallucinations make their way into court filings.
This article highlights some recent cases addressing these emerging issues. While these decisions do not establish bright-line rules for AI use, they make one point clear: The benefits and risks associated with AI depend largely on how lawyers and clients use the technology and evaluate and act on its output.
Privilege and Work Product: Protection Turns on Who Uses AI—and How
Courts have not adopted a uniform rule for determining whether AI-assisted materials are protected from discovery. Instead, the emerging cases show that whether AI-related information is protected from discovery turns heavily on the circumstances surrounding its use, including whether the individual was represented by counsel, whether counsel directed or participated in the use, the platform’s privacy policies and safeguards, and what information was disclosed to the AI platform. The decisions below illustrate there is no simple answer to whether AI use destroys—or preserves—privilege and work-product protections.
United States v. Heppner (Southern District of New York). After Heppner learned he was the target of a criminal investigation, he retained counsel and, thereafter, independently conducted legal research using the non-enterprise, consumer version of AI tool Claude. Heppner entered information he learned from his counsel into Claude, which generated reports outlining potential legal arguments and defense strategies. Heppner later shared those reports with his counsel, who considered them in developing the defense strategy.
After the government seized devices containing Hepper’s exchanges with Claude, Heppner asserted that the AI-generated materials were protected by the attorney-client privilege and work product doctrine because they were created in anticipation of indictment and for the purpose of communicating with counsel. The government moved for a determination that neither doctrine applied.
The court granted the government’s motion, finding:
- No attorney-client privilege. Claude is not an attorney, and an AI tool cannot fulfill this role.
- No reasonable expectation of confidentiality. Claude’s privacy policy states that data may be retained for training purposes and may be disclosed to third parties, including governmental authorities.
- No work-product protection. The materials were not “prepared by or at the behest of counsel” and did not reflect counsel’s strategy at the time of creation.
- No communication for purposes of obtaining legal advice. Claude expressly disclaims providing legal advice, and the fact that Heppner shared the documents with counsel does not create privilege.
The court did not address whether use of a secured AI tool could affect privilege, or whether research done at the direction of counsel might qualify for work-product protection.
Warner v. Gilbarco, Inc. (Eastern District of Michigan). This case involves a pro se plaintiff who used AI tool ChatGPT to draft legal filings. Defendants moved to compel production of “all documents and information concerning her use of third-party AI tools in connection with this lawsuit[,]” arguing that plaintiff waived any work-product protection by sharing information with the AI tool.
The court denied the motion, reasoning:
- Work-product protection may apply to AI-assisted materials. A pro se litigant may assert work-product protection and such protections cover the AI-generated materials because they were prepared in anticipation of litigation or for trial.
- Using an AI tool did not waive work-product protection. AI programs should not be treated as a third party for waiver purposes because they are tools, not persons, “even if they may have administrators somewhere in the background.”
The court noted that ruling otherwise “would nullify work-product protection in nearly every modern drafting environment, a result no court has endorsed.”
Morgan v. V2X Inc. (District of Colorado). In this case, a pro se plaintiff and the defendant disagreed on the use of AI tools in connection with confidential information. The defendant sought an amended protective order restricting the use of AI with confidential information and an order requiring the plaintiff to identify the AI tool used to evaluate whether appropriate safeguards existed to protect its confidential information. The plaintiff opposed disclosure, arguing that information regarding his AI use was protected by the work product doctrine.
The court granted the motion in part, finding that:
- Work-product protection may apply to AI-assisted work. Work-product protections extend to pro se litigants, and those protections are “magnified in the context of AI” where litigants “are forced to act as both party and advocate[.]”
- Using AI does not automatically eliminate confidentiality. Just because an AI tool collects or retains data “does not eliminate all expectations of privacy or automatically waive protections”—it is “entirely reasonable for a person to expect some privacy and confidentiality when interacting with these tools,” even if a third party is collecting and storing their information.
- No automatic waiver of work-product protection. AI interactions do not automatically compromise work-product protections even if it “technically ‘discloses’ information to a third party” because it is “highly unlikely” to fall into the hands of an adversary.
- The identity of AI platform is not protected. Work-product protections do not extend to the identity of the AI tool used because disclosure of the tool itself will not reveal mental impressions or legal strategy.
The court amended the protective order to prohibit the input of confidential information into AI tools unless the provider is contractually prohibited from retaining and disclosing the data to third parties, and provides the user with the ability to delete confidential information upon request.
Tate Automotive Group, LLC v. Legacy Automotive Capital, LLC (Texas Business Court). In this case, a non-lawyer company representative uploaded case-related materials to ChatGPT during the course of litigation. Plaintiff asserted work-product protection under the state’s Rules of Civil Procedure. Defendants argued that work-product protection does not extend to a non-lawyer’s conversations with an AI tool, and even if it did, uploading material to the AI tool waived the protection.
The court issued a minute entry ruling, indicating that:
- AI interactions could be protected. Conversations with AI tools, whether or not generated by non-lawyers, could be protected work product if it consists of “material prepared or mental impressions developed in anticipation of litigation or for trial by or for a party[.]”
- AI use does not meet waiver standard. Work-product protection is only waived when disclosed to an adversary, or in a manner substantially likely to result in disclosure, and AI tool usage does not meet that standard.
While the court ordered the plaintiff to disclose “all discovery materials or products” shared with ChatGPT, it did not require disclosure of protected mental impressions or legal theories simply because they were shared with the AI tool.
Shealy v. Seaside Investments, LLC (Massachusetts Superior Court). In this case, plaintiff received a notice of potential default on a promissory note from the defendants. Despite being represented by “sophisticated, capable counsel,” plaintiff sent the agreement and related documents to his romantic partner and directed her to upload the documents onto ChatGPT along with a “query or prompt” to prepare a response letter. When defendants moved to compel production of the documents, plaintiff argued that the AI-generated documents were “created in anticipation of litigation and qualify for work product protection” under the state’s Rules of Civil Procedures.
The court granted defendants’ motion, finding that:
- Work-product protection does not apply. While the work product doctrine protects mental impressions, conclusions and legal theories prepared by an attorney or representative, the court found that neither plaintiff’s romantic partner nor ChatGPT qualified as a “representative.”
- Distinguishes Morgan and Warner: Whereas Morgan and Warner involved pro se plaintiffs who are forced to act as both party and advocate, plaintiff was represented by counsel who had “nothing to do with [his] decision to ask his romantic partner to upload documents to ChatGPT” for legal assistance.
- Comparison to Heppner: Like Heppner, plaintiff was represented by counsel but used AI on his own volition. Since the documents were not prepared by or at the direction of counsel and did not reflect “counsel’s strategy,” it is not protected under the work product doctrine.
The court concluded that neither the queries of AI programs by a party’s romantic partner nor the output are protected from disclosure unless performed at the direction of counsel.
AI Hallucinations: Verify Before You File—Opposing Counsel Not Off-the-Hook
On AI-generated hallucinations, courts have delivered a more consistent message: lawyers remain responsible for the accuracy of their filings. Recent decisions emphasize the duty to verify AI-generated authorities before filing and the importance of promptly acknowledging and correcting errors upon discovery. Several courts have also criticized opposing counsel for failing to catch the errors.
- The Third Circuit reprimanded an attorney for using his non-lawyer client’s AI-assisted legal research in briefs without verifying the content. The panel noted that after opposing counsel pointed out eight incorrect citations, the attorney minimized the errors as “immaterial misstatements” and “made no attempt” to remediate until ordered by the court to verify the cases. The panel noted the decision to impose sanctions “would be different” had the attorney checked the citations upon notice and alerted the court of the inaccuracies.
- The Ninth Circuit sanctioned two attorneys $2,500 each and suspended them from practicing before the court for six months for filing briefs that contained “multiple nonexistent cases, misattributed quotations, and gross misrepresentations of real cases.” While an attorney attributed the error to unlicensed law school graduates who drafted the briefs, the panel stated that his signature on the briefs was an “attestation” that he personally reviewed the briefs, including the citations.
- The New Jersey Appellate Division sanctioned an attorney $1,000 for “glibly ignor[ing]” false case citations after opposing counsel pointed out one of four errors in the attorney’s brief. Instead of correcting the errors or notifying the court, the attorney “careless[ly] and dismissive[ly]” replied that a review of legal databases confirmed the existence of an opinion “with a similar citation.” The panel noted that opposing counsel and the court “expended wasted time to research the nonexistent cases.”
- The Seventh Circuit admonished an attorney for submitting a brief with nonexistent citations and a false quotation but imposed no sanction because she “took responsibility” for the errors and acknowledged the need to be “more scrupulous.” Notably, the panel stated that opposing counsel’s failure “to catch these errors and bring them [the court’s] attention also gives [the panel] pause[.]”
- The California Court of Appeal sanctioned an attorney $10,000 for filing briefs “peppered” with hallucinated citations and quotations and further referred the matter to the State Bar. However, the court declined to award attorney fee sanctions to opposing counsel because they failed to catch the hallucinations and appeared to recognize the problem only after the court raised it.
- The New York Appellate Division sanctioned an attorney $8,000 and his firm $2,500 for submitting a brief containing nonexistent authority, despite assuring the court “with every fiber” in his body that he would verify every citation going forward. The court also chastised opposing counsel for failing to catch the errors, calling the omission “striking, concerning, disappointing, and saddening” considering the fabricated cases were used as part of the argument.
- The Illinois Appellate Court sanctioned an attorney $15,000 for submitting a brief containing 10 false citations, noting that the “sloppy filing” harms the attorney’s clients, his professional reputation, and the court’s workflow. The court addressed the $1,500 fine per false citation, stating that AI hallucinations are an increasing problem and that lesser fines have not been enough to deter improper AI use.
Employer Takeaways
These developments demonstrate that while AI may be a useful legal tool, its protections—and its risks—turn heavily on how it is used. Collectively, these decisions underscore the importance of carefully controlling AI use and verifying outputs in legal filings. Consider the following when using AI in litigation:
- Establish clear rules for company AI use. Discuss generative AI with internal stakeholders early and advise them of potential privilege issues created by independently using AI for legal research or advice.
- Document counsel-directed AI work. When AI is used to assist with litigation, document counsel’s direction and the purpose for which the work is performed.
- Know the AI platform before sharing sensitive information. Review the provider’s retention, disclosure, deletion, and confidentiality terms before entering confidential or litigation-related information. Where possible, use secured AI platforms.
- Verify every authority before filing. Independently confirm cases, quotations, citations, and propositions generated or identified through AI against reliable legal databases.
- Correct mistakes quickly—and check the other side’s citations. Investigate and correct hallucinations promptly rather than minimizing or defending an error. Verify suspicious authorities cited by opposing parties.
If you have any questions related to these decisions, or need assistance reviewing your organization’s AI policies and protocols, please reach out to the Nukk-Freeman & Cerra Attorney with whom you typically work or call us at 973.665.9100 or 619.292.0515.