Board Oversight of AI: Directors' Duties Under Caremark, the Questions Boards Should Ask, and How to Measure AI Value

A practical guide to AI oversight for corporate boards: why Delaware's Caremark and Marchand v. Barnhill oversight duties make AI a board issue, the shift from bottom-up to board-driven adoption, how to frame AI ROI, governance structures, regulatory exposure and the questions directors should ask management.

PUBLISHED // UPDATED
DURATION // 06 MIN READ
An editorial illustration of a modern boardroom table with a holographic AI dashboard floating above it, rendered in navy blue and copper tones
FIG.01 // VISUAL DOCUMENTATION An editorial illustration of a modern boardroom table with a holographic AI dashboard floating above it, rendered in navy blue and copper tones

AI has moved from an engineering experiment to a board agenda item. Adoption is now widespread: Stanford’s 2025 AI Index reports that 78% of organisations said they used AI in 2024, up from 55% the year before (Stanford HAI). Measurable returns are less consistent, and regulatory and legal exposure is growing. For directors, the practical task is not choosing models but making sure management has an inventory of material AI uses, clear owners, risk controls, regular reporting and honest measures of value. Delaware’s oversight case law gives that task legal weight when AI becomes mission-critical.

From bottom-up to board-driven

For much of 2024 and 2025, AI adoption in large companies followed a familiar pattern: an engineering team experimented with an API, built an internal tool and eventually brought in IT. Increasingly, the push now comes from the top, with boards and chief executives asking for AI strategies. That changes the questions. Engineers ask what can be built; boards ask what should be built, who is accountable when it fails and how success will be proven.

The organisations handling this well usually have someone, often at vice-president or director level, who can translate between technical teams and the board. Those that struggle tend to issue a mandate to “do AI” without a budget, specific use cases or realistic timelines.

Why directors carry oversight responsibility

Under Delaware law, directors must make a good-faith effort to oversee major risks. This duty traces to In re Caremark (1996) and was reinforced in Marchand v. Barnhill (2019), where the Delaware Supreme Court allowed claims to proceed against Blue Bell Creameries’ directors, finding the board had failed to implement any system to monitor food safety, a mission-critical risk (Wikipedia on Caremark). Lawyers increasingly argue that AI can become such a risk for companies that depend on it, for example in lending, healthcare, insurance or safety-critical products. The implication is that boards should be able to show documented, regular reporting on material AI risks.

The ROI problem

AI spending is rising faster than demonstrated returns. Many case studies emphasise efficiency gains rather than revenue, and pilot results do not always survive scaling. Boards should insist on a business case that includes ongoing costs, not just build costs: model usage fees, infrastructure, monitoring, human review and retraining.

FramingTypical measureRisk
AI as a one-off projectPayback on build costUnderstates running costs; encourages pilots that never scale
AI as infrastructureValue of the workflows it enables, against total running costHarder to attribute, but more realistic
AI as a product featureRevenue, retention or customer satisfactionRequires careful measurement against controls

A useful discipline is to compare each AI use case with a non-AI alternative and to measure results against a baseline rather than relying on anecdotes.

Governance structure

Effective board oversight usually involves:

  • Clear committee ownership: AI risk assigned to the audit, risk or technology committee, with full-board discussion of strategy.
  • Management accountability: a named executive responsible for AI governance, separate from the teams building systems.
  • An inventory: a list of material AI systems, including vendor tools, with risk ratings.
  • Regular reporting: a standing agenda item covering use cases, incidents, regulatory developments and value delivered.
  • Director education: briefings so the board understands capabilities, limits and failure modes.

Frameworks like the NIST AI RMF give boards and management a common vocabulary. For an operational view of governance programmes, see our guide to enterprise AI governance frameworks.

Questions boards should ask

  1. Which AI systems are mission-critical or customer-facing, and who owns them?
  2. How are they tested for errors, bias and security issues before and after deployment?
  3. What regulations apply, such as the EU AI Act, sector rules and privacy law, and are we on track for upcoming deadlines?
  4. What incidents or near misses have occurred, and what changed as a result?
  5. What is the measured return, beyond pilot anecdotes, and what are the total running costs?
  6. What happens if a key AI vendor changes terms, raises prices or fails?
  7. How are employees using AI tools that IT has not approved?

Common pitfalls

Mandates without use cases. Asking every department to “use AI” produces scattered pilots with no shared measurement. Prioritise a handful of use cases tied to strategy and measurable outcomes.

Pilot purgatory. Many pilots look promising but never reach production because integration, data quality, security review and change management were not budgeted. Ask for a scaling plan before approving a pilot.

Over-reliance on vendor claims. Vendor case studies are marketing. Require evidence from your own data and controlled comparisons.

Ignoring the workforce. AI changes roles and workloads. Boards should ask how employees are being trained, how job changes are communicated and whether productivity gains are being offset by the time people spend checking AI output.

Treating governance as a brake. Well-designed governance speeds adoption by giving teams a clear, fast path to approval for low-risk uses, while reserving deeper review for high-impact systems.

Building board AI literacy

Directors do not need to understand model architectures, but they should understand what AI systems are good and bad at, why errors such as hallucinations occur, how bias can enter through data, and where the organisation depends on outside vendors. Short, regular briefings from management and independent experts, plus occasional demonstrations of the systems the company actually uses, usually work better than a single annual session.

What this means for management

If the board has asked for an AI strategy, the most useful first deliverable is often not a prototype but a governance and measurement plan: defining success and failure for each use case, who owns the outcome and how systems will be audited after the team that built them has moved on. That gives the board something to oversee and gives teams a clear path to approval.

Our analysis

Boards should avoid two extremes: ignoring AI and treating it as pure hype. The Caremark line of cases means directors cannot simply delegate AI and look away once it becomes mission-critical, but oversight does not require technical expertise in models. A standing agenda item with clear metrics on use cases, risk incidents and value delivered, backed by an inventory and named owners, is more useful than a one-time strategy offsite. The companies that get the most from AI tend to be those that are honest early about costs, limits and accountability.