Skip to content

Content Library

Back to In-depth

Connect Plus In-depth November 6, 2025

Responsible AI in P&C Insurance: Navigating the Inflection Point

Responsible AI in P&C Insurance: Navigating the Inflection Point

Paul Y. Mang, Partner

Vidur Nayyar, Founder

AI has become an industry inflection point, and insurance is uniquely positioned to lead. The paper argues for responsible, objective-based AI adoption that balances innovation with trust, maps 14 emerging risks, and proposes an eight-pillar framework. It
  • AI shifts from tool to organising principle across insurance
  • Insurers’ history of shaping safety makes them natural AI leaders
  • Leaders see superior growth and efficiency from AI at scale
  • Carriers must adopt AI while also underwriting AI risks across industries
  • Guidance lags tech, creating a prudence paradox for cautious firms
  • Fourteen key risks span bias, privacy, security, model drift and vendor dependency
  • Objective-based governance and an eight-pillar framework guide action
  • Human-centred design, transparency and explainability remain critical
  • Agentic micro-services can deliver audited, modular underwriting flows
  • Move now with measurable controls rather than waiting for perfect rules

At rare moments in history, an emerging technology doesn’t just improve business: it revolutionizes and reinvents the very way we do business. Electricity supercharged entire industries, enabling mass production and improving the quality of modern life.  

 

The internet transformed the way we interact with the world, process transactions, and consume information.  

 

The smartphone redefined the way we connect with each other, navigate our surroundings, and conduct the daily business of living. Today, it is artificial intelligence (AI) that stands at this inflection point.  

 

AI is no longer a peripheral tool; it is fast becoming the organizing principle for modern enterprises enterprises and few industries are better positioned to embrace this moment than insurance, an industry that has always been both the catalyst and the safeguard of innovation.  

 

It was insurers who created fire brigades to protect cities, who funded Underwriters Laboratory to make electricity safe, and who promoted automobile seatbelts and fire sprinklers to reduce the risk of personal injury and property damage.  

 

Insurance has never run from risk; it has run toward it, responsibly, deliberately, and with society’s trust in mind. That bold legacy now extends to AI. 

  

With its transformative promise comes the duty to ensure safety, fairness, and accountability. Responsible AI is not simply the “bells and whistles” or an item on the compliance checklist; it is the bedrock of sustainable value creation.  

 

And insurers, with their deep history of shaping risk frameworks, are uniquely qualified to lead the business world into the next era and help people embrace AI’s power with confidence and enthusiasm rather than fear and apprehension. 

 

What is the Opportunity if Done Right 

 

Given the high cost of operating such a complex business, AI has found use across the full insurance value chain: from traditional machine learning models uncovering patterns in structured data to generative AI bringing an unprecedented ability to work with unstructured text, images, and voice, to emerging technologies like agentic AI that can automate entire end-to-end workflows.  

 

This is already boosting underwriting accuracy, claims efficiency, customer service, and even product design. 

 

Most insurers have adopted AI, but many still lack full integration, driving increased investment to stay competitive.  

 

AI leaders outperform peers, showing 6.1x higher total shareholder returns over five years [1]. Top insurers are transforming key domains like distribution, underwriting, policy management, and claims.  

 

AI-driven overhauls yield measurable gains: 10% to 20% improvements in agent success rates, 10% to 15% premium growth, 20% to 40% lower onboarding costs, and 3% to 5% higher claim accuracy [1] [17].  

 

These strategies highlight the untapped potential of scaling AI across the value chain. This fast-evolving landscape has given laggers a chance to level set with leaders and redefine the industry scoreboard.  

 

Insurers are not only users of AI but also underwriters of AI risk across industries. From Boeing designing AI-driven aircraft to banks deploying automated lending, insurers will increasingly be asked to evaluate, price, and cover AI-related risks.  

 

That dual role raises the bar even higher, making it essential for the insurance industry to adopt AI to better serve its customers and stay relevant in today’s world. However, it is this unique position that makes the opportunities even more interesting. 

 

The Unique Risk Landscape of Insurance and the Pace of Technology 

 

For insurance, the implications are especially profound. At its core, insurance is about safeguarding what we value so that people can live their lives to the fullest and businesses can operate with the confidence of having an institute to support them through adverse events and operational disruptions. 

 

Insurers have a well-earned reputation for being careful and seek to be responsible in their AI use.  

 

But AI is evolving so quickly that those who want to be prudent may find it difficult because guidelines for Responsible AI are not keeping up with new developments.  

 

Industry reactions to recent insurance AI guidance reflect this gap: well-intended, but not practical enough for daytoday decisions where accountability cuts across various processes like underwriting, claims, actuarial, and distribution [3]. 

 

The fallout of getting it wrong is real. In 2022, State Farm was sued in the U.S. District Court for the Northern District of Illinois over allegations that its AI discriminated against minority customers.  

 

The class action cited a study of 800 black and white homeowners that found black policyholders experienced more delays and were more likely to be asked for additional documentation, highlighting how algorithmic bias can directly erode trust and trigger legal risk [4].  

 

This creates a prudence paradox. Waiting feels safe, but risks falling behind. Charging ahead without guardrails risks bias, regulatory scrutiny, and reputational harm. 

 

So, how can insurers innovate in real time, prudently and confidently, as standards evolve? 

 

Risks have evolved with the fast-changing tech landscape  

 

With the evolving risk landscape insurers and innovators in the risk space will need to rethink their approaches to emerging risk types, moving beyond traditional data quality and compliance to include advanced guard-rails and ethics testing. We have identified and logged the 14 key risks faced by the insurance industry in the new era of AI: 

  1.  Bias in fairness: Historical biases in data can carry into AI models, leading to unfair pricing, underwriting, or claims decisions for underrepresented groups that erode trust and expose insurers to regulatory risk and lawsuits. 
  2. Regulatory guardrails: AI innovation often outpaces evolving state, federal, and international rules, leaving insurers exposed to uncertainty. Fragmented and sometimes superficial guidelines mean carriers risk scrambling to comply reactively instead of embedding strong proactive internal standards aligned to the EU AI Act, NAIC Model Bulletin, and emerging state laws. Essential both to shield against future regulatory shifts and to avoid business disruptions. Models deployed across markets must also meet conflicting cross-border obligations, so a system compliant in one jurisdiction may be non-compliant in another.
  3. Hallucinations: Models may produce convincingly packaged misinformation, leading to unintended consequences with unclear liabilities. For instance, in underwriting, research becomes harder when models introduce misleading or fabricated content, making it difficult for insurers to separate valid insights from misinformation [5]. 
  4. IP violation: Generative AI trained on copyrighted or proprietary data can expose insurers and Insurtechs to violations of intellectual property rights. If underwriting guidelines or proprietary pricing models are leaked during model training or fine-tuning without proper safeguards, or via shadow AI where employees use unapproved tools or paste sensitive data into public models, insurers risk losing their "secret sauce" and facing costly legal and reputational consequences. 
  5. Data privacy: Insurance customers share sensitive data from medical, financial, and claims records. AI models and handling must comply with HIPAA, ISO, and SOC2 standards. Without strong governance, PII scrubbing, and transparent consent practices, insurers risk breaches, regulatory penalties, and erosion of customer trust. 
  6. Malicious use for Fraud: Generative AI can be weaponized by bad actors to create deepfakes or manipulated content, exposing insurers and customers to fraud, harassment, or reputational damage. For example, a modified image of a damaged Ferrari can result in a significant payout on a fraudulent claim. 
  7. Security threats: Cybercriminals are leveraging AI for more sophisticated phishing, social engineering, and system breaches, raising the stakes for insurer cybersecurity. The training and rules that applied to phishing emails pre-GenAI era are changing at a fast pace, which makes it harder for targets to differentiate between what is legitimate and what is not. New threats like prompt-injection attacks emerge, such as the recent Gmail exploit where attackers embedded invisible commands into emails that AI tools executed [7].  
  8. Performance & Explainability: As the complexity of models increases, particularly for generative artificial intelligence (GenAI) models, there is little scope for transparency or the ability to replicate the results. This lack of clarity hinders trust, especially when dealing with critical matters such as insurance claim rejections or underwriting decisions. 
  9. Strategic risk: The absence of a robust and thoughtful AI adoption strategy can drive workforce disruption and lead to inadvertently higher carbon emissions. Careful consideration of responsible upskilling, effective workforce management, and sustainable commitments is essential. In one such case, artificial intelligence replaced 700 Klarna employees and now the company seeks to rehire them after losing $40 billion in value [9]. 
  10. 3rd party risk: With technology taking center stage and insurers shifting from Build to Buy, reliance on vendors is now a core governance issue. Because vendors themselves are often not subject to the same compliance restrictions, the liability falls back on insurers as the users of these tools, creating a heightened risk of compliance gaps, cascading failures, and systemic disruption. As seen in the July 2024 CrowdStrike incident that crashed millions of systems and took hours to even trace back to the right provider [10]. Industry reliance on a small set of cloud and model providers creates concentration risk that can trigger simultaneous outages. 
  11. Customer experience: Another potential drawback of using AI and chatbots for insurance customer service is their inability to respond appropriately to anything deeper than the most simplistic and superficial of circumstances. Insurance customers may have complex, emotional, or urgent needs that require human interaction and understanding. AI and chatbots may not fully capture these nuances, emotions, and contexts, leading to inappropriate, inaccurate, or insensitive responses. This can cause frustration, dissatisfaction, or distrust. 
  12. Adoption risk: Insurance is an expert-led industry built on judgment, discretion, and trust, which makes AI adoption inherently challenging. End users such as underwriters and claims adjusters may resist AI tools, either skeptical of their outputs or fearful of job displacement, while consumers show uneven comfort and confidence in AI. These combined dynamics create the risk of stalled adoption, erosion of trust among employees and customers, and ultimately limited returns on AI investments. 95% of AI Pilots in companies are failing [11]. 
  13. Foundational model dependency: Generative AI systems are built on external foundational models that change abruptly, making tuned workflows and guardrails obsolete overnight. For insurers, this creates a unique risk of operational disruption, audit gaps, and compliance exposure, since liability remains with the insurer even though the model itself is outside their control. 
  14. Data quality risk: AI outputs are only as reliable as the data they are built on. In insurance, this challenge is amplified by fragmented silos, unclear ownership, and weak governance, which make it difficult to track data lineage, modality, and accuracy. Training on low-quality or AI-generated (synthetic) data can compound distortions over time, creating hidden biases, flawed underwriting or claims outcomes, audit failures, and regulatory exposure. 

 

Moving Forward: A Framework for Responsible Innovation 

 

Move forward with careful deliberation, but don’t be paralyzed by the search for perfection. Use a foundational framework and basic principles to guide actions and proceed with clear objectives that guide action today and adapt for tomorrow. 

 

Think of this as adopting the approach of the commander’s intent. In fastchanging environments, leaders give objectives, with commonly understood principles and constraints, instead of rigid stebbystep instructions.  

With clear direction and appropriate support, a leader empowers their team's distributed decisionmaking while keeping alignment. This requires leadership and training, so teams internalize the principles. 

 

Given the industry’s foundational reliance on trust, regulators and consumers might expect insurers to be held to a higher standard.  

 

The industry has to adopt a framework that derisks its investments in AI by applying a tailored approach that aligns with general responsible AI principles but is customized for the insurance context. 

 

Our Eight-pillar Objectives Framework provides the guidelines to develop a framework that ensures appropriate innovation with the standards expected of this industry: 

 

Strategic Alignment & Culture 
Define clear objectives that align AI initiatives with business goals and are closely linked to ROI. Use “commander’s intent” to guide evolving situations. Build cultural change through training, adoption support, and community development initiatives. 

 

Policy & Governance 

Implement organizational accountability, establish guardrails, and implement protocols for data and AI governance. Assign ownership for each use case, establish clear hierarchical and escalation lines, and engage stakeholders across sectors to establish clear prioritization, guidelines for responsible use cases and identify potential pitfalls. Partner with policymakers to ensure regulatory frameworks align with technological advancements, providing clarity for businesses and consumers. 

 

Fairness & Ethical Inclusivity 
Embed fairness checks into every use case. Define clear fairness criteria for underwriting, claims, fraud detection, and pricing. Build diverse teams and inclusive datasets. 

 

Human Centric Design 

Integrate user-centric design principles into every AI product, prioritizing the needs and preferences of both users and customers. Emphasize the importance of providing an exceptional user and client experience.  

 

Maintain human involvement in critical decision-making processes, such as judgment or empathy; and implement mechanisms to ensure empathy and simulate user perspectives. Design solutions that evoke a sense of necessity, trustworthiness, and seamless integration into users’ existing workflows. 

 

Transparency & Explainability 
Invest in tools and methods that enhance transparency. Explain how AI systems arrive at their conclusions and maintain logs of AI decisions, prompts, and chain-of-thought logic. This is particularly important in areas where the stakes are high, such as claims rejections. 

 

Providing clear explanations and reasons behind decisions goes a long way in building and maintaining consumer trust, as well as in obtaining approvals from regulators. 

 

Ecosystem & Collaboration 

Assess and manage vendors, foundational models, and partners. Ensure visibility into their operating models, sustainability, and resilience. Design exit paths, redundancies in case of failure, and avoid lock-in. 

 

Risk Management & Adaptive Controls 
 

Educate employees about the potential risks and ethical implications associated with AI systems they develop or deploy regularly They should be encouraged to speak up when identifying concerns encountered during any stage of these processes so issues may addressed promptly effectively before causing harm externally and internally. 

Events you may be interested in

View all of our upcoming events

Sign Up to Our Newsletter

Sign up

Connect Plus

Unlock more with Connect Plus membership

This content is available to Connect Plus members.