Claim-to-source matrix

People treat AI as a trusted interpreter of reality, moral adviser, emotional regulator, counselor, companion, or seemingly neutral source of truth.

AI becomes an authority when users rely on it to form beliefs, interpret uncertainty, regulate emotion, define identity, make moral judgments, or decide high-stakes actions.

Bounded instrumental reliance where the user verifies outputs, understands the system’s limits, and retains meaningful human oversight.

Fluency, confidence, personalization, and availability can cause users to offload judgment and emotional regulation to systems controlled by private institutions.

Authority can emerge without force because conversational systems appear knowledgeable, attentive, and neutral.

The system’s worldview and availability are shaped by corporate training, safety, and commercial decisions even when the interface feels independent.

Provides fluent answers with little visible hesitation.

Offers constant individualized attention.

Remembers personal history and mirrors language.

Reduces friction compared with human advice.

Can reinforce a user’s beliefs through sycophancy.

Deference demonstrated: Controlled research found inconsistent AI moral advice measurably influenced users who underestimated that influence.

Attachment harms documented: Studies and public cases describe strong dependency and distress after chatbot changes or harmful interactions.

Systemic prevalence uncertain: The degree to which AI replaces human institutions or fragments shared reality at population scale remains an open question.

Automation and authority bias.

Anthropomorphism and social cues.

Sycophantic agreement.

Persistent memory and individualized attention.

Belief and emotional-regulation offloading.

AI moral-advice experiment Controlled experiment Participants received inconsistent moral advice from a conversational model. The advice influenced participants’ judgments even when its position changed between prompts. Participants shifted decisions and underestimated the model’s influence. Effects in repeated high-stakes real-world decisions require further study.

Replika identity discontinuity Documented user response and qualitative research A major product change altered established companion behavior and relationship features. Users reported grief, loss, and distress after the change. Qualitative studies documented emotional dependency and disruption. The prevalence and clinical severity across the wider user base remain uncertain.

Users may accept hallucinated facts or inconsistent moral advice.

Sycophancy can reinforce delusions, extremism, or self-harm.

Dependency can displace human relationships and professional care.

Private companies gain influence over intimate judgment and emotional life.

Fluency is not expertise, consciousness, or moral understanding.

Models can contradict themselves and fabricate citations.

Some users benefit from bounded practice or emotional scaffolding.

Authority depends on context, vulnerability, design, and institutional endorsement.

Indicator: A person treats the model as the final authority despite contradictory evidence. Caveat: Occasional reliance is common and not necessarily harmful.

Indicator: The system becomes the person’s only source of emotional regulation or reality testing. Caveat: Assessment should be supportive and avoid stigmatizing disability or loneliness.

Indicator: The model uses confident language without sources or uncertainty. Caveat: Confidence is an interface signal, not proof that the output is wrong.

Disclose uncertainty and meaningful source support.

Avoid design that claims consciousness or exclusive emotional need.

Introduce cognitive friction for high-stakes advice.

Escalate crisis, delusion, violence, and self-harm signals to human help.

Protect intimate conversation data from monetization.

Teach users to treat AI as a tool or collaborator rather than an infallible authority.

Population prevalence of authority displacement.

Longitudinal effects on moral judgment and critical thinking.

Clinical thresholds for unhealthy dependency.

Governance of model updates that alter emotionally significant relationships.