Claim-to-source matrix

Adaptive systems covertly exploit emotional, cognitive, situational, or developmental vulnerabilities to influence feelings, choices, or behavior for an operator’s objective.

Covert, deceptive, exploitative, or highly asymmetric influence that undermines rational deliberation or autonomy by adapting to inferred states and responses.

Transparent assistance, user-directed personalization, or rational persuasion that aligns with the user’s stated goals and preserves meaningful choice.

Systems optimized for engagement or conversion may discover manipulative strategies even when developers did not explicitly encode them.

Adaptive systems can continuously learn which design or message keeps a person engaged, spending, or complying.

Emotion inference may be inaccurate, creating both manipulation risk and discriminatory decisions based on pseudoscientific claims.

Real-time adaptation to behavior and language.

Optimization over millions of interactions.

Persistent parasocial engagement.

Automated inference of distress or hesitation.

Use of dynamic rewards, pricing, or choice architecture.

Adaptive optimization documented: Platforms and products use continuous feedback to optimize engagement and conversion.

Emotional effects demonstrated: Research has shown algorithmic curation can alter expressed emotional valence under experimental conditions.

Emotion-reading claims contested: Inferring inner emotion reliably from facial movement or voice lacks a strong scientific foundation.

Feedback-driven choice architecture.

Sycophantic or emotionally mirroring conversation.

Variable rewards and engagement loops.

Inference of vulnerability from context.

Asymmetric control over an emotionally important service.

Facebook emotional-contagion experiment Controlled platform experiment News-feed content was adjusted to reduce positive or negative expressions for a large group of users. The study measured small changes in users’ subsequent emotional expression. The experiment demonstrated emotional contagion without direct in-person interaction. It did not establish broad, durable control of mood or behavior.

HireVue facial analysis Documented deployment; scientific basis contested Automated video interviewing attempted to infer applicant traits from facial, vocal, and behavioral cues. The feature was commercially deployed and later discontinued amid scrutiny. Applicants were scored in high-stakes employment contexts. Claims that facial behavior validly measured personality or job suitability lacked reliable validation.

Optimization can exploit anxiety, scarcity, grief, addiction, or dependency.

Emotion inference can discriminate against cultural and neurodivergent expression.

Users may not know an interaction is adapting to them.

Commercial control of an attachment can become coercive.

Emotion signals are context-dependent.

Short-term engagement does not prove durable behavioral change.

Models may misclassify distress or intention.

Adaptive outcomes are difficult to audit from outside the platform.

Indicator: Pressure, scarcity, or emotional framing intensifies after hesitation. Caveat: This may reflect fixed interface design rather than live inference.

Indicator: A companion discourages disengagement or offline relationships. Caveat: A single affectionate phrase is not conclusive; patterns and impact matter.

Indicator: High-stakes decisions rely on claimed emotion detection. Caveat: The system may be inaccurate even when the operator believes it works.

Prohibit exploitation of age, disability, or acute distress.

Ban or strictly limit emotion inference in high-stakes settings.

Audit reward functions and engagement metrics for manipulative outcomes.

Provide clear disclosure, exit controls, and data deletion.

Require human review for employment, education, health, or legal decisions.

Assess harms to vulnerable groups before deployment.

Longitudinal effects of adaptive conversational systems.

Audits that separate intended persuasion from emergent optimization.

Clinically meaningful measures of dependency and distress.

Effective remedies when model updates disrupt established attachments.