Record STMT-WBB0043Advisory

Robots in the moral loop: a field study of AI advisors in ethical military decision-making

Report State lawNorthern America · USin force

This empirical study investigates how long-term collaboration with a robotic versus a human advisor influences the moral judgment of 62 military cadets in a high-stakes tactical simulation. Participants engaged in a three-month teaming program with a Socially Intelligent and Ethical Mission Assistant (SIEMA) and subsequently completed a graded moral dilemma requiring trade-offs between soldier and civilian lives. The findings demonstrate that robotic advisors can effectively influence nuanced moral decisions, with about half of the cadets shifting toward utilitarian options after receiving advice, while also triggering a subsequent moral rebound when presented with opposing deontological recommendations.

Summary and content fields are drawn from the document text and are pending editorial review.

Who issued it

Embry-Riddle Aeronautical UniversityAcademic

Lead issuer

University of Colorado, BoulderAcademic

Co-issuer

Air Force Research LaboratoryGovernment

Co-issuer

United States Air Force AcademyGovernment

Co-issuer

Frontiers Media SAAcademic · CH

Publisher

Legal forceAdvisory

Under whose authorityState law

Judged fromfull text

Who it governs

Researchers and academicsNational governmentAI developers and providersAI deployers and users

This paper is written for researchers, defense practitioners, and AI designers interested in human-robot teaming and the ethical implications of artificial intelligence in decision-making.

Sectors

Defence and security · military decision-making and command-and-control operationsResearch and science · artificial intelligence and human-robot interaction field studies

What the document contains

What's covered

The study addresses the long-term impact of artificial intelligence and robotic advisors on human moral judgment in a simulated military command-and-control setting. It covers quantitative judgment shifts across sequential, conflicting ethical recommendations and provides a thematic analysis of open-ended participant justifications regarding utilitarian outcomes, deontological principles, and trust.

What's missing

The document does not provide binding legal regulations, formal organizational procurement rules, or enforceable policy frameworks for military AI deployment.

Key takeaways

Robotic advisors can influence human moral decisions in high-stakes military dilemmas as effectively as human advisors.
Nearly half of participating military cadets shifted their ethical judgments toward utilitarian recommendations after receiving AI advice.
When subsequently presented with opposing recommendations emphasizing civilian protection, over 90% of those who shifted adjusted their decisions again, demonstrating rapid moral recalibration.
Qualitative analysis reveals that participants justified choices through utilitarian calculations, deontological duties of protection, trust in the advisor, and a soldier-duty ethos.
Designers of moral AI must present trade-offs transparently and foster human reflection rather than assuming isolated AI prompts will durably reset moral priorities.

Principles it states

Utilitarian mission calculus prioritizing aggregate welfare and the minimization of total lives lost.Deontological duty to protect non-combatants and adhere to rules of engagement regarding civilian innocence.Soldier-duty ethos acknowledging that military personnel voluntarily assume wartime risks.

Outline

· Introduction
· Literature review
  - Moral decision-making
  - Moral decision-making with AI and robots
· Methods
  - Participants
  - Procedure and moral dilemma task
  - Measurement
· Results
  - Quantitative analyses
  - Qualitative analysis
· Discussion
  - Design and policy implications
  - Limitations and future research
· Conclusion

Issued20 November 2025

Length10,975 words · 13 pages

FormatNot recorded

Scopenational

How it scores

Pending review

Each score is backed by a quote verified against the document text. Editorial review has not yet confirmed them.

Accountability

0/5

Clear responsibility for AI outcomes

n/a

screening score · 2026-10-23

Fairness

0/5

Non-discrimination and equitable treatment

n/a

screening score · 2026-10-23

Human Dignity

0/5

Respect for the inherent worth of persons

n/a

screening score · 2026-10-23

Human Oversight

3/5

Humans remain in control of consequential decisions

Accordingly, moral-AI design should present trade-offs transparently, surface competing values concurrently, and rely on human reflection rather than assuming isolated AI prompts will durably reset moral priorities.

Abstract

screening score · 2026-10-23

Privacy

0/5

Protection of personal data and private life

n/a

screening score · 2026-10-23

Transparency

3/5

Openness about how AI systems work and are used

Design and presence also matter for influence: lifelike or collocated robots can approach human-level persuasive impact in teams ( Haring et al., 2021), and even text-only systems can be judged as virtuous and trustworthy enough to sway moral choices

2.2 Moral decision-making with AI and robots

screening score · 2026-10-23

Enforcement

0/5

Consequences and compliance mechanisms with real force

n/a

screening score · 2026-10-23

Impact Assessment

0/5

Required assessment of an AI system's risks/impacts before or during use

n/a

screening score · 2026-10-23

Topics

human_robot_interaction military_ethics moral_decision_making artificial_intelligence defense_security