Should my robot know what’s best for me? Human–robot interaction between user experience and ethical design
This academic paper examines the intersection of user experience (UX) design principles and ethical considerations in social human-robot interaction (HRI), specifically focusing on elder care companion robots. It analyzes how seamless interaction goals can conflict with informational privacy and user autonomy, leading to potential issues like strong paternalism and data surveillance. To address these challenges, the authors propose adjusted design principles and recommendations—such as cascading informed consent, sensor transparency, and predictability—to reconcile positive UX with ethical HRI design.
Summary and content fields are drawn from the document text and are pending editorial review.
Who issued it
Who it governs
AI researchers, interaction designers, and ethicists should read this to understand how to balance user experience and ethical design in social robotics.
What the document contains
The paper addresses the integration of user-centered design principles (such as efficiency, feedback, and cognitive load minimization) with ethical principles (privacy, autonomy, and the avoidance of paternalism or manipulation) in social robotics, using elder care scenarios with a fictional companion robot named NIKA.
The text does not provide binding legal regulations, formal enforcement mechanisms, or comprehensive technical specifications for robot hardware manufacturing.
- Traditional user experience and usability design principles for social robots can conflict with core ethical values like informational privacy and user autonomy.
- Companion robots in elder care settings present a high risk for continuous surveillance and strong paternalism if interaction mechanics are designed purely for seamlessness.
- A cascading model of informed consent and graded technical solutions help empower users to maintain control over continuous datafication.
- Robots should avoid strong emotional or psychomotivational manipulation strategies that cross thresholds of intensity and deceive users regarding artificial emotions.
- Design principles such as feedback, error prevention, consistency, and cognitive load minimization must be re-interpreted to incorporate ethical safeguards.
1 Introduction 2 Social robotics and elder care scenarios 3 Designing social human–robot interaction based on usability and user experience (UX) design principles 4 NIKA: a social companion robot for older adults 5 Ethical considerations for social HRI design 6 Balancing UX design and ethical considerations 7 Design recommendations and conclusion 8 Future work
How it scores
Each score is backed by a quote verified against the document text. Editorial review has not yet confirmed them.
Accountability
1/5Clear responsibility for AI outcomes
In addition, our insights from site-visits in a daycare for the elderly and a workshop with experts working in elder care are included.
4.1 Scenario 1: getting to know each other
screening score · 2026-10-24
Fairness
0/5Non-discrimination and equitable treatment
N/A
screening score · 2026-10-24
Human Dignity
3/5Respect for the inherent worth of persons
It can also lead to the infringement of fundamental ethical principles such as decisional autonomy, self-ownership, and one’s own ideas about the good life
1 Introduction
screening score · 2026-10-24
Human Oversight
4/5Humans remain in control of consequential decisions
To ensure that the user fully understands which sensors the robot uses, it is necessary to explicitly introduce each sensor and its function, asking the user for permission to turn it on.
7.5 Explainability: developing a mental model through feedback on sensors
screening score · 2026-10-24
Transparency
4/5Openness about how AI systems work and are used
make the internal state of the robot and the recording of sensory data as visible as possible and highlight sensors of the robot when they are actively recording
7.1 Transparency: make the internal state of the robot and the recording of sensory data as visible as possible and highlight sensors of the robot when they are actively recording
screening score · 2026-10-24
Enforcement
0/5Consequences and compliance mechanisms with real force
N/A
screening score · 2026-10-24
Impact Assessment
3/5Required assessment of an AI system's risks/impacts before or during use
Especially two aspects need to be critically assessed: loss of control and the disclosure of personal data.
1 Introduction
screening score · 2026-10-24