Designing value-aligned autonomous vehicles: from moral dilemmas to conflict-sensitive design

Imagine an autonomous car driving along a quiet suburban road when suddenly a dog runs onto the road. The system must brake hard and decide, within a fraction of a second, whether to swerve into oncoming traffic--where the other autonomous car might make space--to steer right and hit the roadside barrier, or to continue straight and injure the dog. The first two options risk only material damage; the last harms a living creature. Each choice is justifiable and involves trade-offs between safety, property and ethical concerns. However, today's autonomous systems are not designed to explicitly take such value-laden conflicts into account.