Creative AI for HRI Design Explorations

Human-robot interaction (HRI) research deals with the design of complex sociotechnical systems, but designing conceptually rich robotic artefacts and envisioning novel human-robot relations remains a difficult task. Despite recent efforts towards a paradigm shift in HRI, the majority of designs still centre on a limited design space of simple anthropomorphism or zoomorphism. Researchers argue this is due to preconceived notions of what a robot should do, look like, and behave like, resulting in ideation or design fixation. While HRI design seems particularly prone to the phenomenon of design fixation, it has been recognised in many fields and domains of design.

Overcoming Design Fixation with Generative Image Models

In this project, we explored the potential of using generative text-to-image models, such as StableDiffusion and DALL-E 2, to overcome design fixation and enhance creative processes in HRI design. We conducted a four-week-long design exploration using these models to ideate and visualise robotic artifacts and robot sociotechnical imaginaries. We found that generative text-to-image models can help overcome design fixation and inform divergent and desirable visions of robotic futures.

We explored whether and how the use of creative AI tools can support design ideation in HRI. To do so, we (the first three authors) used generative text-to-image models to envision robotic artefacts and reflect on robot sociotechnical imaginaries; over four weeks, we exchanged images and textual annotations of the first-person design exploration via digital postcards.
Sociotechnical Imaginaries and Creative AI Postcards

The investigation was framed around the theoretical concept of sociotechnical imaginaries, which are collectively held, institutionally stabilized, and publicly performed visions of desirable futures. These imaginaries can be employed as a design resource to inspire the future of technological innovation, for example, by drawing on or deliberately counteracting the imaginations of media and popular culture. The concept also offers an analytical lens to examine how an idea surrounding a sociotechnical assemblage aligns with or challenges a collective vision. 

We also agreed on following a methodological approach inspired by the “Dear Data”-book. “Dear Data” was a one-year-long project in which two information designers exchanged postcards with hand-drawn data visualisations about their lives on a previously agreed-upon topic. Following a similar approach, each week we turned our image generations into “Creative AI”-postcards, which we posted to a Miro board. The postcards contained the image output and prompt, a brief text about the designerly intention, the reflections in regards to HRI design (i.e., what was striking about the output or how it inspires new robot concepts and imaginaries), and the reflections in regards to working with the creative AI tool.


Through reviewing and clustering the generated images, as well as affinity mapping of the textual accounts from the postcards, we identified three themes:

Imagining Novel Robot Concepts

Our design explorations provided a wide range of instances in which the generated images stimulated our thinking towards novel robot concepts and imageries. For example, we noticed some “pleasant surprises” when the depicted robotic artefact would yield some form of robot hybridity. We also found instances of what we referred to as “robot/non-robots”-imaginaries: when the typical robot characteristics (human-or animal-like) were reduced to the point that the robot became almost invisible. We also found that embedding temporal context into the prompts invites for robot time travel in the future and the past. For example, a delivery robot in a medieval city painted by Albrecht Dürer could inspire designers to rethink the design and appearance of existing delivery robots.

Surfacing Existing Assumptions and Robot Stereotypes 

Although our design explorations resulted in outcomes with the potential to inspire novel robot concepts and surrounding imaginaries, we also discovered that many existing assumptions and robot stereotypes were reinforced. For example, robots would be depicted as white-coloured confirming the predominant racialisation of AI as white. We also found the reinforcement of other human biases. When one of the designers prompted “a robot that is a sex worker”, even social stereotypes – assuming that sex workers are mostly female – were reinforced and blended into the robot’s appearance.

Situating Robots in Context 

Our design exploration further demonstrated the potential of image generative models to situate robotic artifacts in various contexts, scenarios and situations. One of us stated in the focus group discussion: “Seeing the robots in context, which doesn’t happen often to me, helps to visualise and account for the elements, actors, and some of the dynamics of the contexts that we are designing for in HRI.”
Project Team

Marius Hoggenmueller, Maria Luce Lupetti, Willem van der Maden, Kajzon Grace

Key Publications
Marius Hoggenmueller, Maria Luce Lupetti, Willem van der Maden, Kazjon Grace. Creative AI for HRI Design Explorations . In Proc. of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23)