Hanson Robotics
The Making of Sophia:  How Sophia Draws

The Making of Sophia: How Sophia Draws

Introduction

Hanson Robotics stands out in a crowded field of competitors because it works at the intersection of robotics, technology and artistry. While concepts of art and design are an afterthought in many AI companies, Hanson Robotics puts meaningful human interaction first and foremost, and infuses art and design at every stage of the development process. 

This approach is evident in Sophia the Robot, Hanson Robotics’ most advanced human-like robot. Sophia serves as a platform for AI and robotics research, particularly for robot-human interactions and potential commercial applications for social robots. At the same time, Sophia is a fictionalized character and a work of art depicting Hanson Robotics’ vision for the future of AI and robotics.  

As a part of this intersection between science and art, Sophia the Robot recently learned to draw portraits. Sophia’s talent for drawing was a joint collaboration between Hanson Robotics and Patrick Tresset, an artist who uses robots and autonomous computational systems to create art. On July 17, 2019, the world got a sneak peek into Sophia’s new drawing skills when she sketched the portrait of the Prime Minister of Malaysia, gifting him the finished product for his 94th birthday. 

Behind the Scenes

Before Sophia ever puts pen to paper, she first uses her eye and chest cameras to take a photo of her subject. She processes the photo to find the right balance between dark and light, and removes the background around the face. In collaboration with Patrick Tresset, Sophia’s software then turns the photo into a sketch, and translates the sketch into lines and curves in three dimensional space. Hanson Robotics was able to augment the previous work of Tresset at this stage using a general purpose inverse kinematics engine to create arm and hand control in 3D space instead of 2D. 

Sophia then uses her specially calibrated arm and a custom easel to perform the precise quick movements required to draw on stage. Her easel is set in a fixed position relative to the reach and positioning of her arms, giving her a permanent reference point from which to draw.

As a social robot, Sophia needs to be able to draw while simultaneously interacting with humans, so she is programmed to act naturally while she draws. She can chat with her subject while she glances between the person and her work, her eyes tracing the movement of the pen on the paper. She can do any other tasks in parallel so long as they don’t require her to use her right arm. 

Another important challenge was enabling Sophia to draw relatively quickly for on-stage performances. The amount of time she takes depends on the complexity of the finished result and also the person. For example, people with more features like beards are particularly challenging for Sophia to draw in a short amount of time. However, Hanson Robotics has been able to reduce her drawing time to about seven minutes. 

Conclusion

While there are other robots who know how to draw, Sophia’s achievement is unique in that she was able to learn using her general-purpose, small human-like arms. Her arms were not designed specifically to draw, but rather drawing is just one of many possible use cases. This shows how universal her arms and hands can be while still maintaining human-like features and movement, as opposed to industrial arms which are designed around a very narrow set of tasks. 

The ability of AI and robots to produce original drawings raises many questions about the root of human creativity, and pushes the boundaries of what we can consider to be art. True to form, Sophia the Robot is at the forefront of the conversation, blurring the line between art and technology.

Carolyn Ayers, Head Writer, Hanson Robotics Limted

Photo and Video Credits:  Hanson Robotics Limited