Adobe created a technical, art-informed activation for the annual de Young Art Party, which raises money for local children’s arts programs. We designed and created a hands-on experience that gave children the opportunity to unlock their tactile creativity and see their analogue artwork come to life. Early on, we agreed that Adobe’s installation should highlight creativity across the analogue and digital domains, reflecting Adobe’s role in fostering creativity.

While Augmented Reality is a popular technology, it is typically experienced via mobile screens. Here, we wanted to showcase room-scale AR in an immersive, rather than alienating, manner. First, children were invited to use markers and colored pencils to design and decorate an analogue butterfly template. Then, a scanner would parse the colors and strokes from the child’s drawing, subsequently transferring the data onto the wings of a 3D representation of the butterfly, which became fully animated for the children to enjoy. Then, our activation utilized human pose tracking technology to create live AR effects as the children interacted with the animated butterflies, colored according to the child’s individual design. The pose-tracking technology is able to continuously detect human body joints, such as head, shoulders, arms, wrists, legs and feet from a camera video stream in real-time. Based on this information, the system animated the children’s own virtual butterflies to follow their arm movements or to land on their bodies, as the children looked on with excitement.