Animating a 15th century Tibetan Thangka painting (A visual effects test)

How, through advanced stereoscopic animation techniques, can the illusion of life respectfully be imbued into an evocative, 15th century Tibetan Thangka painting?

Film Director Bryan Single working with Animation Director Jim Ellis and master, Tibetan Thangka painter Pema Namdol Thaye established a larger animation team that set about actively perusing this creative challenge, which was proposed to us by IMAX Corporation in China.   Using a single 16K pixel resolution photographic image of a 15th century painted masterwork depicting the Bodhisattva Avalokitesvara (held in The Rubin Museum of Art and extended to us for testing purposes by Himalayan Arts Resources), we set about reconstructing the flat painted medium into the realm of stereoscopic 3D motion and 4K animation.  The plan was to adhere the flat 2D painting onto a custom-created, real-time 3D kinematic animation-control-system with a custom skeletal/joint system capable of properly controlling three-dimensional body motion for Avalokitesvara, including all eleven of his heads and  his “thousand-arms” (at least the 42 in the painting).

Starting with the digital photographic replica, the painting was first restored, but only where necessary as to retain its history and age.  Key elements (monks, structures, vines, flowers) were then isolated into separate Z-depth-layers. These elements (figures, bits of lotus blossoms) were then spatially layered in front and behind each other at varying distances from each other.  Each layer was enlarged or shrunk to retain its proper proportions, and to establish 3D depth and scale through a background-foreground relationship. A 3D space was created for 3D motion, including virtual lighting and stereoscopic cameras. This isolation of key elements was done for sections of the Bodhisattva’s body as well.  Skilled matte painters Kate McCabe and Jessica Thaye digitally separated out the 42 arms and eleven heads, the clothing and jewelry, eyes, and mouths.  The new animation/motion frames were then meticulously painted one by one by Pema, skillfully keeping the style of these new paintings consistent with that of the original while also painting subtle variation in facial expression, as well as the broader motion of the arms in prostration.  Pema also patiently created 24 painted motion variations for each second of animation of screen time.  These additional motion paintings are then recombined with the original attributes of the Bodhisattva, and adhered to a 3D animation skeletal/joint control system created by Peter Walker.  Now with any motion within the skeletal system, the body parts (sections of painting) remain attached to the control-bones, and bend properly at the control joints for the 42 arms and eleven heads moving in 3D space.

Depth-Map gradients were digitally painted for each element by Sculptor Andrew Klein to further transition the 2D painting into the realm of 3D.  These Depth-Maps push and pull the previously flat surface of the painting, enabling eyes, mouths, ears, noses and chins to be sculpted out of the flat painting, along with body curvature that the animator can “dial” up or down to transitioning between “the flat” and “the 3D”.  And with all those bump maps comes the 3D interplay of virtual light and shadow upon the surfaces.

With the control system in place, “takes” of real-time animation performance were now able to be recorded for the 42 moving arms.  Once a “keeper take” of gesture-captured animation had been generated for the prostration of the frontal arms, it is shined and tweaked.  This animation is then blended with prime number-based harmonic algorithms to control the motion for the remaining arms, creating a motion for 42 arms that harmoniously drift in and out of synchronization, while retaining the weight, motion, and character inherent in arms.  Any “keeper takes” are added to other animation takes, such as transitioning facial expressions, head position, shimmering halos, virtual lighting, and camera motion.  Real-world atmospherics are then integrated (flame and fog) to retain a sense of connection to the non-animated world that the unaltered original painting still exists in as an object, and the viewer still inhabits.  Each animation take is patiently performed and refined, integrated and edited, until a sense of life, awareness, mystery, and transcendence develop between the clip and the viewer.

All of the aforementioned were the foundational steps for a broader, composite visualization of the Bodhisattva painting set within a live-action tableau.  It’s all a work-in-progress for a larger fiction project in development, but we hope this small step inspires you.  Here’s a screen grab of the 2D rendered video.

Written by Jim Ellis and Bryan Single