Empowering Children to Create AI-Enabled Augmented Reality Experiences

Lei Zhang, Shuyao Zhou, Amna Liaqat, Tinney Mak, Brian Berengard, Emily Qian, Andrés Monroy-Hernández.

UIST 2025, to appear

We introduce Capybara, an AR-based and AI-powered visual programming environment that empowers children to create, customize, and program 3D characters overlaid onto the physical world. Capybara enables children to create virtual characters and accessories using text-to-3D generative AI models, and to animate these characters through auto-rigging and body tracking. In addition, our system employs vision-based AI models to recognize physical objects, allowing children to program interactive behaviors between virtual characters and their physical surroundings.

Interactively Optimizing Layout Transfer for Vector Graphics

Jeremy Warner, Shuyao Zhou, Björn Hartmann

ICML AI-HCI Workshop 2023

One of the most common ways to represent and share visual designs is with vector graphics. Designers working with vector graphics often explore layout alternatives and generate them by moving and resizing elements. The motivation for this can range from establishing a different visual flow, adapting a design to a different aspect ratio, standardizing spacing, or redirecting the design's visual emphasis. Existing designs can serve as a source of inspiration for layout modification across these goals. However, generating these layout alternatives still requires significant manual effort in rearranging large groups of elements. We present VLT, short for Vector Layout Transfer, a novel graphic design tool that enables flexible transfer of layouts between designs. It provides designers with multiple levels of semantic layout editing controls, powered by automatic graphics correspondence and layout optimization algorithms.

Review of light field technologies

Shuyao Zhou, Tianqian Zhu, Kanle Shi, Yazi Li, Wen Zheng, Junhai Yong

VCIBA Journal 2021

Light fields are vector functions that map the geometry of light rays to the corresponding plenoptic attributes. They describe the holographic information of scenes by representing the amount of light flowing in every direction through every point in space. The physical concept of light fields was first proposed in 1936, and light fields are becoming increasingly important in the field of computer graphics, especially with the fast growth of computing capacity as well as network bandwidth. In this article, light field imaging is reviewed from the following aspects with an emphasis on the achievements of the past five years: (1) depth estimation, (2) content editing, (3) image quality, (4) scene reconstruction and view synthesis, and (5) industrial products because the technologies of lights fields also intersect with industrial applications.

Code It, See It, Touch It: Vision-Powered AR Programming Tool for Children

Shuyao Zhou

Master Thesis