Orbit Music Video
Orbit Music Video
Orbit Music Video
Orbit Music Video
Orbit Music Video
NAO | MICROSOFT | LISTEN
NAO | MICROSOFT | LISTEN
Project Type
Commission
Commission
Commission
Commission
Year
Year
2019
2019
2019
2019
We teamed up with collaborators Microsoft & Listen, to create an interactive music video for Nao's song Orbit. Using a host of tech including AI, Depthkit, and the Microsoft Kinect, we developed the music video in WebGL for desktop & mobile. Our team designed the look and feel, coded particle systems, camera moves, and integrated realistic constellation data from the night sky.
We teamed up with collaborators Microsoft & Listen, to create an interactive music video for Nao's song Orbit. Using a host of tech including AI, Depthkit, and the Microsoft Kinect, we developed the music video in WebGL for desktop & mobile. Our team designed the look and feel, developed particle systems, camera moves, and integrated realistic constellation data from the night sky.
We teamed up with collaborators Microsoft & Listen, to create an interactive music video for Nao's song Orbit. Using a host of tech including AI, Depthkit, and the Microsoft Kinect, we developed the music video in WebGL for desktop & mobile. Our team designed the look and feel, developed particle systems, camera moves, and integrated realistic constellation data from the night sky.
To start, enter your birthday and get a customized experience based on your horoscope. As you watch the music video change the colors of particles, the stars, and the sky using sliders. The video ends with a downloadable asset to share on social channels.
Design / Process
Design / Process
Design / Process
Design / Process
Capturing & Rendering Data
We knew early on that we'd need to render Nao's form using a particle system driven by captured 3D data. To do this, we shot with a 4K camera alongside a Microsoft Kinect and used Depthkit to pair the two. We then exported a combined RGB+Depth map file out of DepthKit, which is what we use to drive the experience.
From left to right: Keyed footage from the shoot, Depthkit RGBD footage, Final music video.
From top to bottom: Keyed footage from the shoot, Depthkit RGBD footage, Final music video.
From top to bottom: Keyed footage from the shoot, Depthkit RGBD footage, Final music video.
SOLID SQUIGGLE
Composing Camera Moves in WebGL
The majority of the video was driven by a single wide shot. Our team built custom tools to animate camera movements in WebGL to match a pre-rendered animatic shot for shot.
Originally we stored XYZ position and a quaternion per keyframe. In the end we got much smoother motion by breaking out the camera XYZ position, the target position, and an up-vector. Treating them as three separate tracks gave us much more fluid camera controls.
SOLID SQUIGGLE
Constellation AI Triggers
A video capture of the experience was run through an AI classifier to identify moments when the shape of the dancers' bodies matched up with actual constellations. We animated 12 horoscope constellations in 3D and positioned them in space to match up with the dancers.
SOLID SQUIGGLE
Cutting Room Floor
A small collection of design & code sketches from the discovery phase.
The Team
The Team
Design Director:
Philip Sierzega
Technical Director:
Charlie Whitney
Executive Producer:
Jennifer Vance
Design Director:
Philip Sierzega
Technical Director:
Charlie Whitney
Executive Producer:
Jennifer Vance
Producers:
Jessica Ferrer
Sue Yee Hubbard
Design Director:
Philip Sierzega
Technical Director:
Charlie Whitney
Executive Producer:
Jennifer Vance
Lead WebGL Programmer:
Lars Berg
Front End Development:
Polymer Mallard
Producers:
Jessica Ferrer
Sue Yee Hubbard
Lead WebGL Dev:
Lars Berg
Front End Dev:
Polymer Mallard
Lead WebGL Programmer:
Lars Berg
Front End Development:
Polymer Mallard
Producers:
Jessica Ferrer
Sue Yee Hubbard
Depth Capture Director:
Andrew Gant
3D Design/Animator:
Terra Henderson
Depth Capture Director:
Andrew Gant
3D Design/Animator:
Terra Henderson
Client Side
Client Side
Agency:
Listen
Agency:
Listen
Brand Client:
Microsoft
Music Artist:
Nao
Brand Client:
Microsoft
Music Artist:
Nao
Contact
Contact
Say Hello
& Inquiries
Say Hello
& Inquiries
Say Hello
& Inquiries
On the
Internet
On the
Internet
On the
Internet
Studio
Address
Studio
Address
Studio
Address
Birth is death's first victory. Copyright 2016-2020. All Rights Reserved.
Birth is death's first victory.
Copyright 2018. All Rights Reserved.