Orbit Music Video

Orbit Music Video

Orbit Music Video

Orbit Music Video

Orbit Music Video

NAO | MICROSOFT | LISTEN
NAO | MICROSOFT | LISTEN
Project Type
Commission
Commission
Commission
Commission
Year
Year
2019
2019
2019
2019

We teamed up with collaborators Microsoft & Listen, to create an interactive music video for Nao's song Orbit. Using a host of tech including AI, Depthkit, and the Microsoft Kinect, we developed the music video in WebGL for desktop & mobile. Our team designed the look and feel, coded particle systems, camera moves, and integrated realistic constellation data from the night sky.

We teamed up with collaborators Microsoft & Listen, to create an interactive music video for Nao's song Orbit. Using a host of tech including AI, Depthkit, and the Microsoft Kinect, we developed the music video in WebGL for desktop & mobile. Our team designed the look and feel, developed particle systems, camera moves, and integrated realistic constellation data from the night sky.

We teamed up with collaborators Microsoft & Listen, to create an interactive music video for Nao's song Orbit. Using a host of tech including AI, Depthkit, and the Microsoft Kinect, we developed the music video in WebGL for desktop & mobile. Our team designed the look and feel, developed particle systems, camera moves, and integrated realistic constellation data from the night sky.

Hero2_On_Monitor_v3
Hero2_On_Pixel_v5

To start, enter your birthday and get a customized experience based on your horoscope. As you watch the music video change the colors of particles, the stars, and the sky using sliders. The video ends with a downloadable asset to share on social channels.

Design / Process

Design / Process

Design / Process

Design / Process

Capturing & Rendering Data

We knew early on that we'd need to render Nao's form using a particle system driven by captured 3D data. To do this, we shot with a 4K camera alongside a Microsoft Kinect and used Depthkit to pair the two. We then exported a combined RGB+Depth map file out of DepthKit, which is what we use to drive the experience.

Live Action Shoot
1_Shoot
2_BodyDepth
3_Final

From left to right: Keyed footage from the shoot, Depthkit RGBD footage, Final music video.

From top to bottom: Keyed footage from the shoot, Depthkit RGBD footage, Final music video.

From top to bottom: Keyed footage from the shoot, Depthkit RGBD footage, Final music video.

SOLID SQUIGGLE

Composing Camera Moves in WebGL

The majority of the video was driven by a single wide shot. Our team built custom tools to animate camera movements in WebGL to match a pre-rendered animatic shot for shot.

Originally we stored XYZ position and a quaternion per keyframe. In the end we got much smoother motion by breaking out the camera XYZ position, the target position, and an up-vector. Treating them as three separate tracks gave us much more fluid camera controls.

SOLID SQUIGGLE

Constellation AI Triggers

A video capture of the experience was run through an AI classifier to identify moments when the shape of the dancers' bodies matched up with actual constellations. We animated 12 horoscope constellations in 3D and positioned them in space to match up with the dancers.

AI_1
AI_2
AI_3

SOLID SQUIGGLE

Cutting Room Floor

A small collection of design & code sketches from the discovery phase.

Skeletal_CodeSketch_v2
BodyStyles_Sketck_v2
Body_LinesBlocks

The Team

The Team

Design Director:
Philip Sierzega

Technical Director:
Charlie Whitney

Executive Producer:
Jennifer Vance

Design Director:
Philip Sierzega

Technical Director:
Charlie Whitney

Executive Producer:
Jennifer Vance

Producers:
Jessica Ferrer
Sue Yee Hubbard

Design Director:
Philip Sierzega

Technical Director:
Charlie Whitney

Executive Producer:
Jennifer Vance



Lead WebGL Programmer:
Lars Berg

Front End Development:
Polymer Mallard

Producers:
Jessica Ferrer
Sue Yee Hubbard

Lead WebGL Dev:
Lars Berg

Front End Dev:
Polymer Mallard

Lead WebGL Programmer:
Lars Berg

Front End Development:
Polymer Mallard

Producers:
Jessica Ferrer
Sue Yee Hubbard

 

Depth Capture Director:
Andrew Gant

3D Design/Animator:
Terra Henderson 

Depth Capture Director:
Andrew Gant

3D Design/Animator:
Terra Henderson

 

Client Side

Client Side

Agency:
Listen

Agency:
Listen

Brand Client:
Microsoft

Music Artist:
Nao



Brand Client:
Microsoft

 

 

 

Music Artist:
Nao

 

Contact

Contact

Say Hello
& Inquiries

Say Hello
& Inquiries

Say Hello
& Inquiries

On the
Internet

On the
Internet

On the
Internet

Studio
Address

Studio
Address

Studio
Address

Birth is death's first victory. Copyright 2016-2019. All Rights Reserved.

Birth is death's first victory.
Copyright 2018. All Rights Reserved.