SCT

Record Video and Camera Movement

Are you a movie maker enthusiast? Do you want to add CG content to your movies and want an easy way to synchronise real and virtual cameras? Or are you looking at creating Mixed Reality footage? Then Spatial Camera Tracker is for you!

Spatial Camera Tracker is a virtual production tool allowing you to capture HD video together with 3D positional tracking information of both the camera and a person being filmed! If you have a LiDAR compatible device you can even capture the surrounding environment. No external sensors needed which means you can record anywhere and over huge distances using only your iPhone or iPad!

Even though the images below are from iPad, all features are available for iPhone depending on your device capabilities.

Main View

When launching SCT you are presented with the main view where you access most of the controls and features. Users of the normal Camera app bundled with iOS will feel at home.

Record Button - Press to start recording and press again to stop an ongoing recording. If the button is greyed out it's because SCT haven't acquired tracking yet.
User Anchor Button - User anchors represents locations in the real world that should be tracked and lets you anchor virtual content to those locations. Press the + button just below the record button to place a user anchor on a tracked surface. The anchor is placed using the crosshair in the middle of the screen. To remove an existing anchor, aim at it and press the + button again.
Share Button - Press this to view captured sessions and share them via standard iOS sharing mechanisms.
Settings - Let's you control the behaviour of the app
Capture Modes - Swipe your finger left and right across the screen to change capture mode: World Tracking, Green Screen, Body Tracking

World Tracking

6 DOF tracking of the phone's location and rotation in 3D space. Use this mode to align CG greenscreen footage with camera recorded footage.

Green Screen Recording

Same as Camera World Tracking but with a green screen effect applied segmenting out any humans being recorded. Use this mode to align camera recorded footage with CG rendered scenes. Requires a device with an A12 chip or later.

Body Tracking

Full human skeleton joint tracking of a single person being filmed. Use this mode to anchor CG effects to the person being filmed.Requires a device with an A12 chip or later.

LiDAR

Spatial Camera Tracker also supports capturing the surrounding environment geometry using the LiDAR sensor on the latest iPad or iPhone Pro! Use this mesh to line up CG objects and effects with the real world recorded footage or simply to help recreate a virtual version of the real world.

Transfer data

When you need to transfer your recorded data off the device, press the "Share Button" to view all captured sessions, replay video, transfer the data using standard iOS share mechanisms. To view the video, simply tap the thumbnail. To bring up the "Actions" menu, long press a thumbnail.

Unreal, Blender and Unity Support

Import the tracking information into Unreal or Blender or Unity and drive virtual cameras, replaying the original camera movements, to capture CG footage and combine with your real life footage using standard video editing software or in-engine techniques. As the information is recorded you can iterate on you virtual scenes until you are happy with the result.

Plugins - To make use of the recorded spatial data, use one of the plugins hosted on Github
Unreal: https://github.com/Kodholmen/SCT-Unreal
Blender: https://github.com/Kodholmen/SCT-Blender
Unity: https://github.com/Kodholmen/SCT-Unity

Is your tool not supported? Fear not! The spatial data format used by SCT is no secret and the format is documented with the plugins. Feel free to write your own parser or modify the existing. They are all MIT-licensed.

Compositing

Use your favourite editing tools to combine your captures and create the final footage. In engine or with traditional editing tools. The choice is yours!

Kodholmen AB

Spatial Camera Tracker