Synthetic Data Pipelines for Digital Humans
A rendering pipeline in Unreal Engine that produces labeled, multi-view video of digital humans for training models that reconstruct people from several cameras.
What it is
Models that reconstruct a moving person in 3D from several cameras need training data that is hard to capture: many people, outfits, and motions, seen from synchronized cameras with exact calibration, and labeled with depth, motion, and body pose for every frame. Rendered data can be exact, if every camera sees the same instant and the variety is wide enough to matter.
What we built
Digital humans generated from stock MetaHuman presets and varied in identity, outfit, and hair. Body animation from SMPL-X motion-capture datasets retargeted to the MetaHuman skeleton, with an autonomous tuning loop that compares engine renders against the reference video and adjusts the conversion. A rig of calibrated color and infrared cameras that renders every view on the same engine tick, so physics, hair, and particles match across views. Ground-truth passes per camera: depth, optical flow, normals, albedo, instance segmentation, object coordinates, 2D boxes, skeletal and deformed-mesh exports, and hair guides. Cloud render-farm orchestration, a content-addressed asset registry, a second app for pedestrian multi-view capture in a city scene, and visualization tooling that checks labels against a public 3D pose benchmark.
How it is used
Batches of combinations run on the render farm and land in the registry as labeled multi-view clips, ready to train multi-view and 4D human reconstruction models. The same pipeline produces facial performance data from simulated capture rigs.
Built with
Credit
Built by Dream Awake Labs. The diagram is ours, and the digital humans come from stock engine presets.
Get in touch
Tell us what you have in mind.
Describe it in a few sentences. You get a draft brief to review before anything is sent.