Systems & Research
Synthetic Data Pipelines for Digital Humans
A rendering pipeline in Unreal Engine that produces labeled, multi-view video of digital humans for training models that reconstruct people from several cameras.
Synthetic data & digital humans
Synthetic data for training and testing AI.
Models that reconstruct a moving person from several cameras need many people, outfits, and motions seen by synchronized, calibrated cameras and labeled on every frame. Capturing that is slow and imprecise; rendering it is exact only if every camera sees the same instant.
For teams training perception, reconstruction, or physical-AI models that need labeled multi-view data real capture cannot deliver.
Worked example
A capture studio that exists only in the engine
For a team training multi-view human reconstruction models we built a rendering pipeline in Unreal Engine. Digital humans come from stock MetaHuman presets, varied in identity, outfit, and hair, and animated with SMPL-X motion capture retargeted to the MetaHuman skeleton. A tuning loop compares engine renders against the reference video and adjusts the conversion on its own.
A rig of calibrated color and infrared cameras renders every view on the same engine tick, so physics, hair, and particles match across views. Each camera writes depth, optical flow, normals, albedo, instance segmentation, object coordinates, 2D boxes, skeletal and deformed-mesh exports, and hair guides.
Batches run on a cloud render farm and land in a content-addressed registry as labeled multi-view clips. Visualization tooling checks the labels against a public 3D pose benchmark before a model trains on them.
Read the case study: Synthetic Data Pipelines for Digital Humans
How it works
- Unreal Engine 5 and MetaHuman
supply the digital humans, the physics, the hair, and the renderer, extended with our own plugins where the stock passes are not enough.
- SMPL-X retargeting
moves motion-capture datasets onto the engine skeleton, tuned automatically against the reference video.
- Movie Render Queue and a cloud render farm
render every camera on one engine tick and scale a batch to millions of frames.
- Ground-truth passes and a content-addressed registry
label every frame per camera and keep every asset and output traceable.
What we can build
Digital human generation
Identity, outfit, and hair variants generated from engine presets, animated with retargeted motion capture.
Motion retargeting
SMPL-X and other mocap formats converted to engine skeletons, tuned automatically against reference video.
Synchronized multi-camera rendering
Calibrated color and infrared cameras rendered on the same engine tick, with intrinsics and extrinsics in every output.
Ground truth
Depth, optical flow, normals, albedo, segmentation, boxes, skeletons, and meshes per frame and per camera.
Render orchestration
Cloud render farms, content-addressed asset registries, and batch runs that scale to millions of frames.
Dataset tooling
Visualization and validation against public benchmarks, so labels are checked before a model trains on them.
Where to start
A small labeled set for one rig
We render a small labeled multi-view dataset for one camera rig and your team checks it against its own model before anything scales.
- What you get
- The dataset, its calibration and label formats, and a written note on what to change before a full run.
- What you bring
- The camera rig: count, placement, intrinsics
- The labels and formats your training code reads
- A way to check the output against the model
Next step
Scope a synthetic data pipelineGet in touch
Tell us what you have in mind.
Describe it in a few sentences. You get a draft brief to review before anything is sent.