Illustration Track — the artwork follows my face

FasterLivePortrait · local GPU only · everything below is pre-rendered

This repo has two ways to animate a face, and which one works depends entirely on the artwork. The same model gives opposite results on different kinds of drawings, so we measured it and split the pipeline into two tracks.

Works Standard-proportion illustration · fine art

Vermeer, “Girl with a Pearl Earring” (1665, public domain). The brushwork and canvas craquelure stay intact while only the head pose and expression move.

Six frames of a driven vector illustration — the linework and coloring hold while only the expression changes — Six frames of a vector illustration. The linework and coloring are never regenerated, so the original style survives.

Doesn't work Hand drawings · doodles · chibi

LivePortrait applied to a hand drawing — no face is detected, so all three frames stay unchanged — Same pipeline, same webcam input. On a hand drawing **face detection fails outright**, so nothing happens at all — and if you force detection through, the entire head smears instead. Line art on white paper is outside the model's training distribution.

That is why hand drawings are handled by ARAP warp mirroring rather than this track. It deforms the original strokes over a triangle mesh, so it never invents pixels that were not there, and it runs in real time in the browser.

Why this can't run in the browser

Backend	Per frame	In practice
TensorRT (RTX 4070 Ti)	27–37 ms	Real-time (~30 fps)
ONNX GPU (same card)	118–124 ms	~8 fps — not real-time

Real-time is a TensorRT-only number. And a TensorRT engine is bound to the GPU and versions it was built on — it will not load on any other machine. Free hosting is CPU-only, so it is slower still, and round-tripping webcam frames through a server destroys the feeling that the drawing is following you. That is why this page only shows pre-rendered results.

Try the real-time mirroring How to run it yourself