Genjustsu swaps characters in a video and keeps the original voice

A scene reshoot used to need a studio day and a video editor. Genjustsu puts the same job inside a local web interface, swaps the character or object in the shot with an AI-generated one, keeps the original voice, and sends the compute to hosted APIs so the laptop running it does not need a GPU. The repository shows 955 stars and sits at 243 forks under the MIT licence.

Writing in the Genjustsu README on GitHub, Sirio Berati of TEIN describes a Python workflow that chains five hosted models and three local libraries into a single run. The input is an MP4 or MOV video plus one or more reference images of the new character. The output is a 1080p MP4 with the subject replaced and the original audio restored.

What is wired to what

Berati names each component in the README. Depth is estimated by lucataco/depth-anything-video on Replicate. Masks come from lucataco/sam3-video on Replicate, frame by frame. An optional fast mode uses Google's MediaPipe Face Landmarker to track faces instead of running segmentation on every frame. Vocals are isolated by Demucs on Replicate, so the final cut keeps the speaker's voice. The pitch is shifted up three semitones by the Rubber Band command-line tool. Generation itself runs on Enhancor's Seedance API.

Local processing uses Python 3.12 or higher, FastAPI for the web interface, PyAV, OpenCV and NumPy. The README states the default setup needs no local model weights and no GPU, since every heavy step is a hosted call. The user installs Python packages and Rubber Band and nothing larger.

Where the work still sits with the user

The local web interface is where a reviewer approves what each step produced. Berati's README names the steps as inputs, masks, generations and export, each with its own preview, which is the gate against a run that burns API budget on a mask the user never saw.

A reader with a scene to swap can read what the workflow does, pay the API bills on Replicate and Enhancor, and keep the review on their own machine. The repository is MIT for the code and documentation and makes clear that the artwork in the readme is not covered by that licence. A product marketer with a one-shot ad to reshoot or a creator with a character to retire from an old scene gets a run they can rerun with different reference images without touching an editor.

What a buyer should check before adopting it, since the README does not say: what Replicate and Enhancor charge for a minute of 1080p output, how long one pass takes on a laptop, and whether Seedance will output a face the client agrees looks like the reference. The licence sets the floor for reuse; the hosted API bills set the ceiling for how often it runs.

Source

Genjustsu Open-Source Workflow, Sirio Berati on GitHub.