Facebook’s AI extracts playable characters from real-world videos

Remember those FMV games from the ’90s — the ones that blended prerecorded clips with animated sprites and 3D models? Facebook is bringing them back in style, and improved tenfold. In a newly published preprint paper on Arxiv.org (“Vid2Game: Controllable Characters Extracted from Real-World Videos“), scientists at Facebook AI Research describe a system capable of extracting controllable characters from real-world videos.

“Our method extracts a character from an uncontrolled video and enables us to control its motion,” the paper’s coauthors explain. “The model generates novel image sequences of that person … [and the] generated video can have an arbitrary background, and effectively capture both the dynamics and appearance of the person.”