If you're not going for pixel art, use the video to video models (the character swap ones that take in a video reference of the animation and a still of the character (the tiktok deepfake models)).
You can e.g. download a run cycle from Mixamo, use Blender to render a video from whatever angle you wish, and feed that into the model along with your sprite on a green background, it's pretty much perfect.
Thanks for the info! I'll give this a shot. Most of my knowledge is from over a year ago, when I would set up OpenPose-style positioning with a controlnet at a high strength to guide an image-to-image workflow in comfyui. But it was pretty tedious and prone to errors.
Comments
If you're not going for pixel art, use the video to video models (the character swap ones that take in a video reference of the animation and a still of the character (the tiktok deepfake models)).
You can e.g. download a run cycle from Mixamo, use Blender to render a video from whatever angle you wish, and feed that into the model along with your sprite on a green background, it's pretty much perfect.
Thanks for the info! I'll give this a shot. Most of my knowledge is from over a year ago, when I would set up OpenPose-style positioning with a controlnet at a high strength to guide an image-to-image workflow in comfyui. But it was pretty tedious and prone to errors.