While we thought we were safe, Artificial Intelligence is starting to encroach on the creative professions. We can either fight it or learn to use it, so here is my rolling list of potentially useful AI tools and repos for image and video manipulation. I haven’t checked out all of these in detail, but hopefully a useful starting point if you are interested in still and video creation and processing. Newer links tend to be at the top of each section.

Image creation

• Imagen – research only but super impressive image creation based on text prompt https://imagen.research.google/
• Dall-e 2 – beta (with waiting list) of image creation based on text prompt – https://openai.com/dall-e-2/
• mini-Dall-e- simpler version of original Dall-e but (sometimes) accessible to use online – https://huggingface.co/spaces/dalle-mini/dalle-mini
• midjourney – beta (with waiting list), more artistic image generation via Discord – https://www.midjourney.com

Offline Video upscaling software

• https://videoai.topazlabs.com/ – probably best known commercial solution – not good for interlaced but costs are reasonable – good analysis here: https://captrobau.blogspot.com/2019/10/experimenting-with-gigapixel-ai-for.html
• DVDFab – AI upscaling bundled with DVD ripping software https://m.dvdfab.cn/tips/ai-video-upscaling
• Open source upscaling project – https://github.com/HENDRIX-ZT2/Deep-Video-Deinterlacing
• Pixop – Pay-as-you-go will clean up old film and upscales pretty well and deinterlacing – https://www.pixop.com/

Video restoration and colourisation

• https://github.com/satoshiiizuka/siggraphasia2019_remastering – open-source with pretrained model.

Image fixing

• https://blog.floydhub.com/colorizing-and-restoring-old-images-with-deep-learning/
• https://github.com/DmitryUlyanov/deep-image-prior
• https://letsenhance.io/ – has commercial API

Image colourisation

• https://www.myheritage.com/incolor -colourise or enhance faded colour – online tool
• https://github.com/junyanz/interactive-deep-colorization Open Source AI tool with human picking colour in a GUI, (very useful for things like hair colour)
• Plus Older automatic tool – https://demos.algorithmia.com/colorize-photos – more tinting than colouring
• https://imagecolorizer.com/colorize.html – free online colourisation – one of the better ones for simple portraits although still very random on more than simple compositions

Image upscale service/software (with free plans)

• https://imglarger.com – best for portraits
• https://imgupscaler.com
• https://deep-image.ai – also setting for artifacts
• Offline – use Photoshop’s native AI – increase ‘Image Size’ with ‘Resample’ on and set to use latest version of ‘Preserve Details’

Fill in missing section of images:

• context encoder – https://github.com/pathak22/context-encoder

Animate image / turn into video:

• https://www.myheritage.com/deep-nostalgia – add set animation to faces in the photo so they appear to come to life.
• https://github.com/alievk/avatarify-desktop  https://apps.apple.com/us/app/avatarify-ai-face-animator/id1512669147 – control face in still by tracking your face in webcam.
• https://bhaasha.iiit.ac.in/lipsync/ add lipsync to still or video
• https://github.com/eyaler/avatars4all selection of tools for using ‘style’ of one image with movement from another face or full body

3D

• NERF – using relatively small numbers of images to create 3D meshes with surface textures (like supercharged photogrammetry) https://github.com/bmild/nerf
• Threedy.ai – turns stills into 3D models (although largely limited to furniture and similar items comprising 3D shapes) – https://www.threedy.ai/
• playhybrid AI – decimates complex 3D models to make then work on limited bandwidth platforms without compromising on quality – https://www.playhybrid.com/chewitechnology

Motion Capture

• RADiCAL – cloud based server for capturing body movement via webcam, good foot lock and fast although no hand or finger tracking – https://getrad.co/
• PoseAI – mocap via iOS device – can do face and fingers, foot lock not as good – https://poseai.co.uk/