big-sleep modified to store latents together with intermediate images

From text to image:

python gen.py "a castle made of ice"

This will output, for each epoch, both an image and a checkpoint pth file (which can then be used to generate a morph between two stored checkpoints).

Generate morph:

python bsmorph.py --lat1 examples/a_castle_made_of_ice.1.pth --lat2 examples/a_woodcarving_clock.2.pth --savePath morf --name ice2wood --steps 200

You can then use ffmpeg to make a video from the generated frames, e.g.

ffmpeg -r 12 -i morf/ice2wood%d.png -vf fps=25 -pix_fmt yuv420p ice2wood.mp4

---

artificial intelligence

cosmic love and attention

fire in the sky

a pyramid made of ice

a lonely house in the woods

marriage in the mountains

lantern dangling from a tree in a foggy graveyard

a vivid dream

balloons over the ruins of a city

Big Sleep

Ryan Murdock has done it again, combining OpenAI's CLIP and the generator from a BigGAN! This repository wraps up his work so it is easily accessible to anyone who owns a GPU.

You will be able to have the GAN dream up images using natural language with a one-line command in the terminal.

Original notebook

Simplified notebook

Install

$ pip install big-sleep

Usage

$ dream "a pyramid made of ice"

Images will be saved to whereever the command is invoked

Advanced

You can invoke this in code with

from big_sleep import Imagine

dream = Imagine(
    text = "fire in the sky",
    lr = 5e-2,
    save_every = 25,
    save_progress = True
)

dream()

You can also set a new text by using the .set_text(<str>) command

dream.set_text("a quiet pond underneath the midnight moon")

And reset the latents with .reset()

dream.reset()

To save the progression of images during training, you simply have to supply the --save-progress flag

$ dream "a bowl of apples next to the fireplace" --save-progress --save-every 100

Alternatives

Deep Daze - CLIP and a deep SIREN network

Citations

@misc{unpublished2021clip,
    title  = {CLIP: Connecting Text and Images},
    author = {Alec Radford, Ilya Sutskever, Jong Wook Kim, Gretchen Krueger, Sandhini Agarwal},
    year   = {2021}
}

@misc{brock2019large,
    title   = {Large Scale GAN Training for High Fidelity Natural Image Synthesis}, 
    author  = {Andrew Brock and Jeff Donahue and Karen Simonyan},
    year    = {2019},
    eprint  = {1809.11096},
    archivePrefix = {arXiv},
    primaryClass = {cs.LG}
}