Turning my computer into Van Gogh

Creating art with Neural Style Transfer

Guess which artist painted this?

Content + Style = Baby image

Covenulational Neural Network (CNN) built for images

CNN’s specialize in visual information; they’re great for finding patterns in images. CNN’s have layers that extract different features from an image.

  1. Calculate the loss function of the Style
  2. Combine weighted loss function of both the style + content
  3. Use backpropagation to reduce the overall loss function
Random Noise(G)

Content: what we paint

Content has to do with what we’re painting rather than how it’s painted, different artists can paint a cat in different styles, but the object is found in both images.

VVG-19 CNN
Mean square error loss function

Style: how we paint

Van Gogh can paint hundreds of different objects but all in a similar style.

Feature maps at layer l
finding dot products of the activations to ger gram matrix
gram matrix for the Style image (S)
gram matrix for the Generated image (G)
the loss function for style

Overall loss = Content loss + Style loss

The goal of the algorithm is to minimize the loss functions of both style and content.

Overall loss function

Executing Neural Style Transfer

  1. Find activations at 4_2 layer of our content image, when passing it through VGG-19
  2. Find gram matrixes and activations at multiple layers of our style image, when passing it through VGG-19
  3. Generate a random image
  4. Run random image through VGG-19, repeating 1&2 for the generated image, repeatedly run backpropagation until loss function is minimized.

Playing around with blockchain and machine learning. 17. Trying to understand my self and the world.