Deep learning

Many small models stacked in layers, each learning from the one before.

Neural networks

Weighted sums, activations and backpropagation.

What the network answers

  • / the two classes (filled: training)
  • validation (dark outline)

Loss per epoch

  • training loss
  • validation loss

The network: what each neuron has learned

Each hidden neuron draws its own picture of the plane.

Convolutional networks (CNN)

One small filter sliding over the whole image.

Image: Click or drag in the image to draw.

The image and the filter's place

Feature map

After 2×2 max pooling

A filter that learns

The filters are weights, learnt with gradient descent.

The filter's nine weights

Loss per epoch

  • training loss
  • validation loss

Press Train the filter.

Recurrent networks (RNN)

A hidden state carried from one character to the next.

Loss per epoch

The network writes

It learns:

Given the first character, it feeds its own best guess back in:

Press Train.

Unrolled: one column per character

Each column is one step.

How far back the memory reaches

The vanishing gradient, and LSTM and GRU.

Semantic search and RAG

Chunks, vectors, cosine similarity and the prompt.

Try:
Use your own text

The chunks, best match first

    The question as a vector

      The largest weights in the question's vector.

      The chunks on a map

      PCA from part 7.

      The prompt a language model would get

      
    
      The augmented in RAG.
      
    
      Word counts and real embeddings
      
    
      Real embeddings are made by a neural network.
      
    
      
  
      
  
  
      
    
      Attention
      
    
      A soft lookup: query, key and value.
      

    
      A soft lookup
      
    
      Click to place the query.
      
    
      
      
    
      
    
      
      
      
        
      
      
      
      
        
        
      
      
      
    
      

    
      Self-attention in a sentence, two layers
      
    
      Who is "it"?
      
    
      
      
      
        The last word:
        
        
      
      
    
      
    
      
      
      
        
      Layer 1: who looks at whom
      
        
      
      
      
      
      
        
      Layer 2: who looks at whom
      
        
      
      
      
    
      
    
      Each row is a word asking.