Topics for Exam 2
Topics for Exam 2
General background
  • Python, NumPy, and Pytorch fundamentals
  • General concepts and functions
  • Vector, matrix, and tensor operations
  • Equations of lines and planes in multi-dimensional space (hyperplanes)    
Conceptual Understanding (From Textbook)
  • Neuron model and network architectures
  • Single neuron and activation functions
  • Layers of neurons, weight matrices, and biases
  • Multi-layer neural network architecture
  • Decision boundaries and their relationship to hyperplanes
  • Performance surfaces and optimum points
  • Gradient, Hessian, and Taylor series
  • Directional derivatives
  • Minima and maxima
  • Necessary and sufficient conditions for optimality
  • Performance Optimization
  • Steepest Descent
  • Minimizing along a line
Supplementary Topics
  • Understanding Computational Graphs and their forward and backward passes
  • Loss functions: MSE, MAE, Hinge, and Cross-entropy
  • PyTorch:
  • Loading and preparing datasets
  • Creating and manipulating tensors
  • Building multi-layer neural networks
  • Setting the loss function and performance measure
  • Computing outputs, errors, and gradients
  • Training and adjusting weights
  • Evaluating model performance
  • Convolutional Neural Networks (CNN):
  • Creating and analyzing CNNs using PyTorch
  • Understanding convolutional filters, padding, and stride.
  • Creating convolutional, pooling, flattening, and fully connected layers
  • Determining shape of the weight matrix
  • Determining shape of the output for each layer
  • Determining number of parameters
  •  Autoencoders
    • Encoder and decoder
    • Latent space
    • Variational autoencoders
    • Implementation and training
  • Transformers
    • Multi-Head attention
    • Feed forward network structure
    • Queries, Keys, and Values
    • Positional Encoding
    • Decoder structure
    • Masked multi- head attention
  •  Stable Diffusion
  • Overview of diffusion models: Forward and reverse diffusion processes.
  • Adding and progressively denoising Gaussian noise.
  • The role of Gaussian noise in generating latent representations.
  • Sampling techniques for generating outputs from noise.
  • U-Net
 
 
Coding Component
 
The exam will include coding questions requiring you to write or complete Python code (using NumPy or PyTorch) based on concepts covered in lectures or assignments.
These questions may be similar to or derived from portions of your assignments.
 
 
Top