Title: 544
Tags: lec6

Q: PyTorch: if y is a function of x, and you're trying to find the best x to optimize y, what do you need to do to let PyTorch know you'll later need a gradient?
A: x = torch.tensor(????, requires_grad=True)

Q: PyTorch: if y is a function of x, and you turned on gradient tracking for x, how can you get the y-over-x gradient (after you've calculated y)?
A: y.backward()
x.grad

Q: PyTorch: if you call .backward() on more than one result without zeroing gradients, what happens, say to x.grad?
A: x.grad will acculumate gradients: it will be the some of gradients with respect to all results on which you called .backward().

Q: PyTorch: say you've calculated y as a function of x, and you're trying to find a better x value.  You're using an optimizer opt for this purpose, and you've already called backward() to get gradients.
How do you make a small change to x?
A: opt.step()

Q: What does SGD stand for?
A: Stochastic Gradient Descent

Q: PyTorch: by default, the SGD optimizer is trying to get small results.  What should you pass to the SGD constructor if you want to get big results?
A: maximize=True

Q: PyTorch: what does the "lr" parameter for SGD stand for?
A: learning rate

Q: PyTorch: when you call optimizer.step(), how much will an input tensor x change?
A: x.grad * (learning rate of optimizer)
maximization: increase by that amount
minimization: decrease by that amount

Q: PyTorch: if you're optimizing y as a function of x and using an optimizer opt, how do you reset x.grad to zero?
A: opt.zero_grad()

Q: What are the big three areas of machine learning?
A: Reinforcement learning: agent makes series of actions to maximize reward
Unsupervised learning: looking for general patterns
Supervised learning: train models to predict unknowns

Q: What is the difference between regression and classification
A: regression: predict a number
classification: predict a category

Q: PyTorch: when training on a larger dataset, it is often efficient to use the whole dataset for each step of the optimizer.  What do we do?
A: We break the dataset into batches.  Within the loop over epochs, we have another loop over batches of the dataset.

Q: What does MSE stand for?
A: Mean Squared Error

Q: What PyTorch type lets us slice over our data, getting batches of features (x values) and corresponding batches of labels (y values).
A: Dataset.  An in-memory one is TensorDataset

Q: What PyTorch type provides additional features to a DataSet, like shuffling, batching, and iteration?
A: DataLoader