What is Regularization?

What is regularization in machine learning? Explain why we use it and describe the main types.

Hints

Hint 1

Think about what happens when a model learns the training data too well. What problem does this cause?

Hint 2

Regularization adds something extra to the loss function. What would penalizing large weights accomplish?

Hint 3

There are two main types of regularization: L1 (Lasso) and L2 (Ridge). What's the key difference in how they penalize weights?