Almost every problem in ML and data science starts with the same ingredients:

• The dataset $\mathbf{x}$ (could be some observable quantity of the system we are studying)
• A model which is a function of a set of parameters $\mathbf{\alpha}$ that relates to the dataset, say a likelihood function $p(\mathbf{x}\vert \mathbf{\alpha})$ or just a simple model $f(\mathbf{\alpha})$
• A so-called cost function $\mathcal{C} (\mathbf{x}, f(\mathbf{\alpha}))$ which allows us to decide how well our model represents the dataset.

We seek to minimize the function $\mathcal{C} (\mathbf{x}, f(\mathbf{\alpha}))$ by finding the parameter values which minimize $\mathcal{C}$. This leads to various minimization algorithms. It may surprise many, but at the heart of all machine learning algortihms there is an optimization problem.