搜索
Andrew Ng Linear regression with one variable Model representa6on Machine Learning Andrew Ng 0 100 200 300 400 500 0 500 1000 1500 2000 2500 3000 0 100 200 300 400 500 0 500 1000 1500 2000 2500 3000 Housing Prices (Portland, OR) Price (in 1000s of dollars) Size (feet2) Supervised Learning Given the “right answer” for each example in the data. Regression Problem Predict real-‐valued output Andrew Ng Nota6on: m = Number of training examples x’s = “input” variable / features y’s = “output” variable / “target” variable Size in feet2 (x) Price ($) in 1000's (y) 2104 460 1416 232 1534 315 852 178 … … Training set of housing prices (Portland, OR) Andrew Ng Training Set Learning Algorithm h Size of house Es6mated price How do we represent h ? Linear regression with one variable. Univariate linear regression. Andrew Ng Cost func6on Machine Learning Linear regression with one variable Andrew Ng How to choose ‘s ? Training Set Hypothesis: ‘s: Parameters Size in feet2 (x) Price ($) in 1000's (y) 2104 460 1416 232 1534 315 852 178 … … Andrew Ng 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 Andrew Ng y x Idea: Choose so that is close to for our training examples Andrew Ng Cost func6on intui6on I Machine Learning Linear regression with one variable Andrew Ng Hypothesis: Parameters: Cost Func6on: Goal: Simplified Andrew Ng 0 1 2 3 0 1 2 3 y x (for fixed , this is a func6on of x) (func6on of the parameter ) 0 1 2 3 -‐0.5 0 0.5 1 1.5 2 2.5 Andrew Ng 0 1 2 3 0 1 2 3 y x (for fixed , this is a func6on of x) (func6on of the parameter ) 0 1 2 3 -‐0.5 0 0.5 1 1.5 2 2.5 Andrew Ng 0 1 2 3 -‐0.5 0 0.5 1 1.5 2 2.5 y x (for fixed , this is a func6on of x) (func6on of the parameter ) 0 1 2 3 0 1 2 3 Andrew Ng Cost func6on intui6on II Machine Learning Linear regression with one variable Andrew Ng Hypothesis: Parameters: Cost Func6on: Goal: Andrew Ng (for fixed , this is a func6on of x) (func6on of the parameters ) 0 100 200 300 400 500 0 1000 2000 3000 Price ($) in 1000’s Size in feet2 (x) Andrew Ng Andrew Ng (for fixed , this is a func6on of x) (func6on of the parameters ) Andrew Ng (for fixed , this is a func6on of x) (func6on of the parameters ) Andrew Ng (for fixed , this is a func6on of x) (func6on of the parameters ) Andrew Ng (for fixed , this is a func6on of x) (func6on of the parameters ) Andrew Ng Gradient descent Machine Learning Linear regression with one variable Andrew Ng Have some func6on Want Outline: • Start with some • Keep changing to reduce un6l we hopefully end up at a minimum Andrew Ng θ1 θ0 J(θ0,θ1) Andrew Ng θ0 θ1 J(θ0,θ1) Andrew Ng Gradient descent algorithm Correct: Simultaneous update Incorrect: Andrew Ng Gradient descent intui6on Machine Learning Linear regression with one variable Andrew Ng Gradient descent algorithm Andrew Ng Andrew Ng If α is too small, gradient descent can be slow. If α is too large, gradient descent can overshoot the minimum. It may fail to converge, or even diverge. Andrew Ng at local op6ma Current value of Andrew Ng Gradient descent can converge to a local minimum, even with the learning rate α fixed. As we approach a local minimum, gradient descent will automa6cally take smaller steps. So, no need to decrease α over 6me. Andrew Ng Gradient descent for linear regression Machine Learning Linear regression with one variable Andrew Ng Gradient descent algorithm Linear Regression Model Andrew Ng Andrew Ng Gradient descent algorithm update and simultaneously Andrew Ng θ1 θ0 J(θ0,θ1) Andrew Ng θ0 θ1 J(θ0,θ1) Andrew Ng Andrew Ng (for fixed , this is a func6on of x) (func6on of the parameters ) Andrew Ng (for fixed , this is a func6on of x) (func6on of the parameters ) Andrew Ng (for fixed , this is a func6on of x) (func6on of the parame