Skip to main content

Linear Regression, Maths behind it and Best Reason for using least squares

 Linear Regression

Linear Regression is quite a way that will attempt to fit a line through observed variables (x,y). If we look into the graphs it will try to find the line which is more touching to the point (x^i,y^i) for all I in the dataset, y as a linear function of x and it will seem like

Image for post

As it will try to prophesy a line most possible it will do by the least-squares method which is going to decrease the squares of the distance from the line for all the points.

Catholic rule of Least Squares

As the method will try to find the line Y = MX+ b and for solving it by math just find the distance from the line by putting the point in the equation and all the distances and just minimize it by taking derivative and assigning it to the zero. The general equation looks like

Image for post

and to view it on the graph it will look like

Image for post

Why we use least squares

One purpose is that the equations required in solving for the best-fit line are easy to solve. The only satisfying reason for using it will be proved by the gauss theorem which states

Suppose y=f(x). We measure values x^i,y^i and compute errors y^i-f(x^i). If these errors are independent and normally distributed, then consider, for any possible linear function f the probability P^f​ of getting the measurements (x^i​,y^i​) if y=f(x) were the correct model. The least-squares line is the line for which P^f​ is maximized.

This is the only best reason why we used it over the other methods.

Future Readings

  • Björck, Å. (1996). Numerical Methods for Least Squares Problems. SIAM.
  • Kariya, T.; Kurata, H. (2004). Generalized Least Squares. Hoboken

I hope this helps you in understanding you, linear regression easy way.

Labels:

Comments

Post a Comment

Popular posts from this blog

NEW TREND OF DATA SCIENCE: REINFORCEMENT LEARNING

Reinforcement Learning (RL) is a machine learning method that empowers a specialist to learn in an intuitive environment by performing trial and error utilizing observations from its very own activities and encounters. In spite of the fact that both direct and reinforcement learning use mapping among input and output, not at all like supervised learning where input gave to the specialist is basically the right set of activities for playing out a task, reinforcement learning utilizes prizes and discipline as signs for positive and negative conduct. When compared with unsupervised learning, reinforcement learning is distinctive as far as objectives are taken into consideration. While the objective in unsupervised learning is to discover synonymities and contrasts between data points, in reinforcement learning the objective is to locate a reasonable activity model that would boost the aggregate total reward of the specialist. Reinforcement learning will be a huge thing in Data science in ...
Post a Comment
Read more