Machine Learning

Introduction to Linear Regression Model (LRM)

Today we will learn about Linear regression models:

  1. Introduction
  2. Equations
  3. Example

Graphical representation of Linear model



In statistics, linear regression is an approach for modeling the relationship between  dependent variable  and
independent variable. The case of one independent variable is called simple linear regression. For more than one independent variable, the process is called multiple linear regression.

In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data. Such models are called linear models. 

Linear regression was the first type of regression analysis to be studied rigorously, and to be used extensively in practical applications. This is because models which depend linearly on their unknown parameters are easier to fit than models which are non-linearly related to their parameters and because the statistical properties of the resulting estimators are easier to determine.

Linear regression has many practical uses. Most applications fall into one of the following two broad categories:

  • Linear regression is used when to predict things, forecasting, or error reduction in every dataset to observed the values of X and Y.
  • Given a variable y and a number of variables X1, …, Xp that may be related to y, linear regression analysis can be applied to quantify the strength of the relationship between y and the Xj, to assess which Xj may have no relationship with y at all, and to identify which subsets of the Xj contain redundant information about y.

Linear regression models are often fitted using the least squares approach. Conversely, the least squares approach can be used to fit models that are not linear models. Thus, although the terms least squares and linear model are closely linked, they are not synonymous.


Lets assume simple linear model, in simple linear model there is only one independent variable (X) and one dependent variable (Y).

Equation for Y on X

y= n∈+βx      →1

Equation for X on Y

x= n∈+βy      →2

Using equation 1 and 2 we will calculate the value of ∈ and β

taking summation of these equations

∑y= n∑∈+β∑x    →3

∑x= n∑∈+β∑y    →4

on multiplying eq (3) by x and eq (4) by y

∑xy= ∈∑x+β∑x²   →5

∑xy= ∈∑y+β∑y²   →6

By using these equations we will find the values of coefficient(β) and error(∈)


x x^2 y y^2 xy
69 4761 70 4900 4830
63 3969 65 4225 4095
66 4356 68 4624 4488
64 4096 65 4225 4160
67 4489 69 4761 4623
64 4096 66 4356 4224
70 4900 68 4624 4760
66 4356 65 4225 4290
68 4624 71 5041 4828
67 4489 67 4489 4489
65 4225 64 4096 4160
71 5041 72 5184 5112
total 800 53402 810 54750 54059


800 = 12∈+810β     →1

810 = 12∈+800β     →2

54059 = 810∈+54750β   →3

54059 = 800∈ +53402β     →4

On solving these equations we will get the values of

∈= ?

β= ?

After getting the values we will compare the F-value, and p-value, According to these comparisions  we will decide the relationship importance between the variables [which variable is dependent on which variable and how it will effect the model]

Note: In next article I will teach you how to make a linear regression model in R


Categories: Machine Learning

Tagged as:

3 replies »

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s