Machine Learning

# Introduction to Linear Regression Model (LRM)

Today we will learn about Linear regression models:

1. Introduction
2. Equations
3. Example

Graphical representation of Linear model

Introduction

In statistics, linear regression is an approach for modeling the relationship between  dependent variable  and
independent variable. The case of one independent variable is called simple linear regression. For more than one independent variable, the process is called multiple linear regression.

In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data. Such models are called linear models.

Linear regression was the first type of regression analysis to be studied rigorously, and to be used extensively in practical applications. This is because models which depend linearly on their unknown parameters are easier to fit than models which are non-linearly related to their parameters and because the statistical properties of the resulting estimators are easier to determine.

Linear regression has many practical uses. Most applications fall into one of the following two broad categories:

• Linear regression is used when to predict things, forecasting, or error reduction in every dataset to observed the values of X and Y.
• Given a variable y and a number of variables X1, …, Xp that may be related to y, linear regression analysis can be applied to quantify the strength of the relationship between y and the Xj, to assess which Xj may have no relationship with y at all, and to identify which subsets of the Xj contain redundant information about y.

Linear regression models are often fitted using the least squares approach. Conversely, the least squares approach can be used to fit models that are not linear models. Thus, although the terms least squares and linear model are closely linked, they are not synonymous.

Equations

Lets assume simple linear model, in simple linear model there is only one independent variable (X) and one dependent variable (Y).

Equation for Y on X

y= n∈+βx      →1

Equation for X on Y

x= n∈+βy      →2

Using equation 1 and 2 we will calculate the value of ∈ and β

taking summation of these equations

∑y= n∑∈+β∑x    →3

∑x= n∑∈+β∑y    →4

on multiplying eq (3) by x and eq (4) by y

∑xy= ∈∑x+β∑x²   →5

∑xy= ∈∑y+β∑y²   →6

By using these equations we will find the values of coefficient(β) and error(∈)

Example

 x x^2 y y^2 xy 69 4761 70 4900 4830 63 3969 65 4225 4095 66 4356 68 4624 4488 64 4096 65 4225 4160 67 4489 69 4761 4623 64 4096 66 4356 4224 70 4900 68 4624 4760 66 4356 65 4225 4290 68 4624 71 5041 4828 67 4489 67 4489 4489 65 4225 64 4096 4160 71 5041 72 5184 5112 total 800 53402 810 54750 54059

800 = 12∈+810β     →1

810 = 12∈+800β     →2

54059 = 810∈+54750β   →3

54059 = 800∈ +53402β     →4

On solving these equations we will get the values of

∈= ?

β= ?

After getting the values we will compare the F-value, and p-value, According to these comparisions  we will decide the relationship importance between the variables [which variable is dependent on which variable and how it will effect the model]

Note: In next article I will teach you how to make a linear regression model in R

Categories: Machine Learning

Tagged as: