Linear Regression?

topsquark · #1 $Old$ June 25th, 2009, 04:08 PM

So I'm working on this Physics problem, which (alas!) turns into an unsolvable Math problem. I have the equation (as a function of t):
$e^{at} = a^{\prime} t + b^{\prime};~~0 \leq t \leq T$
where a is presumed known.

The question boils down to how to find reasonable values for a' and b' that best model the above equation. (I can't simply do a Taylor Expansion on the left hand side because T could be large.) Now, being the Physicist I am I would know how find an estimate if the equation were
$e^{at} = b^{\prime};~~0 \leq t \leq T$
The "best fit" in this case would be to find the average of the exponential function as t goes from 0 to T.

Is there a way to think of this equation as a "best fit" problem by using the exponential function as if it were "data?"

-Dan

CaptainBlack · #2 $Old$ June 26th, 2009, 01:09 AM

Quote:

Originally Posted by topsquark $View Post$

So I'm working on this Physics problem, which (alas!) turns into an unsolvable Math problem. I have the equation (as a function of t):
$e^{at} = a^{\prime} t + b^{\prime};~~0 \leq t \leq T$
where a is presumed known.

The question boils down to how to find reasonable values for a' and b' that best model the above equation. (I can't simply do a Taylor Expansion on the left hand side because T could be large.) Now, being the Physicist I am I would know how find an estimate if the equation were
$e^{at} = b^{\prime};~~0 \leq t \leq T$
The "best fit" in this case would be to find the average of the exponential function as t goes from 0 to T.

Is there a way to think of this equation as a "best fit" problem by using the exponential function as if it were "data?"

-Dan

There are at least two answers to this (assuming a is a known numeric value)

1. Set up a model in Excel and use the solver to minimise:

$\int_{t=0}^T [e^{at}-(a't+b')]^2 dt$

of course this will be discretised so you will have to find $(a', b')$ that minimises:

$\sum_{t_i} [e^{at_i}-(a't_i+b')]^2$

2. Expand $e^{at}$ in orthogonal polynomials on $(0,T)$ stopping at the linear term.

CB

CaptainBlack · #3 $Old$ June 27th, 2009, 03:00 AM

Quote:

Originally Posted by CaptainBlack $View Post$

There are at least two answers to this (assuming a is a known numeric value)

1. Set up a model in Excel and use the solver to minimise:

$\int_{t=0}^T [e^{at}-(a't+b')]^2 dt$

of course this will be discretised so you will have to find $(a', b')$ that minimises:

$\sum_{t_i} [e^{at_i}-(a't_i+b')]^2$

2. Expand $e^{at}$ in orthogonal polynomials on $(0,T)$ stopping at the linear term.

CB

I have run method 2 through Maxima and the results are shown in the attachment (I ought to condense this down to a function definition since this is at least the second time I have done this).

(this is just using Gram-Schmidt to find the first few orthogonormal basis polynomials then using the first couple of terms of the generalised Fourier series corresponding to the orthonormal basis - note I am using the inner product of real functions not that for complex)

CB

CaptainBlack · #4 $Old$ June 27th, 2009, 04:01 AM

Quote:

Originally Posted by CaptainBlack $View Post$

I have run method 2 through Maxima and the results are shown in the attachment (I ought to condense this down to a function definition since this is at least the second time I have done this).

(this is just using Gram-Schmidt to find the first few orthogonormal basis polynomials then using the first couple of terms of the generalised Fourier series corresponding to the orthonormal basis - note I am using the inner product of real functions not that for complex)

CB

IIRC there is also a trick that will allow the minimisation of the square relative error by defining the weight to be one over the square of the function you are trying to fit.

CB

Sampras · #5 $Old$ June 27th, 2009, 10:12 AM

Wouldn't it depend whether $a <0$ or $a \geq 0$ ?

topsquark · #6 $Old$ June 30th, 2009, 04:12 PM

Quote:

Originally Posted by Sampras $View Post$

Wouldn't it depend whether $a <0$ or $a \geq 0$ ?

This is from a rocket motion equation. a will be positive for any case I'm considering.

-Dan