Solving simple derviatve numerically from a dataset with two columns x and y

[Charles Smith]

Hello all, its been ages since I took calc. and well I want to make sure I get this right for a lil side research project of mine. If I had a dataset with two columns x and y and I needed to solve dx/dy it would look something like this for numerically solving it x2 - x1 / y2-y1 (where the 1 and 2 denote location in my dataset, e.g. maybe x is time and x2 is 2 minutes have passed and x1 is 1 minute has passed) very simple question I know but going to build off of it. Thank you.

 

[khansaheb]

Hello all, its been ages since I took calc. and well I want to make sure I get this right for a lil side research project of mine. If I had a dataset with two columns x and y and I needed to solve dx/dy it would look something like this for numerically solving it x2 - x1 / y2-y1 (where the 1 and 2 denote location in my dataset, e.g. maybe x is time and x2 is 2 minutes have passed and x1 is 1 minute has passed) very simple question I know but going to build off of it. Thank you.

That is the "simplest" approach to "estimate" first derivative. For a quick refresher, look at:

Numerical Differentiation -- from Wolfram MathWorld

Numerical differentiation is the process of finding the numerical value of a derivative of a given function at a given point. In general, numerical differentiation is more difficult than numerical integration. This is because while numerical integration requires only good continuity properties...

mathworld.wolfram.com

 

[blamocur]

numerically solving it x2 - x1 / y2-y1

I am sure you mean [imath](x_2-x_1)/(y_2-y_1)[/imath]. I know I am nitpicking, but in my software-writing experience one can spend hours trying to catch a typo like this in source code.

 

[Charles Smith]

I am sure you mean [imath](x_2-x_1)/(y_2-y_1)[/imath]. I know I am nitpicking, but in my software-writing experience one can spend hours trying to catch a typo like this in source code.

Thank you great catch, and besides for the ( ) you are indicating I am correct right?

 

[Dr.Peterson]

Thank you great catch, and besides for the ( ) you are indicating I am correct right?

I would want to suspend judgment until you showed more of what you plan to do; as it is, you have a very rough approximation to dx/dy, which may or may not be adequate.

Also, it seems odd that you would want dx/dy rather than dy/dx. There's nothing wrong with the former in principle, but it's possible you might really want the other -- again, depending on details you haven't shown.

 

[Charles Smith]

And if we were to take this one step farther dx/d(ln(y)) it would be (x2−x1) / (1/ (y2−y1)). or (x2−x1) * (y2−y1) . Thanks in advance for the patience with the easy questions Right now I just need a lil confidence builder.

 

[Charles Smith]

I would want to suspend judgment until you showed more of what you plan to do; as it is, you have a very rough approximation to dx/dy, which may or may not be adequate.

Also, it seems odd that you would want dx/dy rather than dy/dx. There's nothing wrong with the former in principle, but it's possible you might really want the other -- again, depending on details you haven't shown.

Your right dy/dx is what I meant.

To answer your other question my array is column 1 =velocity and column 2 = time. So numerically
acceleration = (velocity[i + 1] - velocity)/time[i+1] - time

 

[khansaheb]

Your right dy/dx is what I meant.

To answer your other question my array is column 1 =velocity and column 2 = time. So numerically
acceleration = (velocity[i + 1] - velocity)/(time[i+1] - time)

Small correction (yet very important) indicated above

 

[Dr.Peterson]

Your right dy/dx is what I meant.​

To answer your other question my array is column 1 =velocity and column 2 = time. So numerically
acceleration = (velocity[i + 1] - velocity)/time[i+1] - time

If you mean

acceleration = (velocity[ i + 1 ] - velocity[ i ])/(time[ i+1 ] - time[ i ]),​

then, yes, that's a reasonable approximation of dv/dt if the time intervals are reasonably short.

(I observe that "bracket i bracket" does some funny stuff involving italics, which I had to work around.)

And if we were to take this one step farther dx/d(ln(y)) it would be (x2−x1) / (1/ (y2−y1)). or (x2−x1) * (y2−y1) . Thanks in advance for the patience with the easy questions Right now I just need a lil confidence builder.

Not quite. Can you explain your thinking?

I also don't know why you want to do that.

 

[ashley2]

dx/dy ≈ (x2 - x1) / (y2 - y1)

Where:

x1 and x2 are the values of the x variable at two specific data points.
y1 and y2 are the corresponding values of the y variable at those same data points.
This formula gives you the average rate of change of x with respect to y between the two data points. Keep in mind that this is an approximation and assumes a constant rate of change within that interval. If the relationship between x and y is more complex, you may need to use more advanced techniques to estimate the derivative, like curve fitting or regression analysis, depending on your specific research project.

 

[Charles Smith]

If you mean

acceleration = (velocity[ i + 1 ] - velocity[ i ])/(time[ i+1 ] - time[ i ]),​

then, yes, that's a reasonable approximation of dv/dt if the time intervals are reasonably short.

(I observe that "bracket i bracket" does some funny stuff involving italics, which I had to work around.)


Not quite. Can you explain your thinking?

I also don't know why you want to do that.

If you mean

acceleration = (velocity[ i + 1 ] - velocity[ i ])/(time[ i+1 ] - time[ i ]),​

then, yes, that's a reasonable approximation of dv/dt if the time intervals are reasonably short.

(I observe that "bracket i bracket" does some funny stuff involving italics, which I had to work around.)


Not quite. Can you explain your thinking?

I also don't know why you want to do that.

dx/dy ≈ (x2 - x1) / (y2 - y1)

Where:

x1 and x2 are the values of the x variable at two specific data points.
y1 and y2 are the corresponding values of the y variable at those same data points.
This formula gives you the average rate of change of x with respect to y between the two data points. Keep in mind that this is an approximation and assumes a constant rate of change within that interval. If the relationship between x and y is more complex, you may need to use more advanced techniques to estimate the derivative, like curve fitting or regression analysis, depending on your specific research project.

Thank

dx/dy ≈ (x2 - x1) / (y2 - y1)

Where:

x1 and x2 are the values of the x variable at two specific data points.
y1 and y2 are the corresponding values of the y variable at those same data points.
This formula gives you the average rate of change of x with respect to y between the two data points. Keep in mind that this is an approximation and assumes a constant rate of change within that interval. If the relationship between x and y is more complex, you may need to use more advanced techniques to estimate the derivative, like curve fitting or regression analysis, depending on your specific research project.

Thank you Ashley2, I am aware it is a approximation, if one had a very high resoltion dataset then it would be an acceptable method.

Also Looking where I went wrong here from the reply with Dr. Peterson

And if we were to take this one step farther dx/d(ln(y)) it would be (x2−x1) / (1/ (y2−y1)). or (x2−x1) * (y2−y1) . Thanks in advance for the patience with the easy questions Right now I just need a lil confidence builder.

My reasoning is that the d/dx (ln x) = 1/x, which is based on the ln derivative rule so where did I go wrong in the above statement? Again I am right now just reviewing and getting back into this been a long time since I used calc.

 

[Dr.Peterson]

And if we were to take this one step farther dx/d(ln(y)) it would be (x2−x1) / (1/ (y2−y1)). or (x2−x1) * (y2−y1) . Thanks in advance for the patience with the easy questions Right now I just need a lil confidence builder.

My reasoning is that the d/dx (ln x) = 1/x, which is based on the ln derivative rule so where did I go wrong in the above statement? Again I am right now just reviewing and getting back into this been a long time since I used calc.

First, can you answer my question about why you want to find [imath]\dfrac{dx}{d(\ln(y))}[/imath]? I want to make sure you really meant what you said.

Then, can you show how you used [imath]\dfrac{d}{dx}(\ln(x))=\dfrac{1}{x}[/imath] to arrive at what you wrote? Writing out your thinking often helps you to discover your own error. (I do that all the time.)

Your result is similar to the truth, but different enough that it's actually very wrong.

Hint: [imath]\dfrac{d}{dx}(\ln(x))=\dfrac{1}{x}[/imath], not [imath]\dfrac{1}{x_2-x_1}[/imath].

Another thing: You said,

Your right dy/dx is what I meant.

But here you're back to using things like dx/dy. Which do you really want? Or does it not really matter because you're just trying things out?

 

[Charles Smith]

Dr. Peteresen you are correct and I thought I did answer this, my mistake, I want dy/d(ln(x)) not dx/d(ln(y)) it was a typo lets move on to the real question now.

Here is my thought process behind it, please if you could aid in provding a solution or point me to where I am wrong that would be very helpful.

If a decent numerical approximation of dy/dx ≈ (y2 - y1) / (x2 - x1) (Again I understand this is a numerical approximation) or to make it a little more concrete in my mind. If we wanted to do it for velocity and time or dv/dt it would be the following

acceleration = (velocity[ i + 1 ] - velocity[ i ])/(time[ i+1 ] - time[ i ]),

Now all we want to do is go one step farther with dv/d(ln(t)) (please lets just focus on the math at this point in time)

Then I figured to solve this numerically we would use the ln rule which is the following the derivative of ln x is 1/x.

Therefore I came to the conclusion that maybe

(velocity[ i + 1 ] - velocity[ i ]) / 1 / (time[ i+1 ] - time[ i ])

However, you have indicated that this is incorrect. It would be helpful if you could provide a solution that would help to understand where i am going wrong. Thank you

 

[Dr.Peterson]

Now all we want to do is go one step farther with dv/d(ln(t)) (please lets just focus on the math at this point in time)

Then I figured to solve this numerically we would use the ln rule which is the following the derivative of ln x is 1/x.

Therefore I came to the conclusion that maybe

(velocity[ i + 1 ] - velocity[ i ]) / 1 / (time[ i+1 ] - time[ i ])

However, you have indicated that this is incorrect. It would be helpful if you could provide a solution that would help to understand where i am going wrong. Thank you

Please take it in smaller steps. HOW did you get from dv/d(ln(t)) to (velocity[ i + 1 ] - velocity[ i ]) / 1 / (time[ i+1 ] - time[ i ])?

Taking big leaps is the best way to make mistakes. The way to do math is to take one step after another, each of which you can explain.

 

[Charles Smith]

I think the use of the Leibniz notation is confusing me. When I go back and look at I am not sure how to best proceed. I am looking for a lil more explanation or help on this. I would really appreciate a lil more 'hand holding' on this one cheers

 

[Dr.Peterson]

I think the use of the Leibniz notation is confusing me. When I go back and look at I am not sure how to best proceed. I am looking for a lil more explanation or help on this. I would really appreciate a lil more 'hand holding' on this one cheers

Now all we want to do is go one step farther with dv/d(ln(t)) (please lets just focus on the math at this point in time)

Then I figured to solve this numerically we would use the ln rule which is the following the derivative of ln x is 1/x.

Thanks for trying. That (and so, discovering what you don't know) is an essential part of getting things right.

If (for some not yet clear reason) I wanted to find dv/d(ln(t)), I would start by defining an intermediate variable u = ln(t), so that what I am trying to find is dv/du.

Now the chain rule says that dv/du = (dv/dt) / (du/dt) = (dv/dt) / (1/t) = t dv/dt.

Then, using your approximation, this becomes (v₂ - v₁)/(t₂ - t₁) * t₁.

Do you see why I said the answer is similar to your guess, but not at all the same? You said

(velocity[ i + 1 ] - velocity[ i ]) / 1 / (time[ i+1 ] - time[ i ])

which in my notation would be (v₂ - v₁) / (1/(t₂ - t₁)) = (v₂ - v₁) * (t₂ - t₁) .

 

[blamocur]

Now all we want to do is go one step farther with dv/d(ln(t)) (please lets just focus on the math at this point in time)

We try to focus on math, but we do not provide answers. Instead we try to help you to get your answers. For this we need A) a clear statement of the problem, and B) a good understanding of what you've done so we can see what kind of help you might need.

Therefore I came to the conclusion that maybe

(velocity[ i + 1 ] - velocity[ i ]) / 1 / (time[ i+1 ] - time[ i ])

I don't understand what you mean by this formula. Is this your approximation for [imath]\frac{dv}{d(\ln t)}[/imath] ? If it is then it looks wrong. Moreover, (velocity[ i + 1 ] - velocity[ i ]) / 1 / (time[ i+1 ] - time[ i ]) = (velocity[ i + 1 ] - velocity[ i ]) / (time[ i+1 ] - time[ i ]) because for arbitrary x,y in the absence of parentheses you always have x/1/y = x/y.

Finally, if you need an approximation for [imath]\frac{dv}{d(\ln t)}[/imath] why not use [imath]\frac{v_2 - v_1}{\ln t_2 - \ln t_1}[/imath] ?

 

[Charles Smith]

Thanks for trying. That (and so, discovering what you don't know) is an essential part of getting things right.

If (for some not yet clear reason) I wanted to find dv/d(ln(t)), I would start by defining an intermediate variable u = ln(t), so that what I am trying to find is dv/du.

Now the chain rule says that dv/du = (dv/dt) / (du/dt) = (dv/dt) / (1/t) = t dv/dt.

Then, using your approximation, this becomes (v₂ - v₁)/(t₂ - t₁) * t₁.

Do you see why I said the answer is similar to your guess, but not at all the same? You said

which in my notation would be (v₂ - v₁) / (1/(t₂ - t₁)) = (v₂ - v₁) * (t₂ - t₁) .

After looking it over and thinking about it your answer makes sense. I appreciate it as I did not have much time to focus on this.