A Numerical Second Derivative from Three Points

Three Unevenly-Spaced PointsWhat is the second derivative of three unevenly spaced points?  Is there a formula for the numerical second derivative?

In practice, these points could represent measured data that we want to analyze like weather patterns or financial data. We will see that the second derivative is a linear combination of the three points.

First Derivatives Occur Between the PointsWe have three points, so we can find two first derivatives using those points.

Label the x and y coordinates for the three points and use the finite difference formula to calculate the first derivatives.

These derivatives are not at the points; they occur between pairs of points.  For example, the derivative labeled x=1.5 is the derivative somewhere between x1 and x2. The number 1.5 suggests that it is precisely halfway between them, which is what we are assuming.

First Derivatives and Distance in xThe red dots represent the first derivatives at x=1.5 and x=2.5. We can call these D1 and D2.

We use another finite difference formula to calculate the second derivative. The second derivative is the change in the first derivative divided by the distance between the points where they were evaluated. This is the same as “rise over run,” except that we replace the difference in y coordinates (the “rise”) with the difference in the first derivatives. In this example the difference between the first derivatives is D2 – D1. The distance between those derivatives (the “run”) is half of the distance between x1 and x3.


Equation D2-D1

Equation Second Derivative Initial Answer

Now we plug in what we know about the first derivatives.  This is the answer, but let’s take it a step further.

One thing we can do is to write the second derivative as a linear combination of the y coordinates for the three points.  From this point forward we are doing algebra…

Equation Remove the Denominator

Equation Common DenominatorEquation Expand the ProductsEquation Group the Coefficients
Equation Linear Combination of y Values

Now it is easy to see that the second derivative can be expressed as a linear combination of the y values.

Numerical Second Derivative Equation in Matrix Notation

Here is the Numerical Second Derivative Equation in Matrix Notation

Now for some examples

Example 1 – Evenly Spaced Points

Let’s see how it works with evenly spaced points.  The second derivative of y = x2 is always 2, so this function is a good example.  It turns out that the coefficients [1, -2, 1] work for any three points separated by 1 unit in x.

Second Derivative Example 1

Example 2 – Unevenly Spaced Points

This is similar to the previous example, but the points are no longer evenly spaced.  The third point is further away, and we see that the coefficients [2/3, -1, 1/3] place the smallest emphasis on the third point.  That point is further away and for that reason its value could be very different from the values of the other two points. (In this case 25 is much larger than 4 or 9.)  The small magnitude of the third coefficient helps to balance this out.

Second Derivative Example 2

Example 3 – Nonquadratic Function

Now let’s try something different.  What if we use y = x3 instead of y = x2 to generate the points?

Second Derivative Example 3

From calculus we know that the second derivative for y = x3 is 6*x.  At x = 3 this is 18… but the answer we got was 20.  What happened???

The answer is: This is all being done numerically, not analytically. The numerical process does not have a priori knowledge about the function represented in those three points, so it uses the only model it knows how to use with three data points – a quadratic model.

Here’s another way to look at it.  This calculation process is automatically fitting a quadratic function through the three points and reporting the second derivative for that quadratic function.

If we had four points, we could derive a formula that works for four points.  That formula would do the same thing, except that it would fit a cubic function instead of a quadratic function.  We could do this for any number of points, but then the equations get complicated.

Plot y vs x Cubic and Quadratic Functions

If we look at the cubic function and the fitted quadratic function, it is clear that the quadratic function is just an approximation in the neighborhood of x= 2, 3, and 5.  However, it is a fairly good approximation in that neighborhood, so it makes sense that its first and second derivatives are also reasonably good approximations in that neighborhood.  We can see from the x2 coefficient (which is 10) that the quadratic function has a second derivative of 20.  This agrees with the answer we got from the numerical model.

Advertisements

7 comments

  1. Interesting, thanks!
    How could we prove rigorously that the finite difference method gives the exact result for a quadratic function?

    • Hi Lorfus. Thanks for the question. You would have to fit a quadratic polynomial ax^2 + bx+ c to the three points and show that the second derivative from these equations equals 2*a from your fitted model. That would be sufficient to prove it.

      On Monday, October 3, 2016, Math for Mere Mortals < comment-reply@wordpress.com > wrote:

      >

  2. How could we prove
    rigorously that the finite
    difference method gives
    the exact result for a
    quadratic function?

  3. This is brilliant and very informative. I was looking for information regarding taking the derivative of unevenly spaced points (or irregular spaced point) and had a hard time finding information. Could you show how to extend this for the next, higher level derivative?

    • Hi, Chris. Here is the process for calculating higher orders of derivatives numerically. Suppose you have four points of x and y coordinates and you want the *third* derivative. Start by calculating the three first-derivatives between those four points using rise over run. Use that information to create a set of three *new* points. The new points’ x coordinates will be the halfway points between pairs of x coordinates. The new points’ y coordinates should be the derivatives that you calculated (the rise over run). That leaves you with three points whose x coordinates represent x coordinates and whose y coordinates represent first derivatives. Use these three points as if you are taking the second derivative like in the article above. The result will be the third derivative of your four original points.

      This works because your three new points *are* first derivatives. When you apply the formula to calculate the second derivative of them, you are taking a second derivative of a first derivative. That’s the same as taking the third derivative! You can apply this pattern as many times as you like, and each time you will find a higher order of derivative. When you are down to three points, apply the formula. And of course the points can be unevenly spaced in x because the handy-dandy formula accounts for that.

      Here is another way to look at it. In calculus you can take derivatives in multiple steps or all at once and you will always arrive at the same result: d3y/dx3 = d/dx(d/dx(d/dx(y(x)))) What you are doing here is taking a first derivative and then finishing with a second derivative like this: d3y/dx3 = d2/dx2(d/dx(y(x)))

      It is also possible to find a single formula that does all of this, but it would be too tedious and cumbersome to be of use to mere mortals. Working with the data step by step is more manageable, including in computer programs. I hope this helps answer your query.

      On Thu, May 4, 2017 at 9:22 PM Math for Mere Mortals wrote:

      >

      • Thanks Kintaar. I have a set of unevenly spaced points (x,y) in a robot trajectory and I wanted to differentiate the points to find the resulting velocity, acceleration, and jerk. I tried using this approach and the result is only good if my data is uniformly spaced. I also tried using Bengt Fornberg’s formula for generating weights for unevenly spaced data and still the results are ugly. Are there any caveats to be aware of when using this on irregularly spaced data?

      • Hi, Chris. It’s hard to say without seeing the data, but here are some ideas. If the dataset is noisy, differentiating will make the noise that much more apparent. You said that it seems to work when the points are evenly spaced. That is peculiar. Does this only affect the first or second derivative, or does it affect all of them? Have you tried different techniques for differentiating the data to see if they give consistent results? For example, try calculating a second derivative with the 1st, 2nd, and 3rd points. Then do the same with the 1st, 2nd, and 4th points. If you get different answers, the dataset is too coarse (i.e. suffers from quantization noise) or it suffers from other kinds of noise. Are you sure that you moved the x coordinates to recenter each first derivative? Another thing you can try is regularizing the dataset. That way you can smooth the dataset you have or generate a new dataset that is equivalent but has evenly spaced points.

        On Sun, May 7, 2017 at 7:51 PM, Math for Mere Mortals wrote:

        >


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s