Introduction to Regularizing with 2D Data – Part 3 of 3

In Part 2 of Introduction to Regularizing with 2D Data we set up an overdetermined system of 13 equations with 10 variables and showed how to find the best solution using the normal equations. In this part we will look at the solution to the example and calculate the residual errors for this. This and several other examples of higher resolution (higher numbers of output points) are demonstrated in an Excel spreadsheet.

Introduction to Regularizing with 2D Data – Part 2 of 3

In Part 1 of Introduction to Regularizing with 2D Data we set up a system of fidelity equations to map five input points to ten output points. In this part we will add second derivative equations and impose a scaling factor to ensure consistent results.

Introduction to Regularizing with 2D Data – Part 1 of 3

Introduction to Regularizing with 2D Data

Start with some noisy measured data.  You want to subsample the data or change the point intervals.  You could fit a polynomial to what you measured, but the noise interferes with the fit.

A better method is to regularize the data.  This means to construct a numerical dataset similar to what you measured but with better resolution, custom spacing in x, and smoothing in y.

This series of articles shows how to regularize 2D data with a simple and elegant process.

Bicubic Interpolation Excel Spreadsheet Function

Excel Spreadsheet Function for Bicubic Interpolation

Suppose you have a grid of points, but you need to know the values somewhere between the points.  Bicubic interpolation is a great way to do this.  This spreadsheet with VBA source code gives a ready-to-use worksheet function that interpolates between any data laid out in a grid using bicubic Lagrange interpolation.

This example shows how to use the custom Excel function BicubicInterpolation() to calculate a value on a volatility surface, something commonly used in financial math for pricing derivatives.

A Numerical Second Derivative from Three Points

What is the second derivative of three unevenly spaced points?  Is there a formula for the numerical second derivative?

In practice, these points could represent measured data that we want to analyze like weather patterns or financial data. We will see that the second derivative is a linear combination of the three points.