Extrapolation methods

If you have ever worked on a Computer Vision project, you might know that using augmentations to diversify the dataset is the best practice. However, some augmentations are not straightforward and built upon various mathematical concepts, for example, extrapolation. On this page, we will:

• Сover the general extrapolation concept;

• Draw a separating line between interpolation and extrapolation;

• And check out such extrapolation techniques as Constant, Replicate, Reflect, Wrap, and Reflect 101.

Let’s jump in.

Extrapolation method explained

In maths, extrapolation is the process of estimating an unknown value that exists outside a sequence of data points. In other words, it is about drawing some conclusion based on known facts.

In the Computer Vision field in general and image processing in particular, such a definition is also applicable. Let’s break the extrapolation concept down and draw a simple example.

Imagine having a painting that nicely fits in a stunning gilded frame. At some point, you might get bored of looking at a painting from one perspective only and want to rotate it a bit without moving the frame. In such a case, some parts of the frame will get blank. This is where extrapolation comes into the mix.

To define the term, in image processing, extrapolation is a technique that dictates how to fill the empty space left in the “frame” after the initial image got downscaled, rotated, or shifted.

Extrapolation method Vs. Interpolation method

The difference between interpolation and extrapolation in image processing is the following:

• Interpolation is used when we resize the image, which results in the image size changes. In Hasty, it is applied for Random sized crop, Resize, Shift Scale Rotate, Longest max size, and Smallest max size augmentations.
• Extrapolation is used when we rotate, downscale, or shift the image without changing its size. As a result of such transformations, some image parts are left empty, and extrapolation helps fill them in. In Hasty, it is applied for Shift Scale Rotate augmentation.

To draw an analogy with paintings, when you use interpolation, you change the painting’s size together with the frame, so there are no blank parts left between the image and the frame.

In comparison, you might rotate, downscale, or transform your painting without changing the frame. After that, some blank spaces might appear, and that’s when you use extrapolation to fill them in.

Both interpolation and extrapolation propose many various techniques, so let’s get straight into business and cover extrapolation methods one by one.

Constant extrapolation explained

The easiest way to fill in all the blank spaces is to assign a specific pixel value to each empty pixel. This is precisely how the Constant extrapolation works. When selected, it simply fills all the empty spaces with black color by default.

Additionally, Hasty allows you to modify the fill color using two parameters:

• Padding value if border mode is Constant;

• And Padding value if border mode is Constant applied to masks.

If you put some value into these parameters, the empty space will be filled by the given padding value in the red palette instead of filling it with black.

Replicate extrapolation explained

Another practical yet simple extrapolation technique is Replicate. It is logical to fill in a blank pixel using the value of the nearest border pixel. This is precisely how the Replicate option works.

Reflect extrapolation explained

Reflect extrapolation flips the original image horizontally and vertically until all the empty spaces are filled.

Wrap extrapolation explained

The Wrap extrapolation technique is about copy-pasting the original image until all the empty spaces are filled. As a result, the augmented image will feature most of the parts of the original picture, even if it has been massively shifted, scaled, or rotated.

Reflect 101 extrapolation explained

Reflect 101 is similar to Reflect. It also flips the original image horizontally and vertically until all the empty spaces are filled. The difference, however, is that Reflect 101 cuts a bit from each side of the picture. As a result, Reflect 101 flips just a part of the original image.