Bilinear filtering

File:Image-resample-sample.png

A zoomed small portion of a bitmap, using nearest-neighbor filtering (left), bilinear filtering (center), and bicubic filtering (right).

Bilinear filtering is a texture filtering method used to smooth textures when displayed larger or smaller than they actually are.

Most of the time, when drawing a textured shape on the screen, the texture is not displayed exactly as it is stored, without any distortion. Because of this, most pixels will end up needing to use a point on the texture that is "between" texels, assuming the texels are points (as opposed to, say, squares) in the middle (or on the upper left corner, or anywhere else; it does not matter, as long as it is consistent) of their respective "cells". Bilinear filtering uses these points to perform bilinear interpolation between the four texels nearest to the point that the pixel represents (in the middle or upper left of the pixel, usually).

The formula

In a mathematical context, bilinear interpolation is the problem of finding a function f(x,y) of the form

$f(x,y) = c_{11} xy + c_{10} x + c_{01} y + c_{00}$

satisfying

$\begin{array}{lcl} f(x_1,y_1) = z_{11} \\ f(x_1,y_2) = z_{12} \\ f(x_2,y_1) = z_{21} \\ f(x_2,y_2) = z_{22} \\ \end{array}$

The usual, and usually computationally least expensive way to compute $f$ is through linear interpolation used twice, for example to compute two functions $f_1$ and $f_2$ satisfying

$\begin{array}{lcl} f_1(y_1) = z_{11} \\ f_1(y_2) = z_{12} \\ f_2(y_1) = z_{21} \\ f_2(y_2) = z_{22} \\ \end{array}$

and then to combine these functions (which are linear in $y$) into one function $f$ satisfying

$\begin{array}{lcl} f(x_1,y) = f_1(y) \\ f(x_2,y) = f_2(y) \\ \end{array}$

In computer graphics, bilinear filtering is usually performed on a texture during texture mapping, or on a bitmap during resizing. In both cases, the source data (bitmap or texture) can be seen as a two-dimensional array of values $z_{ij}$, or several (usually three) of these in the case of full-color data. The data points used in bilinear filtering are the 2x2 points surrounding the location for which the color is to be interpolated.

Additionally, one does not have to compute the actual coefficients of the function $f$; computing the value $f(x,y)$ is sufficient.

The largest integer not larger than x shall be called $[x]$, and the fractional part of $x$ shall be $\{x\}$. Then, $x = [x] + \{x\}$, and $\{x\} < 1$. We have $x_1 = [x]$, $x_2 = [x] + 1$, $y_1 = [y]$, $y_2 = [y] + 1$. The data points used for interpolation are taken from the texture / bitmap and assigned to $z_{11}$, $z_{12}$, $z_{21}$, and $z_{22}$.

$f_{1}(y_{1}) = z_{11}$, $f_{1}(y_{2}) = z_{12}$ are the two data points for $f_{1}$ subtracting the former from the latter yields

$f_{1}(y_{2}) - f_{1}(y_{1}) = z_{12} - z_{11}$

Because $f_{1}$ is linear, its derivative is constant and equal to

$(z_{12} - z_{11}) / (y_{2} - y_{1}) = z_{12} - z_{11}$

Because $f_{1}(y_{1}) = z_{11}$,

$f_{1}(y_{1} + \{y\}) = z_{11} + \{y\}(z_{12} - z_{11})$

and similarly,

$f_{2}(y_{1} + \{y\}) = z_{21} + \{y\}(z_{22} - z_{21})$

Because $y_{1} + \{y\} = y$, we have computed the endpoints $f_{1}(y)$ and $f_{2}(y)$ needed for the second interpolation step.

The second step is to compute $f(x,y)$, which can be accomplished by the very formula we used for computing the intermediate values:

$f(x,y) = f_{1}(y) + \{x\}(f_{2}(y) - f_{1}(y))$

In the case of scaling, y remains constant within the same line of the rescaled image, and storing the intermediate results and reusing them for calculation of the next pixel can lead to significant savings. Similar savings can be achieved with all "bi" kinds of filtering, i.e. those which can be expressed as two passes of one-dimensional filtering.

In the case of texture mapping, a constant x or y is rarely if ever encountered, and because today's (2000+) graphics hardware is highly parallelized, there would be no time savings anyway.

Another way of writing the bilinear interpolation formula is

$f(x,y) = (1-\{x\})((1-\{y\})z_{11} + \{y\}z_{12}) + \{x\}((1-\{y\})z_{21} + \{y\}z_{22})$

Sample code

This code assumes that the texture is square (an extremely common occurrence), that no mipmapping comes into play, and that there is only one channel of data (not so common. Nearly all textures are in color so they have red, green, and blue channels, and many have an alpha transparency channel, so we must make three or four calculations of y, one for each channel). The location of UV-coordinates is at center of texel. For example, {(0.25,0.25), (0.75,0.25), (0.25,0.75), (0.75,0.75)} are values for 2x2 texture.

 double getBilinearFilteredPixelColor(Texture tex, double u, double v) {
   u = u * tex.size - 0.5;
   v = v * tex.size - 0.5;
   int x = floor(u);
   int y = floor(v);
   double u_ratio = u - x;
   double v_ratio = v - y;
   double u_opposite = 1 - u_ratio;
   double v_opposite = 1 - v_ratio;
   double result = (tex[x][y]   * u_opposite  + tex[x+1][y]   * u_ratio) * v_opposite + 
                   (tex[x][y+1] * u_opposite  + tex[x+1][y+1] * u_ratio) * v_ratio;
   return result;
 }

Limitations

Bilinear filtering is rather accurate until the scaling of the texture gets below half or above double the original size of the texture - that is, if the texture was 256 pixels in each direction, scaling it to below 128 or above 512 pixels can make the texture look bad, because of missing pixels or too much smoothness. Often, mipmapping is used to provide a scaled-down version of the texture for better performance; however, the transition between two differently-sized mipmaps on a texture in perspective using bilinear filtering can be very abrupt. Trilinear filtering, though somewhat more complex, can make this transition smooth throughout.

For a quick demonstration of how a texel can be missing from a filtered texture, here's a list of numbers representing the centers of boxes from an 8-texel-wide texture (in red and black), intermingled with the numbers from the centers of boxes from a 3-texel-wide down-sampled texture (in blue). The red numbers represent texels that would not be used in calculating the 3-texel texture at all.

0.0625, 0.1667, 0.1875, 0.3125, 0.4375, 0.5000, 0.5625, 0.6875, 0.8125, 0.8333, 0.9375

Special cases

Textures aren't infinite, in general, and sometimes one ends up with a pixel coordinate that lies outside the grid of texel coordinates. There are a few ways to handle this:

• Wrap the texture, so that the last texel in a row also comes right before the first, and the last texel in a column also comes right above the first. This works best when the texture is being tiled.
• Make the area outside the texture all one color. This may be of use for a texture designed to be laid over a solid background or to be transparent.
• Repeat the edge texels out to infinity. This works best if the texture is not designed to be repeated.

See also

• Anisotropic filtering
• Bilinear interpolation
• Trilinear filtering
• Lanczos resampling