Interpolate Values From Gridded Dataset Values
This converter efficiently interpolates values from a gridded dataset's values.
As such, it is particularly useful for researchers working with animal track data
(e.g., the
IOOS Animal Telemetry Network).
This converter takes in a table with latitude, longitude, and time columns
(and perhaps other columns) and
returns a table with additional columns with interpolated values based on
the user-specified Request CSV with datasetID/variable/algorithm/nearby
combinations (for example,
jplMURSST41/analysed_sst/Bilinear/4 ).
In all cases, for each latitude, longitude, and time point that you provide, the converter
finds the specified number of nearby values from the specified datasetID/variable,
then applies the specified algorithm to create the interpolated value.
Few people will routinely use this web page. Instead, most users will write and/or use
a script which uses this web page's underlying web service.
Notes:
- Always check the results from the converters.
If you think that the converter gave an incorrect result,
please email the details to bob dot simons at noaa dot gov .
- DISCLAIMER OF LIABILITY
No person or organization associated with this website
makes any warranty, express or implied, including warranties of merchantability
and fitness for a particular purpose, or assumes any legal liability for the accuracy,
completeness, or usefulness, of any information at this website.
- If a specified datasetID and/or variable isn't currently available,
or if the algorithm or nearby value is invalid,
this returns an error message.
- The algorithm options are:
- Nearest
This looks at the n nearby dataset points and
returns the
nearest
non-NaN dataset value (or NaN if they are all NaN), as is.
If >1 dataset values are equally close to the source point,
this method will return one of them (unspecified).
- Bilinear (the default)
This looks at the 4 nearby dataset points
in the latitude longitude plane and returns the
bilinear interpolated value.
This handles NaN values and only returns NaN for a point if all 4 nearby points are NaNs.
For this algorithm, the only allowed value of nearby is 4.
This algorithm is the default. If you don't specify an algorithm, Bilinear/4 will be used.
- Mean
This returns the mean of the non-NaN, nearby dataset values.
This does the calculations with doubles and returns doubles, regardless of the variable's data type.
The default nearby setting for this option is 4.
- SD
This returns the standard deviation of the non-NaN, nearby dataset values.
This does the calculations with doubles and returns doubles, regardless of the variable's data type.
The default nearby setting for this option is 4.
- Median
This returns the median of the non-NaN, nearby dataset values.
The default nearby setting for this option is 4.
This does the calculations with doubles and returns doubles, regardless of the variable's dataset type.
- Scaled
This returns the scaled, inverse distance squared interpolation
of the non-NaN, nearby dataset values (Y).
The weight for each nearby dataset value is w = (1 - D/Dmax)^{2} / (D/Dmax)
, where D is the distance (Davis, 1986, eq 5.68, page 371).
Thus, the furthest nearby dataset value gets weight=0.
Each interpolated point is E = ∑(w Y)/∑(w)
If a nearby dataset point has a distance of 0, that point's value is used as the interpolated value.
This does the calculations with doubles and returns doubles, regardless of the variable's data type.
The default nearby setting for this option is 4.
- InverseDistance
This returns the inverse distance interpolation
of the non-NaN, nearby data values (Y).
The weight for each nearby data value is w = 1/D
, where D is the distance (Davis, 1986, eq 5.67, page 367).
Each interpolated value is E = ∑(w Y)/∑(w)
If a nearby dataset point has a distance of 0, that point's value is used as the interpolated value.
This does the calculations with doubles and returns doubles, regardless of the variable's data type.
The default nearby setting for this option is 4.
- InverseDistance2
This is like InverseDistance, but the weight for each nearby data value is
w = 1/D^{2} .
- InverseDistance4
This is like InverseDistance, but the
weight for each nearby data value is
w = 1/D^{4} .
- InverseDistance6
This is like InverseDistance, but the
weight for each nearby data value is
w = 1/D^{6} .
All of the algorithms that calculate distances
(Nearest, Bilinear, Scaled and all InverseDistance algorithms),
calculate the distances in index space, not with the actual time, latitude, and longitude values.
Thus, this gives equal weight to one lat index unit and one lon index unit (and
if 3D, to one time index unit).
If this is inappropriate for a given datasetID/variable,
use one of the other algorithm options or see
Not what you want? below.
- nearby specifies the number of points to be used by the algorithm.
Only certain numbers are allowed.
For all algorithm options, there are 2D and 3D options for nearby.
The 2D options (4, 16, and 36, corresponding to 2D squares with sides that
are 1, 2, or 3 index values long (2^{2}, 4^{2}, 6^{2})) include nearby points in the 2D lat/lon plane.
The 3D options (8, 64, and 216, corresponding to 3D cubes with edges that
are 1, 2, or 3 index values long (2^{3}, 4^{3}, 6^{3})) include nearby points in 3D lat/lon/time space.
The algorithm=Nearest option also supports a nearby value of 1,
which only looks at the nearest data point in the 2D plane.
If a given longitude, latitude, time point is outside of a dataset's range,
the resulting data value will be NaN. However, if adding or subtracting
360 to the longitude value makes the point in the dataset's range,
the results will have the corresponding data value.
WARNING: If the closest longitude value is near the minimum or maximum
longitude value, this method only finds nearby points near that same minimum
or maximum longitude value -- it doesn't wrap around to find nearby points.
Instead, it duplicates the points at the minimum or maximum longitude value,
as needed. This is technically not correct behavior, but it only affects
some people, some of the time. If this affects you and is not acceptable, see
Not what you want? below.
- Other Dimensions
If a specified dataset has dimensions other than latitude, longitude,
and time (for example, depth), this converter will request the
dimension value closest to 0 for each of those other dimensions.
If this is not the desired behavior, see
Not what you want? below.
- The interpolation methods used here are from
Davis, J.C. 1986. Statistics and Data Analysis in Geology, 2nd Ed. John Wiley and Sons. New York, New York.
- Missing Data Values
If a dataset has no missing values, then a simple
algorithm like Bilinear yields reliably good results. But
when the dataset has missing values, the result can be a biased value
or a missing value. In this common situation, you may prefer to choose
the Scaled algorithm or one of the InverseDistance
algorithms and a larger number of nearby points. These
algorithms are better at dealing with missing values and are less likely
to return a missing value.
- In the response table, the names of the new columns are created by
taking each part of the datasetID/variable/algorithm/nearby CSV
(e.g., jplMURSST41/analysed_sst/Bilinear/4 ) and changing the /'s to _'s
(e..g, jplMURSST41_analysed_sst_Bilinear_4 )
so that they are valid variable names in all types of data files
- In the response table, except for algorithm=Nearest (which returns the original data type),
the interpolated values are returned as doubles, not the original data type.
Missing values are returned as NaN, never a missing_value stand in.
- Centered Grids?
The latitude and longitude values for gridded datasets may
identify the center of the relevant cell or the cell's lower left.
The
CF standard
says "If bounds are not provided, an application might reasonably
assume the gridpoints to be at the centers of the cells, but we do not require
that in this standard."
This converter treats the dataset's latitude and longitude values as points
so that it can interpolate values which fall in between the dataset's latitude
and longitude points. This is more consistent with the assumption that the
dataset's latitude and longitude values identify the center of the cell.
If this isn't the correct approach for a given dataset, in some cases
it may be appropriate to subtract half of the cell spacing from latitude and
longitude values that you supply to this converter.
- Not what you want?
If the methods or options offered by this converter are not suitable
for your purposes or otherwise not what you want, don't use them.
Instead, you can do it yourself, your own way:
download subsets of the dataset and apply your own algorithm.
To request that other interpolation algorithms be added to this converter,
please email bob dot simons at noaa dot gov .
- ERDDAP administrators:
You can change the default Request CSV by
by adding/editing the <convertInterpolateRequestCSVExample> tag in
datasets.xml.
Similarly, you can change the list of datasetID/variable options
by adding/editing the <convertInterpolateDatasetIDVariableList> tag in
datasets.xml, which can have a comma-separated list of suggested
datasetID/variable combinations.
The form on this web page helps humans generate a request URL that is
sent to ERDDAP when the human presses the Convert button.
However, you can bypass this form by generating the URL by hand or in a computer program
or script, and submitting the URL directly to ERDDAP.
- The time, latitude, longitude table in the form becomes the TimeLatLonTable parameter in the request.
- The Request CSV in the form becomes the requestCSV parameter in the request.
- The File type in the form becomes the file type of the request.
A sample request URL is
https://erddap.gcoos.org/erddap/convert/interpolate.htmlTable?TimeLatLonTable=
time%2Clatitude%2Clongitude%0A
2020-01-01T06%3A00%3A00Z%2C35.580%2C-122.550%0A
2020-01-01T12%3A00%3A00Z%2C35.576%2C-122.553%0A
2020-01-01T18%3A00%3A00Z%2C35.572%2C-122.568%0A
2020-01-02T00%3A00%3A00Z%2C35.569%2C-122.571%0A
&requestCSV=jplMURSST41%2Fanalysed_sst%2FBilinear%2F4
(That has line breaks inserted to make it easier to read, but it is really just one long line.)
URL Length - Note that as the amount of data in the TimeLatLonTable increases, the request URL
becomes very long. Because Tomcat has a default limit of 4096 characters per request URL,
this converter is limited to 100 rows of input -- 100 rows is the most you can
reliably request at one time. Even with this limit, this converter is much more
efficient than making a separate request for each source Time Latitude Longitude
value because bulk requests have 1/100th the network overhead and also get the data
from the source dataset much more efficiently (often almost 100 times more efficiently).
Percent Encoding - The TimeLatLonTable and requestCSV parameter values in the URL must be properly
percent encoded:
all characters other than A-Za-z0-9_-!.~'()* must be encoded as %HH,
where HH is the 2 digit hexadecimal value of the character, for example, a space becomes
%20. Characters above #127 must be converted to UTF-8 bytes, then each UTF-8 byte must
be percent encoded (ask a programmer for help). There are
websites that percent encode and decode for you.