Data hacks

Handling missing values

NCToolkit has built-in methods for setting or changing missing values: as_missing, missing_as and set_fill.

Change a value, or range of values, to missing with as_missing — here, zeroes, and then everything from -1000 to 0:

ds.as_missing(0)
ds.as_missing([-1000, 0])

Change missing values to a constant with missing_as, or change the netCDF fill value itself with set_fill — handy when combining files that used different fill values:

ds.missing_as(-9999.99)
ds.set_fill(-9e38)

Shifting time

If you're missing a year of data and want to reuse the prior year's values, you'll first need to shift its times forward by a year. shift moves time forward or backward by hours, days, months or years:

ds.shift(years=-1)   # shift backward one year
ds.shift(hours=12)   # shift forward 12 hours

Arguments allow partial matches, so hour, day, month or year work just as well.

Adding cell areas

Add grid-cell area (in square metres) to a dataset:

ds.cell_area()                # adds cell area to the dataset
ds.cell_area(join=False)   # dataset of cell areas only

This only works where it's possible to calculate the area of each grid cell from the file's metadata.

Changing netCDF format

format sets the netCDF format used for the files in a dataset:

ds.format("nc4")

Dimensions with only one value

Drop a dimension that only has one value — a single leftover time step, say — with reduce_dims:

ds.reduce_dims()

Removing leap days

ds.drop(month=2, day=29)

Renaming variables

Use rename with a dictionary mapping original names to new ones:

ds.rename({"x": "y"})