Development version only: not yet in a tagged release. Install from GitHub to use it — see below.
The oceanval browser window is on the main branch but not yet in a tagged release (the latest is v0.5.7; see version history), so the conda-forge package does not include it. Set up the environment as described in Installing, then install from GitHub: pip install git+https://github.com/pmlmodelling/oceanval.git.
OceanVal can be used without writing any Python. Run oceanval in a terminal and a page opens in your web browser that takes you from a folder of model output to a validation report: you choose what to do, point it at your simulation, choose the observations to validate against, check the units, and watch the run.
In a terminal, in the directory to work in:
oceanval
OceanVal and python -m oceanval do the same. Matchups, scripts and reports are written in that directory unless you choose another, so make a new directory for each simulation.
Try a demo, below the four choices on the first page, shows how OceanVal works before you use it on your own model. First you choose one of three CMIP6 climate models, each downloaded from a different ESGF data server (NorESM2-LM from noresg.nird.sigma2.no, MPI-ESM1-2-LR from esgf3.dkrz.de and UKESM1-0-LL from esgf.ceda.ac.uk), and one to three of its variables, in a grid with a model on each row and sea surface temperature, surface nitrate and sea surface salinity as the columns. Ticking a variable of another model unticks the first model’s, and only the files ticked are downloaded: about 50 MB for NorESM2-LM and 26 MB for MPI-ESM1-2-LR for all three, but 460 MB for UKESM1-0-LL, as its monthly files cover 1950–2014. A short page of instructions follows (press Continue on each page, and tick the agreement when asked to run the validation; change the options and settings if you like), which ends by asking how you want your validation sample: Concise and fast, the default, with results in a couple of minutes, or Detailed, but slower, in 4 or 5 minutes. Then OceanVal takes you through the usual steps, a matchup and validate run, without the step for your own data, as only OceanVal’s own datasets are used, with the options filled in for you in red and bold: the simulation, the model’s name as the only file names to use (the files of every model tried share one directory), the year 2010 only, global longitude and latitude limits, COBE-SST 2 to validate the temperature against and the World Ocean Atlas 2023 for the nitrate and salinity, the concise report you chose, with no subregions, or the detailed one, with the global subregions. The units step suggests converting the observed nitrate, which you check as you would for your own model. It shows how OceanVal works, not how to validate a climate model, which needs many years of output rather than one.
The demo creates a directory called oceanval_demo, in the directory you started oceanval in, holding the downloaded files, the matchups and the report. Remove it when you have finished. Running the demo again uses the files already downloaded.
The window shows where you are in a row of steps at the top. Every step after the first has Back, which keeps what you have entered: get as far as the report’s options, spot a mistake, go back as far as the simulation to fix it, and carry on with everything else as you left it. Whatever you change replaces what was there, in the steps after it too. The units always have to be confirmed again, and once anything is matched up, there is no going back.
Closing the browser window quits oceanval in the terminal, within about 15 seconds, stopping anything it is running, whichever browser you use. So do Quit (which is instant) and Ctrl+C. Reloading the page, or moving between OceanVal’s own windows, does not.
Match up new data and validate it, match up only (and build the report later, from the same matchups), or validate matchups made earlier, which asks for the report’s options instead of the next steps.
Add your own validation data for future use is the odd one out, as it matches nothing up. It saves observations of your own, point or gridded, as recipes in a .oceanvalrc file, for this directory or for everywhere, to use in any matchup like OceanVal’s built-in ones. It asks which kind of data, then for a description of the data: its variable, the name of its source, where it is (for gridded data on this machine, on a THREDDS server or at a web address; point data is csv files on this machine), what is in it and where to save the recipe. OceanVal opens the data as you describe it, and checks it, before a recipe can be saved. What it works out is in red and bold. The recipes then appear, tagged Yours, in the recipes window of a matchup.
Where the model output is, how many directories down its files are, which files to skip or keep, and the domain (global or the Northwest European Shelf). Type the directory, with folders suggested as you go, or use Browse…, which looks through the folders on the machine OceanVal runs on. Once you have matched up a simulation, the arrow at the right of the box lists the simulations you have validated before (remembered once you confirm the matchups), latest first, so that you can pick one instead of typing it. They are remembered in a .oceanvalcache JSON file in your home directory (the OCEANVALCACHE environment variable names another file to use in its place), which OceanVal only adds to and never replaces, so a new version of OceanVal keeps it. As you type, the page counts the files that pass the filters. The next page starts with years inferred from their names, which you can change there. It also asks where to save the matchups and, for a matchup and validate run, the report: the directory you started oceanval in, unless you choose another, and it warns you if there are matchups there already. It also says where the matchup script will be written.
Whether you have observations of your own as well as OceanVal’s built-in datasets. If you do, a page for point data (csv files) and one for gridded data (netCDF) let you add them one at a time, with every argument of add_point_comparison or add_gridded_comparison; the ones that have to be given are marked in red.
Each is added with one of two buttons. Add this data for use now only adds it to this matchup. Add this data for use now and in future also saves it as a recipe in a .oceanvalrc file, as Add your own validation data for future use does, so that later matchups can use it by name. It first shows a page with what a recipe needs that the form did not ask for: the units, where to save the recipe and, if you left them out, the source information and the variable’s names in the report. What OceanVal will use unless you change it is in amber: the units in a netCDF file, “Source for” and the source’s name, the variable’s name, and everywhere (your home directory) for where to save it. Type over any of them to change it. The units of csv files, which do not say, and of a netCDF variable without them, are marked in red, as they have to be given. Data on a THREDDS server or at a web address is checked first, and only saved once it has been. The source name has to be letters and numbers only, as a recipe’s is, and anything else that stops it being saved is said on the form, where it can still be added for now only. Back keeps what you typed. Once saved, the dataset is listed as also saved for future use; in this matchup it is registered as your own data, so its recipe is left out of the recipes window, while later ones offer it with the rest.
OceanVal reads one file of each kind from the simulation, works out which model variable holds each observational variable, and shows the recipes window. Check the variables it found (a pop-out under each model variable box lists every variable in the output with its long_name; search it, and tick several to sum them as var1+var2+var3, with the ones selected listed at the top), tick the datasets to validate against, and set the global settings (years, the spatial subset to match up, cores, thickness and missing values). The report’s options are asked for later, once you have checked the matchups (see step 7). Clear all selections unticks everything so you can start from none. The output directory is the one chosen in step 2, and OceanVal always asks you to check the matchups before matching (see step 6). A dataset set to Vertical needs a thickness — z_level, a cell thickness variable or a file — before you can carry on.
Back from a later step shows the recipes window as you left it. Go back before it, and the simulation is read again when you carry on, as you may have changed it: the window then keeps your global settings and each row you changed whose model variable is still in the output, with its ticked datasets and their options unless you changed the domain, when they are ticked afresh for it. What it could not keep of yours is said in red and bold. Reset still puts back what OceanVal found.
The model’s and the observations’ units, with a conversion suggested where OceanVal thinks they differ, to check and always confirm. See below.
The page starts by saying, in large letters, that it is identifying the files that meet your criteria. Once it has, it shows what it found as a table: each variable, its model variable, the observations it is compared with, the pattern of the files it is in and their time resolution: monthly, or 1d, 2d, 5d and so on for output every so many days (anything more often than daily is 1d). Where a variable is in files of more than one resolution, the finest is used. List all files lists every file of a variable, and says first that OceanVal applies temporal subsetting to them, so only their times in the years being matched up (or a dataset’s own years) are used. Yes, continue goes on to the report’s options for a matchup and validate run, and starts the matchup for a match up only run, unless a point dataset needs checking first (see below). Both it and Back are at the foot of the page. Back, while the files are being found or asked about, stops the matchup before it has matched anything up, and goes back to the units with your conversions as you left them, so you can start again with other model variables or file filters; the files are found again when you carry on.
If any point dataset is matched by day (its point_time_res includes day, as the default does) but the model output for its variable is coarser than daily, Yes, continue first shows a page saying so: its observations are matched to model output on the same day, so most of them may find nothing to match. It lists each such dataset with its time resolution and how it is matched now, and offers to change point_time_res for all of them at once (OceanVal suggests Year, month, in red and bold), for each one, or to keep them as they are. Nothing is matched up until you carry on, and the choice is written into the matchup script. Back shows the files again.
For a matchup and validate run, one last page: “One last thing... How would you like your validation report?” Nothing has been matched up yet, and nothing is until you carry on. Choose a subregion to validate, regional summaries, a transect to validate the gridded datasets along (given as a start and end longitude and latitude, which must run north–south or east–west: the page marks anything else in red and does not let you carry on), fixed colour scales, PDF and Word versions of the report, a zipped copy, and whether the report is concise or detailed. If a point dataset is matched up with Vertical ticked, the page ends with the depth bins its depth summaries use, as a table of From and To depths in metres, starting from OceanVal’s own: change any of them, remove a bin with its ×, add one with Add a bin, or Restore defaults. Leave To empty on the deepest bin for everything below it; bins that overlap are marked in red, and the page does not let you carry on until they are put right. The report is built in oceanval_report, beside the matchups, in the directory chosen in step 2. Back shows the matchups again (or the page about point datasets matched by day, if it was shown), and keeps the options you chose. Match up and validate starts the matchup, and the options are written into the matchup script’s validate() call, so running it again from a terminal builds the same report. With jupyter-book 2 or later, they are written into its matchup() call too, as live_validation, for the interim report (see step 8).
The matchup runs, and then, for a matchup and validate run, validate() with the report’s options. The page says what the run is doing (for a matchup and validate run, how many of the matchups are made), and, once the matchups have been checked, shows the output as it appears in the terminal, with Copy output, as it does if the run fails. Anything else OceanVal asks, such as whether to try again for observations a server could not supply, is asked in the page. When a matchup and validate run finishes, the page shows where the HTML report is, with Open the validation report, which opens it in a tab of its own, and validate() opens it in your browser too, where one can be opened. After a match up only run, Validate these matchups builds the report from them.
With jupyter-book 2 or later, a matchup and validate run also builds an interim validation report while the matchups are made, so that you can look at the results long before the last one is done. Once the first matchup is in it, Open the interim validation report links to it, with how many of the matchups it has so far. Each matchup’s page is added as soon as the matchup is made, and the summary is run again, so reload the report to see the latest. The interim report is HTML only, in oceanval_interim_report, beside the matchups, and every page of it says it is interim. Once all the matchups are in it, the full report is built, with PDF and Word versions if you asked for them. The window serves both reports itself, so their links work when the window is forwarded from a remote machine too.
A model and its observations can report the same quantity in different units, for example nutrients in mmol/m³ against µmol/kg. OceanVal compares the values as they are, so a difference shows up as a large error in the report. After the recipes, the window reads the units of every matchup and lists them in two sections, Gridded datasets and Point datasets, each with:
The observations become observations × multiply by + add before they are compared with the model, for example adding 273.15 to observations in °C when the model is in kelvin. Where OceanVal thinks the units differ and can tell how to convert one into the other, it suggests a conversion in the boxes and says what it assumed:
Salinity and pH are never converted: psu, 1 and 1e-3 are the same scale of salinity, and pH has no units. Units OceanVal does not recognise, or that measure different things, such as moles and grams, are marked Not checked, for you to look at. OceanVal can get this wrong, so check every row, not only the conversions it suggests, and leave the boxes empty where you agree that no conversion is needed. Anything OceanVal has filled in, and the note under it, is in red and bold, so that none of it is missed. If you change a number OceanVal filled in, it is yours, and no longer red. The conversions are written into the matchup script as obs_multiplier and obs_adder, and one you gave with your own data is kept.
The units always have to be confirmed. Beside Back and Continue, tick the box labelled “Tick if you are happy with the units” once you have checked the units of every matchup and, with these conversions, the observations are in the same units as the model. Continue stays disabled until you have, and OceanVal refuses to carry on without it, whatever is sent to it. You confirm the units every time you reach this step, however many of them OceanVal found to match, and changing a conversion after you have ticked the box clears the tick, so that what you confirmed is what is used. When you come back to this step, the boxes hold the conversions you left in them, for each matchup that is still the same (its model variable and units, and its observations’ variable and units); one that has changed starts afresh.
Point data is csv files, which have no units, so OceanVal cannot tell you what they are. It is up to you to make sure they match the model’s, and to convert them with the boxes if they do not.
The window does nothing you could not do in Python. It writes the matchup script first, to matchup.py unless you choose another name, and then runs it. The script holds your own data, every recipe (those you did not tick are commented out), the conversions, the oceanval.matchup() call and, for a matchup and validate run, the oceanval.validate() call, so you can read it, edit it and run it again later with python matchup.py.
If oceanval cannot open a browser for you, it prints a link to open. VS Code forwards its port for you. Over plain SSH, for example from a Windows terminal, it prints the command that forwards the port, such as ssh -L 8765:127.0.0.1:8765 you@server: run it in a new terminal on your own computer (PowerShell on Windows 10 or later has ssh), leave it open, and open the link in your browser. In an SSH session oceanval uses port 8765 if it is free, so the command is the same every time; choose another port with --port. The printed login is a guess: use the one you normally connect with. The run carries on if you close the page, and you can come back to it with the same link.
