baseflow

A matlab toolbox for characterizing aquifer properties from streamflow measurements using baseflow recession analysis. Developed at the Pacific Northwest National Laboratory to study changes in soil water storage in Arctic and Subarctic watersheds. Supported by the Interdisciplinary Research for Arctic Coastal Environments (InteRFACE) project.

For an overview, see Getting Started.

Requirements

• MATLAB®, developed on version 9.9.0 (R2020b).
  • Mathworks Statistics and Machine Learning Toolbox.
  • Mathworks Curve Fitting Toolbox.
• Octave: tests/octave_smoke.m verifies the core workflow (load example data, getevents, fitevents, fitab with nls) on version 11.3.0. The toolbox ran on version 8.2.0 in 2023.
  • Struct, Optimization, Statistics, Tablicious, Statistics-resampling (formerly Statistics-bootstrap), and Financial packages.

The DESCRIPTION file lists the required dependencies. The theory demos (baseflow_linear_theory and baseflow_nonlinear_theory) also need the Symbolic Math Toolbox on MATLAB or the symbolic package on Octave. DESCRIPTION does not list these optional demo dependencies.

The core analysis chain (getevents, fitevents, fitab, eventfinder, eventtau, globalfit, fitphi) needs no functions or data files outside the toolbox.

Four public functions do not run with the toolbox files alone. The core analysis chain does not call them. The toolbox does not currently support these functions for public use.

• loadbasins, loadflow, loadmeta, and loadprops load .mat files from the folder that the BASEFLOW_DATA_PATH environment variable names. The toolbox does not ship these files.
• loadflow also references other functions that the toolbox does not ship.

One optional feature also has an external requirement: the plfitb method 'hanel' calls r_plfit, the Hanel et al. (2017, PLOS ONE 12:e0170920) supplementary code. That code carries no license grant, so the toolbox does not vendor it. Get the code from the paper's supplement and put it on the MATLAB path. baseflow.internal.dependencies lists r_plfit in known_external and does not count it as missing. Note: the Hanel supplement code returns one out struct, while plfitb calls a copy edited to return [alpha, xmin, L, D, out]. Edit the function signature of your copy to return these five outputs. toolbox/docs/baseflow_powerlaw_notation.m maps the notation between plfit, r_plfit, the MATLAB generalized Pareto, and the toolbox's b and tau conventions.

Install

• Clone this repository.
• cd into the toolbox directory.
• Run Setup.m:
  • Type msg = Setup('install') at the command window then press enter.
  • Check msg for information about the installation.
• If running in Octave, see .octaverc for recommended startup options.
• Unit tests are in tests/. To run them from the top-level folder:
  • Type runtests('tests') at the command window and press enter.
  • To debug, try runtests('tests', 'Debug', true).
• In new Matlab sessions, cd to the toolbox directory and try Setup('addpath') or just Setup to add the toolbox to your search path (or manage the search path however you normally do).

The optional Matlab Package Installer (MPI) route uses this repository's explicit URL and a numeric release tag:

mpi install -u https://github.com/mgcooper/baseflow.git -t <numeric-tag>

The selected release must include a root pathList.m containing toolbox. It must also pass an installation check from a clean checkout. No existing release has been qualified for this command. Use the clone route until a qualified release is published. Do not install this project by the bare name baseflow.

For more options, see Configuration.
For Octave compatibility, see Octave.

Get started

Toolbox functions are in the +baseflow namespace. To see a list of toolbox functions, type help +baseflow at the command line then press enter. To see the help for a specific function, click on any of the hyperlinks, or type help baseflow.<function_name> at the command line then press enter.

Toolbox documentation is available in the Matlab help browser. To get started, in your Matlab command window:

• Type baseflow.help() to open the toolbox documentation in the help browser.
• Type baseflow.help('<function_name>') to open documentation for a specific function.
• Type doc baseflow or try doc +baseflow to see the package contents in the help browser.
• If the documentation does not open in the help browser, try doc without any arguments, then scroll down to "Supplemental Software" and click on "Baseflow Recession Analysis Toolbox". You can also try docsearch baseflow.

Notebooks are in toolbox/demos/. The notebook baseflow_demo_kuparuk.mlx replicates the analysis in the paper Detecting Permafrost Active Layer Thickness Change From Nonlinear Baseflow Recession. Each demo is available as an html file in toolbox/docs/html. Double click to view them in the Matlab help browser, or use baseflow.help(<docname>). All demos are available as live .mlx files compatible with Matlab, and as .m files in the toolbox/demos/mfiles folder. See Octave for the demos that run on Octave.

Contribute

If you find a bug, have a question, or want to contribute, feel free to open an issue or start a discussion.

How do I cite this?

If you find this software useful, please consider citing the software release in CITATION.cff and/or the following paper for which the software was developed:

    @ARTICLE{10.1029/2022WR033154,
    AUTHOR={Cooper, Matthew G, Zhou, Tian, Bennett, Katrina E, Bolton, W Robert, Coon, Ethan T, Fleming, Sean W, Rowland, Joel C, and Schwenk, Jon},
    TITLE={Detecting Permafrost Active Layer Thickness Change From Nonlinear Baseflow Recession},
    JOURNAL={Water Resources Research},
    VOLUME={59},
    PAGES={e2022WR033154},
    YEAR={2023},
    URL={https://doi.org/10.1029/2022WR033154},
    DOI={10.1029/2022WR033154},
    ISSN={1944-7973},
    }

Configuration

Installing baseflow is as simple as cloning this repo and adding it to the Matlab search path. Users should manage the toolbox however they normally manage their search path, for example using startup.m. Note that functions in the +baseflow namespace will not interfere with functions sharing the same name on the search path, so feel free to put the toolbox on your userpath if you want it available on startup.

For more control, use the convenience function Setup.m to manage the toolbox installation and paths. Starting from an initial install in your local repo directory:

• Running Setup('install') does the following:
  • Toolbox paths are added to the search path (not persistent between sessions).
  • Default toolbox preferences are added to a new user preferences group baseflow (persistent between sessions).
  • The dependency check (baseflow.internal.dependencies('', 'check')) runs on MATLAB and stores its results in msg. On Octave, Setup skips the check.
• Note that Setup does not modify userpath, does not call savepath, and never modifies the root-level pathdef.m file. It only calls addpath and rmpath to add and remove the toolbox from the search path.
• In subsequent sessions, toolbox paths can be managed like so:
  • Setup('addpath') or simply Setup with no arguments adds the toolbox to the search path for the current session.
  • Setup('rmpath') removes the toolbox from the search path for the current session.
• To display the current toolbox preferences try getpref('baseflow').

Running Setup('install') should only be necessary once (or not at all, if you choose to manage your search path however you normally manage third-party Matlab/Octave software). However, Octave users may find Setup particularly convenient because it will load the required packages. The core analysis chain needs no files outside the toolbox. Requirements lists the functions that need external functions or data. If you find another missing dependency, please open an issue.

To run the dependency check at any time on MATLAB, use msg = Setup('dependencies'). The check stores these results in msg:

• missing_dependencies: the required files that resolve outside the toolbox, or 'all dependencies are installed'.
• undeclared_products: the detected MATLAB products that the DESCRIPTION MatlabProducts line does not list, or 'all products are declared'.

The analysis cannot see a call that does not resolve on the search path. On a machine without the external files, missing_dependencies can report 'all dependencies are installed' while a function still calls a file that the toolbox does not ship. The dependencies_checked preference is true only when missing_dependencies reports 'all dependencies are installed'.

Package namespace

The baseflow toolbox uses the package namespace +baseflow. Package functions are accessed using dot notation: baseflow.<function name>. The package does not need to be imported, but the parent folder containing the +baseflow package folder (i.e., the toolbox folder) needs to be on the Matlab search path.

If desired, package functions can be imported into a workspace using import baseflow.<function name>. Subsequent calls to the imported function can omit the package prefix. The entire package can be imported using import baseflow.*. However, imported functions are only available in the calling workspace. To use imported package functions in called functions or class definition files, import them again in those files or just use dot notation at all times, which is the convention used throughout the baseflow toolbox.

Octave

Octave support covers the core analysis functions and the demos. baseflow was developed on Matlab, and users may encounter unexpected behavior on Octave (please open an issue). In particular, baseflow uses the tablicious package for string and datetime support, but tablicious does not fully support these objects. tests/octave_smoke.m verifies the core workflow on GNU Octave 11.3.0. The toolbox ran on Octave 8.2.0 in 2023. See the limitations below. Octave can be downloaded here. If running in Octave, the following packages are required:

struct
optim
statistics
tablicious
statistics-resampling (the former statistics-bootstrap; either name works)
financial

For some demos, the Symbolic package is needed.

To see which packages are installed: pkg list

To install packages, use the pkg command in Octave. pkg install checks package dependencies but does not install them, so install the packages in this order. financial depends on io, and statistics 1.9.1 depends on datatypes. statistics 1.9.1 and datatypes 1.3.4 need Octave 11.1 or later.

pkg install -forge struct
pkg install -forge io
pkg install -forge datatypes
pkg install -forge statistics
pkg install -forge optim
pkg install -forge financial

Install statistics-resampling from the repository:
pkg install "https://github.com/gnu-octave/statistics-resampling/archive/refs/heads/master.zip"

Install tablicious from the repository:
pkg install https://github.com/apjanke/octave-tablicious/archive/refs/heads/master.zip

To load a package, use pkg load <pkgname>. To see which packages are loaded, use pkg list, loaded packages will have an asterisk next to their name. Use the convenience function Setup.m to automatically import required packages.

The .octaverc file loads the required packages with pkg load. Depending on your configuration, it may or may not be sourced at startup. Octave users are encouraged to run Setup when using the toolbox, it will load the required packages and manage warnings. See Setup.m for more information.

Limitations when running in Octave:

• The live scripts in the toolbox/demos/ folder will not work on Octave. Use the .m files in toolbox/demos/mfiles instead.
• On Octave 11.3.0 with the statistics package 1.9.1, baseflow_demo_1, baseflow_demo_2, baseflow_demo_3, and baseflow_demo_kuparuk run. The theory demos need the symbolic package.
• Functions relying on datetime objects may not work on Octave.
• Graphics objects are not supported in Octave, including gobjects which may cause errors.
• checkevent and eventpicker call yyaxis, which Octave does not implement.
• Latex interpreter is not supported in Octave.
• The +internal maintenance tooling and its private helpers are MATLAB-only. Setup('dependencies') skips the analysis on Octave.

Work is ongoing to patch these incompatibilities. See +baseflow/private/isoctave to patch errors.

When running in Octave, be careful with blanket warning on or warning off commands. Octave ships with about a dozen warning states off, listed below. If they are turned on by a warning on command, there will be endless warning messages. If this happens, type warning in the commandwindow to confirm if the following warnings are off. If so, simply restart Octave to reset them, or reset them manually.

    State  Warning ID
    off  Octave:array-as-logical
    off  Octave:array-to-scalar
    off  Octave:array-to-vector
    off  Octave:imag-to-real
    off  Octave:language-extension
    off  Octave:missing-semicolon
    off  Octave:neg-dim-as-zero
    off  Octave:separator-insert
    off  Octave:single-quote-string
    off  Octave:str-to-num
    off  Octave:mixed-string-concat
    off  Octave:variable-switch-label

Acknowledgement

The Interdisciplinary Research for Arctic Coastal Environments (InteRFACE) project funded this work through the United States Department of Energy, Office of Science, Biological and Environmental Research (BER) Regional and Global Model Analysis (RGMA) program. Awarded under contract grant # 89233218CNA000001 to Triad National Security, LLC (“Triad”).