• Welcome
  • I Introduction
  • 1 Project Overview
  • 2 Contributor Covenant Code of Conduct
  • 3 About the PEcAn Book
    • 3.1 General Feedback/Comments/Suggestions
    • 3.2 Editing this book
    • 3.3 How to create your own version of Documentation
  • II Tutorials, Demos and How To’s
  • 4 Install PEcAn
    • 4.1 Docker
    • 4.2 Installation From r-universe
    • 4.3 (Advanced) Native install
  • 5 Tutorials
    • 5.1 Running PEcAn Basic Run using Quarto
      • 5.1.1 Prerequisites
      • 5.1.2 Steps to Run
      • 5.1.3 View Generated Artifact
    • 5.2 Running Uncertainty Analysis using Quarto
      • 5.2.1 Prerequisites
      • 5.2.2 Steps to Run
      • 5.2.3 View Generated Artifact
    • 5.3 Demo 01: Basic Run PEcAn
    • 5.4 Demo 02: Sensitivity and Uncertainty Analysis
      • 5.4.1 Run Specification
      • 5.4.2 Additional Outputs:
    • 5.5 Other Vignettes
      • 5.5.1 Simple Model-Data Comparisons
      • 5.5.2 Data Assimilation Concepts
      • 5.5.3 Parameter Data Assimilation
      • 5.5.4 State-Variable Data Assimilation
      • 5.5.5 PEcAn: Testing the Sensitivity Analysis Against Observations”
    • 5.6 Advanced User Guide
      • 5.6.1 Submitting Workflow from Command Line
  • 6 Basic Web workflow
    • 6.1 Site and model selection
      • 6.1.1 Selecting a model
      • 6.1.2 Selecting a site
      • 6.1.3 Site Groups
      • 6.1.4 Using existing sites
      • 6.1.5 Adding a new site
      • 6.1.6 Troubleshooting
    • 6.2 Model configuration
      • 6.2.1 Choosing meteorology
      • 6.2.2 Met workflow
      • 6.2.3 Troubleshooting meteorological conversions
    • 6.3 Selecting Plant Functional Types (PFTs) and other parameter groupings.
      • 6.3.1 Using existing PFTs
      • 6.3.2 Creating new PFTs
      • 6.3.3 Choosing initial vegetation
      • 6.3.4 US FIA
      • 6.3.5 Spin up
      • 6.3.6 Selecting a soils product
      • 6.3.7 Soil texture, depth, and physical parameters
      • 6.3.8 Other model inputs
  • 7 More on the PEcAn Web Interface
    • 7.1 Additional web configuration
      • 7.1.1 Web interface setup
      • 7.1.2 Advanced Setup
      • 7.1.3 Editing model configurations
    • 7.2 Settings-configured analyses
      • 7.2.1 Parameter data assimilation (PDA)
      • 7.2.2 State data assimilation (SDA)
      • 7.2.3 Running SDA on remote
      • 7.2.4 Restart functionality in SDA
      • 7.2.5 State Data Assimilation Methods
      • 7.2.6 MultiSettings
      • 7.2.7 Benchmarking
  • 8 Developer guide
    • 8.1 Updating PEcAn Code and Bety Database
      • 8.1.1 Updating PEcAn
    • 8.2 Git and GitHub Workflow
      • 8.2.1 Using Git
      • 8.2.2 Recommended Git Workflow
      • 8.2.3 Useful Git tools
      • 8.2.4 Advanced
      • 8.2.5 References
      • 8.2.6 GitHub use with PEcAn
    • 8.3 Coding Practices
      • 8.3.1 Coding Style
      • 8.3.2 dplyr
      • 8.3.3 Logging
      • 8.3.4 Package Data
      • 8.3.5 Documenting functions using roxygen2
    • 8.4 Docker
    • 8.5 Introduction to Docker?
      • 8.5.1 What is Docker?
      • 8.5.2 Working with Docker
      • 8.5.3 docker-compose
    • 8.6 Quick-start docker install
    • 8.7 The PEcAn docker install process in detail
      • 8.7.1 Configure docker-compose
      • 8.7.2 Initialize PEcAn (first time only)
      • 8.7.3 Troubleshooting
    • 8.8 PEcAn Docker Architecture
      • 8.8.1 Overview
      • 8.8.2 PEcAn’s docker-compose
      • 8.8.3 Top-level structure
      • 8.8.4 traefik
      • 8.8.5 portainer
      • 8.8.6 minio
      • 8.8.7 thredds
      • 8.8.8 postgres
      • 8.8.9 rabbitmq
      • 8.8.10 bety
      • 8.8.11 docs
      • 8.8.12 web
      • 8.8.13 executor
      • 8.8.14 monitor
      • 8.8.15 Model-specific containers
    • 8.9 Models using Docker
      • 8.9.1 Model information
      • 8.9.2 Model build
      • 8.9.3 Common problems
      • 8.9.4 Debugging missing libraries
    • 8.10 Building and modifying images
      • 8.10.1 Local development and testing with Docker
    • 8.11 Troubleshooting Docker
      • 8.11.1 “Package not available” while building images
    • 8.12 Migrating PEcAn from VM to Docker
      • 8.12.1 Running BETY as a docker container
    • 8.13 The PEcAn Docker API
    • 8.14 RabbitMQ
      • 8.14.1 Producer – PEcAn.remote::rabbitmq_post_message
      • 8.14.2 Consumer – receiver.py
      • 8.14.3 RabbitMQ and the PEcAn web interface
      • 8.14.4 RabbitMQ in the PEcAn XML
      • 8.14.5 RabbitMQ configuration in Dockerfiles
      • 8.14.6 Case study: PEcAn web interface
    • 8.15 Testing
      • 8.15.1 Unit testing
      • 8.15.2 Integration testing
      • 8.15.3 Continuous Integration
    • 8.16 Download and Compile PEcAn
      • 8.16.1 Download, compile and install PEcAn from GitHub
      • 8.16.2 PEcAn Testrun
    • 8.17 Directory structure
      • 8.17.1 Overview of PEcAn repository as of PEcAn 1.5.3
      • 8.17.2 Generic R package structure:
  • III Topical Pages
  • 9 PEcAn standard formats
    • 9.1 Defining new input formats
    • 9.2 Time Standard
    • 9.3 Input Standards
      • 9.3.1 Meteorology Standards
      • 9.3.2 Soils and Vegetation Inputs
    • 9.4 Output Standards
  • 10 The PEcAn XML
    • 10.1 Core configuration
      • 10.1.1 Top-level structure
      • 10.1.2 info: Run metadata
      • 10.1.3 outdir: Output directory
      • 10.1.4 database: PEcAn database settings
      • 10.1.5 pft: Plant functional type selection
      • 10.1.6 meta.analysis: Trait Meta Analysis
      • 10.1.7 model: Model configuration
      • 10.1.8 run: Run Setup
      • 10.1.9 host: Host information for remote execution
    • 10.2 Advanced features
      • 10.2.1 ensemble: Ensemble Runs
      • 10.2.2 sensitivity.analysis: Sensitivity analysis
      • 10.2.3 Parameter Data Assimilation
      • 10.2.4 Multi-Settings
      • 10.2.5 (experimental) State Data Assimilation
      • 10.2.6 State Variables for State Data Assimilation
      • 10.2.7 Observation Preparation for State Data Assimilation
      • 10.2.8 (experimental) Benchmarking
      • 10.2.9 Remote data module
  • 11 PEcAn workflow (web/workflow.R)
    • 11.1 Read Settings
    • 11.2 Input Conversions
    • 11.3 Input Data
    • 11.4 Initial Conditions
      • 11.4.1 CONUS (NEON/FIA/BADM) Initial Conditions.
      • 11.4.2 North America (NA) Initial Conditions.
    • 11.5 Meteorological Data
      • 11.5.1 Downloading Raw data (Description of Process)
      • 11.5.2 Converting raw data to PEcAn standard
      • 11.5.3 Downscaling and gapfilling (optional)
      • 11.5.4 Converting from PEcAn standard to model-specific format
    • 11.6 Input phenological Data
    • 11.7 Traits
    • 11.8 Meta Analysis
    • 11.9 Model Configuration
    • 11.10 Run Execution
    • 11.11 Post Run Analysis
  • 12 PEcAn Models
    • 12.1 BioCro
      • 12.1.1 Introduction
      • 12.1.2 PEcAn configuration file additions
      • 12.1.3 Model specific input files
      • 12.1.4 Model configuration files
      • 12.1.5 Installation notes
    • 12.2 CLM
    • 12.3 DALEC
    • 12.4 ED2
      • 12.4.1 Introduction
      • 12.4.2 PEcAn configuration file additions
      • 12.4.3 PFT configuration in ED2
      • 12.4.4 Model specific input files
      • 12.4.5 Model configuration files
      • 12.4.6 Installation notes
    • 12.5 GDAY
    • 12.6 LDNDC
      • 12.6.1 Introduction
      • 12.6.2 Model specific input files
      • 12.6.3 Installation notes
    • 12.7 LINKAGES
    • 12.8 LPJ-GUESS
    • 12.9 MAESPA
    • 12.10 PRELES
    • 12.11 RothC
    • 12.12 SiPNET
    • 12.13 STICS
  • 13 Available Meteorological Drivers
    • How to choose a meteorological driver
    • Summary of available drivers
    • 13.1 Ameriflux
    • 13.2 AmerifluxLBL
    • 13.3 Fluxnet2015
    • 13.4 NARR
    • 13.5 CRUNCEP
    • 13.6 CMIP5
    • 13.7 NLDAS
    • 13.8 GLDAS
    • 13.9 PalEON
    • 13.10 FluxnetLaThuile
    • 13.11 Geostreams
    • 13.12 ERA5
    • 13.13 ICOS Drought 2018
    • 13.14 ICOS Ecosystem Archive
    • 13.15 Download GFDL
    • 13.16 CM3
    • 13.17 ESM2M & ESM2G
  • 14 PEcAn Project API
    • 14.1 Introduction
    • 14.2 Authentication
    • 14.3 RESTful API Endpoints
    • 14.4 Examples:
      • 14.4.1 Prerequisites to interact with the PEcAn API Server
      • 14.4.2 GET /api/ping
      • 14.4.3 GET /api/status
      • 14.4.4 GET /api/models/
      • 14.4.5 GET /api/models/{model_id}
      • 14.4.6 GET /api/sites/
      • 14.4.7 GET /api/sites/{site_id}
      • 14.4.8 GET /api/pfts/
      • 14.4.9 GET /api/pfts/{pft_id}
      • 14.4.10 GET /api/formats/
      • 14.4.11 GET /api/formats/{format_id}
      • 14.4.12 GET /api/inputs/
      • 14.4.13 GET /api/inputs/{input_id}
      • 14.4.14 GET /api/workflows/
      • 14.4.15 POST /api/workflows/
      • 14.4.16 GET /api/workflows/{id}
      • 14.4.17 GET /api/workflows/{id}/status
      • 14.4.18 GET /api/workflows/{id}/file/{filename}
      • 14.4.19 GET /api/runs/
      • 14.4.20 GET /api/runs/{run_id}
      • 14.4.21 GET /api/runs/{run_id}/input/{filename}
      • 14.4.22 GET /api/runs/{run_id}/output/{filename}
      • 14.4.23 GET /api/runs/{run_id}/graph/{year}/{y_var}
  • 15 Database synchronization
    • 15.1 How does it work?
    • 15.2 Set up
    • 15.3 Fetch latest data
    • 15.4 Sharing data
    • 15.5 Automation
    • 15.6 Database maintentance
      • 15.6.1 Reindexing the database
      • 15.6.2 Vacuuming the database
    • 15.7 Troubleshooting
      • 15.7.1 Username and password
      • 15.7.2 db_hba.conf file
    • 15.8 Network Status Map
    • 15.9 Tasks
  • 16 Standalone tools (modules)
    • 16.1 Loading Data in PEcAn
      • 16.1.1 Inputs
      • 16.1.2 Output
      • 16.1.3 Example
    • 16.2 Remote data module
      • 16.2.1 Configuring remote_process
  • 17 Shiny
    • 17.1 Testing the Shiny Server
    • 17.2 Debugging Shiny Apps
    • 17.3 Checking Log Files
  • 18 Adding to PEcAn
    • 18.1 Adding An Ecosystem Model
      • 18.1.1 Interface Modules
    • 18.2 Example met conversion wrapper function
      • 18.2.1 Integrate MODEL into the PEcAn build system
      • 18.2.2 Add model info to PEcAn Database
      • 18.2.3 Define MODEL_TYPE
      • 18.2.4 MACHINE
      • 18.2.5 MODEL
      • 18.2.6 FORMATS
      • 18.2.7 MODEL_TYPE -> Formats
      • 18.2.8 INPUTS
      • 18.2.9 Add Plant Functional Types (PFTs)
      • 18.2.10 Adding Priors for Each Variable
    • 18.3 Adding input data
      • 18.3.1 Input records in BETY
      • 18.3.2 Create a database file record for the input data
      • 18.3.3 Creating a new Input record in BETY
      • 18.3.4 Adding a new input converter
    • 18.4 Pecan Data Ingest via Web Interface
      • 18.4.1 Loading Data
      • 18.4.2 Local Upload Example
      • 18.4.3 3. Creating a format record
    • 18.5 Creating a new format
      • 18.5.1 Formats in BETY
      • 18.5.2 Creating a new format in BETY
    • 18.6 Creating a new benchmark reference run
    • 18.7 Editing records
  • 19 Troubleshooting and Debugging PEcAn
    • General troubleshooting approach
    • 19.1 Common topics
    • 19.2 Cookies and pecan web pages
    • 19.3 Warning: mkdir() [function.mkdir]: No such file or directory
    • 19.4 After creating a new PFT the tag for PFT not passed to config.xml in ED
    • 19.5 Debugging
      • 19.5.1 Using tests/workflow.R
    • 19.6 Useful scripts
  • 20 BETY Database Administration
    • 20.1 Best practices
    • 20.2 Backup of BETY database
    • 20.3 Restore of BETY database
  • 21 Workflow modules
    • 21.1 Overview
    • 21.2 Load Settings
      • 21.2.1 read.settings("/home/pecan/pecan.xml")
    • 21.3 Query Database
      • 21.3.1 get.trait.data()
    • 21.4 Meta Analysis
      • 21.4.1 run.meta.analysis()
    • 21.5 Write Configuration Files
      • 21.5.1 write.configs(model)
    • 21.6 Start Runs
      • 21.6.1 start.runs(model)
    • 21.7 Get Model Output
      • 21.7.1 get.model.output(model)
    • 21.8 Ensemble Analysis
      • 21.8.1 run.ensemble.analysis()
    • 21.9 Sensitivity Analysis, Variance Decomposition
      • 21.9.1 run.sensitivity.analysis()
    • 21.10 Glossary
  • 22 Installation details
    • 22.1 AWS Setup
      • 22.1.1 Convert PEcAn VM
      • 22.1.2 Set up an account on AWS
      • 22.1.3 Install EC2 command line tools
      • 22.1.4 Create an AWS S3 ‘bucket’ to upload VM to
      • 22.1.5 Upload
      • 22.1.6 Configuring the VM
      • 22.1.7 Set up multiple instances (optional)
      • 22.1.8 Booting the VM
    • 22.2 Shiny Setup
      • 22.2.1 Install the Shiny R package and Shiny server
      • 22.2.2 Modify the shiny configuration file
      • 22.2.3 Set the Apache proxy
      • 22.2.4 Enable and start the shiny server, and restart apache
      • 22.2.5 Troubleshooting
      • 22.2.6 Further reading
    • 22.3 Thredds Setup
      • 22.3.1 Install the Tomcat 8 and Thredds webapp
      • 22.3.2 Update the catalog
      • 22.3.3 Troubleshooting
      • 22.3.4 Further reading
    • 22.4 OS Specific Installations
      • 22.4.1 Ubuntu
      • 22.4.2 CentOS/RedHat
      • 22.4.3 Mac OSX
      • 22.4.4 Installing BETY
    • 22.5 See Also
      • 22.5.1 Install Models
      • 22.5.2 Installing data for PEcAn
  • 23 Remote execution with PEcAn
    • 23.1 Basics of SSH
    • 23.2 SSH authentication – password vs. SSH key
    • 23.3 SSH tunneling
    • 23.4 SSH tunnels and PEcAn
    • 23.5 Basic remote execute functions
    • 23.6 Remote model execution with PEcAn
    • 23.7 XML configuration
    • 23.8 Configuration for PEcAn web interface
    • 23.9 Running PEcAn code for remotely
    • 23.10 Special case: geo.bu.edu
  • 24 Data assimilation with DART
  • IV Appendix
  • 25 Miscellaneous
    • 25.1 TODO
    • 25.2 Using the PEcAn download_file() function
  • 26 FAQ
  • 27 PEcAn Project Used in Courses
    • 27.1 University classes
      • 27.1.1 GE 375 - Environmental Modeling - Spring 2013, 2014 (Mike Dietze, Boston University)
    • 27.2 Summer Courses / Workshops
      • 27.2.1 Annual summer course in flux measurement and advanced modeling (Mike Dietze, Ankur Desai) Niwot Ridge, CO
      • 27.2.2 Assimilating Long-Term Data into Ecosystem Models: Paleo-Ecological Observatory Network (PalEON) Project
      • 27.2.3 Integrating Evidence on Forest Response to Climate Change: Physiology to Regional Abundance
      • 27.2.4 Ecological Society of America meetings
    • 27.3 Selected Publications
  • 28 Package Dependencies
    • 28.1 Executive Summary: What to usually do
    • 28.2 Big Picture: What’s possible to do
    • 28.3 Declaring Dependencies: Depends, Suggests, Imports
    • 28.4 Importing Functions: Use Roxygen
    • 28.5 Loading Code: Don’t… But Use requireNamespace When You Do
    • 28.6 Installing dependencies: Let the machines do it
  • 29 Testing with the testthat package
    • 29.1 List of Expectations
    • 29.2 Basic use of the testthat package
    • 29.3 Data for tests
    • 29.4 Settings
    • 29.5 Helper functions for unit tests
  • 30 devtools package
  • 31 singularity
  • 32 SiBCASA

The Predictive Ecosystem Analyzer

19.1 Common topics

  • Workflow failures
  • Database issues
  • Model errors
  • Docker-related problems
  • Debugging