23.5 See Also

23.5.0.1 Install Database + Data

# install database (code assumes password is bety)
sudo -u postgres createuser -d -l -P -R -S bety
sudo -u postgres createdb -O bety bety
sudo -u postgres ./scripts/load.bety.sh -c YES -u YES -r 0
sudo -u postgres ./scripts/load.bety.sh -r 1
sudo -u postgres ./scripts/load.bety.sh -r 2

# configure for PEcAn web app (change password if needed)
cp web/config.example.php web/config.php 

# add models to database (VM only)
./scripts/add.models.sh

# add data to database
./scripts/add.data.sh

# create outputs folder
mkdir ~/output
chmod 777 ~/output

23.5.0.2 Installing BETYdb Web Application

⚠️ Legacy (Deprecated)

This section documents manual PHP and Ruby-based BETYdb web application installations. These workflows are deprecated and are not recommended for new installations. New users should use the Docker-based BETY deployment described in the PEcAn Docker section or consult the official BETY documentation: https://pecan.gitbook.io/betydb-documentation

There are two flavors of BETY, PHP and RUBY. The PHP version allows for a minimal interaction with the database while the RUBY version allows for full interaction with the database.

23.5.0.2.1 PHP version

⚠️ This section describes a deprecated manual installation path. > > This PHP-based BETYdb web interface is part of a legacy workflow. > New installations should use the Docker-based BETY deployment instead.

The php version comes with PEcAn and is already configured.

23.5.0.2.2 RUBY version

⚠️ This section describes a deprecated manual installation path. > > This Ruby-based BETYdb web application setup is a legacy, manual workflow. > It is retained for existing deployments and is not recommended for new installations.

The RUBY version requires a few extra packages to be installed first.

Next we install the web app.

# install bety
cd
git clone https://github.com/PecanProject/bety.git

# install gems
cd bety
sudo gem2.0 install bundler
bundle install --without development:test:javascript_testing:debug

and configure BETY

# create folders for upload folders
mkdir paperclip/files paperclip/file_names
chmod 777 paperclip/files paperclip/file_names

# create folder for log files
mkdir log
touch log/production.log
chmod 0666 log/production.log

# fix configuration for vm
cp config/additional_environment_vm.rb config/additional_environment.rb
chmod go+w public/javascripts/cache/

# setup bety database configuration
cat > config/database.yml << EOF
production:
  adapter: postgis
  encoding: utf-8
  reconnect: false
  database: bety
  pool: 5
  username: bety
  password: bety
EOF

# setup login tokens
cat > config/initializers/site_keys.rb << EOF
REST_AUTH_SITE_KEY         = 'thisisnotasecret'
REST_AUTH_DIGEST_STRETCHES = 10
EOF

23.5.1 Install Models

This page contains instructions on how to download and install ecosystem models that have been or are being coupled to PEcAn. These instructions have been tested on the PEcAn unbuntu VM. Commands may vary on other operating systems. Also, some model downloads require permissions before downloading, making them unavailable to the general public. Please contact the PEcAn team if you would like access to a model that is not already installed on the default PEcAn VM.

23.5.1.1 BioCro

# Version 0.951
echo 'devtools::install_github("ebimodeling/biocro@0.951")' | R --vanilla
# Version 1.x 
echo 'devtools::install_github("ebimodeling/biocro-dev@26e0125b")' | R --vanilla

BioCro Developers: request access from @dlebauer on GitHub

23.5.1.2 CLM 4.5

The version of CLM installed on PEcAn is the ORNL branch provided by Dan Ricciuto. This version includes Dan’s point-level CLM processing scripts

Download the code (~300M compressed), input data (1.7GB compressed and expands to 14 GB), and a few misc inputs.

mkdir models
cd models
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm4_5_1_r085.tar.gz
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/ccsm_inputdata.tar.gz
tar -xvzf clm4_5*
tar -xvzf ccsm_inputdata.tar.gz

#Parameter file:
mkdir /home/carya/models/ccsm_inputdata/lnd/clm2/paramdata
cd  /home/carya/models/ccsm_inputdata/lnd/clm2/paramdata
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm_params.c130821.nc
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm_params.c140423.nc

#Domain file:
cd /home/carya/models/ccsm_inputdata/share/domains/domain.clm/
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/domain.lnd.1x1pt_US-UMB_navy.nc

#Aggregated met data file:
cd /home/carya/models/ccsm_inputdata/atm/datm7/CLM1PT_data/1x1pt_US-UMB
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/all_hourly.nc 

## lightning database
cd /home/carya/models/ccsm_inputdata/atm/datm7/NASA_LIS/
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clmforc.Li_2012_climo1995-2011.T62.lnfm_Total_c140423.nc

## surface data
cd /home/carya/models/ccsm_inputdata/lnd/clm2/surfdata
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/surfdata_360x720cru_simyr1850_c130927.nc
cd /home/carya/models/ccsm_inputdata/lnd/clm2/surfdata_map
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc_new
mv surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc_new surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc

Required libraries

sudo apt-get install mercurial csh tcsh subversion cmake

sudo ln -s /usr/bin/make /usr/bin/gmake

Compile and build default inputs

cd ~/carya/models/clm4_5_1_r085/scripts
python runCLM.py --site US-UMB ––compset I1850CLM45CN --mach ubuntu --ccsm_input /home/carya/models/ccsm_inputdata --tstep 1 --nopointdata --coldstart --cpl_bypass --clean_build
23.5.1.2.1 CLM Test Run

You will see a new directory in scripts: US-UMB_I1850CLM45CN Enter this directory and run (you shouldn’t have to do this normally, but there is a bug with the python script and doing this ensures all files get to the right place):

./US-UMB_I1850CLM45CN.build

Next you are ready to go to the run directory:

/home/carya/models/clm4_5_1_r085/run/US-UMB_I1850CLM45CN/run

Open to edit file: datm.streams.txt.CLM1PT.CLM_USRDAT and check file paths such that all paths start with /home/carya/models/ccsm_inputdata

From this directory, launch the executable that resides in the bld directory:

/home/carya/clm4_5_1_r085/run/US-UMB_I1850CLM45CN/bld/cesm.exe

not sure this was the right location, but wherever the executable is

You should begin to see output files that look like this: US-UMB_I1850CLM45CN.clm2.h0.yyyy-mm.nc (yyyy is year, mm is month) These are netcdf files containing monthly averages of lots of variables.

The lnd_in file in the run directory can be modified to change the output file frequency and variables.

23.5.1.3 DALEC

cd
curl -o dalec_EnKF_pub.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/dalec_EnKF_pub.tgz
tar zxf dalec_EnKF_pub.tgz
rm dalec_EnKF_pub.tgz

cd dalec_EnKF_pub
make dalec_EnKF
make dalec_seqMH
sudo cp dalec_EnKF dalec_seqMH /usr/local/bin

23.5.1.4 ED2

23.5.1.4.1 ED2.2 r46 (used in PEcAn manuscript)
# ----------------------------------------------------------------------
# Get version r46 with a few patches for ubuntu
cd
curl -o ED.r46.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r46.tgz
tar zxf ED.r46.tgz
rm ED.r46.tgz
# ----------------------------------------------------------------------
# configure and compile ed
cd ~/ED.r46/ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.r46

Perform a test run using pre configured ED settings for ED2.2 r46

# ----------------------------------------------------------------------
# Create sample run
cd
mkdir testrun.ed.r46
cd testrun.ed.r46
curl -o ED2IN http://isda.ncsa.illinois.edu/~kooper/EBI/ED2IN.r46
sed -i -e "s#\$HOME#$HOME#" ED2IN
curl -o config.xml  http://isda.ncsa.illinois.edu/~kooper/EBI/config.r46.xml
# execute test run
time ed2.r46
23.5.1.4.2 ED 2.2 r82
cd
curl -o ED.r82.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r82.tgz
tar zxf ED.r82.tgz
rm ED.r82.tgz

cd ED.r82
curl -o ED.r82.patch http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r82.patch
patch -p1 < ED.r82.patch
cd ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.r82

Perform a test run using pre configured ED settings for ED2.2 r82

cd
mkdir testrun.ed.r82
cd testrun.ed.r82
curl -o ED2IN http://isda.ncsa.illinois.edu/~kooper/EBI/ED2IN.r82
sed -i -e "s#\$HOME#$HOME#" ED2IN
curl -o config.xml  http://isda.ncsa.illinois.edu/~kooper/EBI/config.r82.xml
# execute test run
time ed2.r82
23.5.1.4.3 ED 2.2 bleeding edge
cd
git clone https://github.com/EDmodel/ED2.git

cd ED2/ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
./generate_deps.sh
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.git

23.5.1.5 CLM-FATES

Prerequisites

sudo apt-get upgrade libnetcdf-dev
sudo apt-get install subversion
sudo apt-get install csh
sudo apt-get install cmake
sudo ln -s /usr/bin/make /usr/bin/gmake
sudo rm /bin/sh
sudo ln -s /bin/bash /bin/sh

wget https://github.com/Unidata/netcdf-fortran/archive/v4.4.4.tar.gz
cd netcdf-4.4.4
./configure
make
sudo make install

you might need to mess around with installing netcdf and netcdf-fortran to get a version FATES likes…

Get code from Github (currently private) and go to cime/scripts directory

git clone git@github.com:NGEET/ed-clm.git
cd ed-clm/cime/scripts/

Within CLM-FATES, to be able to build an executable we need to create a reference run. We’ll also use this reference run to grab defaults from, so we’ll be registering the location of both the reference case (location of executable, scripts, etc) and the reference inputs with the PEcAn database. To begin, copy reference run script from pecan

cp ~/pecan/models/fates/inst/create_1x1_ref_case.sh .

Edit reference case script to set NETCDF_HOME, CROOT (reference run case), DIN_LOC_ROOT (reference run inputs). Also, make sure DIN_LOC_ROOT exists as FATES will not create it itself. Then run the script

./create_1x1_ref_case.sh

Be aware that this script WILL ask you for your password on the NCAR server to download the reference case input data (the guest password may work, haven’t tried this). If it gives an error at the pio stage check the log, but the most likely error is it being unable to find a version of netcdf it likes.

Once FATES is installed, set the whole reference case directory as the Model path (leave filename blank) and set the whole inputs directory as an Input with format clm_defaults.

23.5.1.6 GDAY

Navigate to a directory you would like to store GDAY and run the following:

git clone https://github.com/mdekauwe/GDAY.git

cd GDAY

cd src

make

gday is your executable.

23.5.1.7 JULES

INSTALL STEPS: 1) Download JULES and FCM JULES: Model requires registration to download. Not to be put on PEcAn VM Getting Started documentation: https://jules.jchmr.org/content/getting-started Registration: http://jules-lsm.github.io/access_req/JULES_access.html

FCM:

    https://github.com/metomi/fcm/
    wget https://github.com/metomi/fcm/archive/2015.05.0.tar.gz
  1. edit makefile
open etc/fcm-make/make.cfg

set JULES_NETCDF = actual instead of dummy
set path (e.g. /usr/) and lib_path /lib64 to netCDF libraries
  1. compile JULES
cd etc/fcm-make/
{path.to.fcm}/fcm make -f etc/fcm-make/make.cfg --new
UBUNTU VERSION: installed without having to add any perl libraries
#perl stuff that I had to install on pecan2 not PEcAN VM
sudo yum install perl-Digest-SHA
sudo yum install perl-Time-modules
sudo yum install cpan
curl -L http://cpanmin.us | perl - --sudo App::cpanminus
sudo cpanm Time/Piece.pm
sudo cpanm IO/Uncompress/Gunzip.pm

Executable is under build/bin/jules.exe

Example rundir: examples/point_loobos

23.5.1.8 LINKAGES

23.5.1.8.1 R Installation
# Public
echo 'devtools::install_github("araiho/linkages_package")' | R --vanilla
23.5.1.8.2 FORTRAN VERSION
#FORTRAN VERSION
cd
git clone https://github.com/araiho/Linkages.git
cd Linkages
gfortran -o linkages linkages.f
sudo cp linkages /usr/local/bin/linkages.git

23.5.1.9 LPJ-GUESS

Instructions to download source code

Go to LPJ-GUESS website for instructions to access code.

23.5.1.10 MAESPA

Navigate to a directory you would like store MAESPA and run the following:

git clone https://bitbucket.org/remkoduursma/maespa.git

cd maespa

make

maespa.out is your executable. Example input files can be found in the inpufiles directory. Executing measpa.out from within one of the example directories will produce output.

MAESPA developers have also developed a wrapper package called Maeswrap. The usual R package installation method install.packages may present issues with downloading an unpacking a dependency package called rgl. Here are a couple of solutions:

23.5.1.10.1 Solution 1
### From the Command Line
sudo apt-get install r-cran-rgl

then from within R

install.packages("Maeswrap")
23.5.1.10.2 Solution 2
### From the Command line 
sudo apt-get install libglu1-mesa-dev

then from within R

install.packages("Maeswrap")

23.5.1.11 SIPNET {inst-sipnet}

cd
curl -o sipnet_unk.tar.gz http://isda.ncsa.illinois.edu/~kooper/EBI/sipnet_unk.tar.gz
tar zxf sipnet_unk.tar.gz
rm sipnet_unk.tar.gz

cd sipnet_unk
make
sudo cp sipnet /usr/local/bin/sipnet.runk
23.5.1.11.1 SIPNET testrun
cd
curl -o testrun.sipnet.tar.gz http://isda.ncsa.illinois.edu/~kooper/EBI/testrun.sipnet.tar.gz
tar zxf testrun.sipnet.tar.gz
rm testrun.sipnet.tar.gz
cd testrun.sipnet
sipnet.runk

23.5.2 Installing data for PEcAn

PEcAn assumes some of the data to be installed on the machine. This page will describe how to install this data.

23.5.2.1 Site Information

These are large-ish files that contain data used with ED2 and SIPNET

rm -rf sites
curl -o sites.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/sites.tgz
tar zxf sites.tgz
sed -i -e "s#/home/kooper/Projects/EBI#${PWD}#" sites/*/ED_MET_DRIVER_HEADER
rm sites.tgz

rm -rf inputs
curl -o inputs.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/inputs.tgz
tar zxf inputs.tgz
rm inputs.tgz

23.5.2.2 FIA database

FIA database is large and will add an extra 10GB to the installation.

# download and install database
curl -o fia5data.psql.gz http://isda.ncsa.illinois.edu/~kooper/EBI/fia5data.psql.gz
dropdb --if-exists fia5data
createdb -O bety fia5data
gunzip fia5data.psql.gz
psql -U bety -d fia5data < fia5data.psql
rm fia5data.psql

23.5.2.3 Flux Camp

Following will install the data for flux camp (as well as the demo script for PEcAn).

cd
curl -o plot.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/plot.tgz
tar zxf plot.tgz
rm plot.tgz

23.5.2.4 Harvard for ED tutorial

Add datasets and runs

curl -o Santarem_Km83.zip http://isda.ncsa.illinois.edu/~kooper/EBI/Santarem_Km83.zip
unzip -d sites Santarem_Km83.zip
sed -i -e "s#/home/pecan#${HOME}#" sites/Santarem_Km83/ED_MET_DRIVER_HEADER
rm Santarem_Km83.zip

curl -o testrun.s83.zip http://isda.ncsa.illinois.edu/~kooper/EBI/testrun.s83.zip
unzip testrun.s83.zip
sed -i -e "s#/home/pecan#${HOME}#" testrun.s83/ED2IN
rm testrun.s83.zip

curl -o ed2ws.harvard.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/ed2ws.harvard.tgz
tar zxf ed2ws.harvard.tgz
mkdir ed2ws.harvard/analy ed2ws.harvard/histo
sed -i -e "s#/home/pecan#${HOME}#g" ed2ws.harvard/input_harvard/met_driver/HF_MET_HEADER ed2ws.harvard/ED2IN ed2ws.harvard/*.r
rm ed2ws.harvard.tgz

curl -o testrun.PDG.zip http://isda.ncsa.illinois.edu/~kooper/EBI/testrun.PDG.zip
unzip testrun.PDG.zip
sed -i -e "s#/home/pecan#${HOME}#" testrun.PDG/Met/PDG_MET_DRIVER testrun.PDG/Template/ED2IN
sed -i -e 's#/n/scratch2/moorcroft_lab/kzhang/PDG/WFire_Pecan/##' testrun.PDG/Template/ED2IN
rm testrun.PDG.zip

curl -o create_met_driver.tar.gz http://isda.ncsa.illinois.edu/~kooper/EBI/create_met_driver.tar.gz
tar zxf create_met_driver.tar.gz
rm create_met_driver.tar.gz