34 Installing PEcAn

We recommend that new users download the PEcAn VM instead of creating from one from scratch. These instructions are provided to document how to install PEcAn on different Operating Systems.

34.1 Installing Prerequisites

Check specific notes in:

• VM Creation
• Installing PEcAn Data
• Enabling Remote Execution

OS specific installation instructions:

• Installing PEcAn Ubuntu
• Installing PEcAn OSX
• Installing PEcAn CentOS
• Installing PEcAn RedHat deprecated

To update PEcAn see:

• Updating PEcAn
• Updating BETY

The rest of the instructions assumes you have done the appropriate steps in each of the above guides.

34.1.1 Set R_LIBS_USER

CRAN Reference

# point R to personal lib folder
echo 'export R_LIBS_USER=${HOME}/R/library' >> ~/.profile
source ~/.profile
mkdir -p ${R_LIBS_USER}

34.2 Install models

34.2.1 ED2

34.2.1.1 ED2.2 r46 (used in PEcAn manuscript)

# ----------------------------------------------------------------------
# Get version r46 with a few patches for ubuntu
cd
curl -o ED.r46.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r46.tgz
tar zxf ED.r46.tgz
rm ED.r46.tgz
# ----------------------------------------------------------------------
# configure and compile ed
cd ~/ED.r46/ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.r46

Perform a test run using pre configured ED settings for ED2.2 r46

# ----------------------------------------------------------------------
# Create sample run
cd
mkdir testrun.ed.r46
cd testrun.ed.r46
curl -o ED2IN http://isda.ncsa.illinois.edu/~kooper/EBI/ED2IN.r46
sed -i -e "s#\$HOME#$HOME#" ED2IN
curl -o config.xml  http://isda.ncsa.illinois.edu/~kooper/EBI/config.r46.xml
# execute test run
time ed2.r46

34.2.1.2 ED 2.2 r82

cd
curl -o ED.r82.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r82.tgz
tar zxf ED.r82.tgz
rm ED.r82.tgz

cd ED.r82
curl -o ED.r82.patch http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r82.patch
patch -p1 < ED.r82.patch
cd ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.r82

Perform a test run using pre configured ED settings for ED2.2 r82

cd
mkdir testrun.ed.r82
cd testrun.ed.r82
curl -o ED2IN http://isda.ncsa.illinois.edu/~kooper/EBI/ED2IN.r82
sed -i -e "s#\$HOME#$HOME#" ED2IN
curl -o config.xml  http://isda.ncsa.illinois.edu/~kooper/EBI/config.r82.xml
# execute test run
time ed2.r82

34.2.1.3 ED 2.2 bleeding edge

cd
git clone https://github.com/EDmodel/ED2.git

cd ED2/ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
./generate_deps.sh
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.git

34.2.2 SIPNET

34.2.2.1 Sipnet Installation

cd
curl -o sipnet_unk.tar.gz http://isda.ncsa.illinois.edu/~kooper/EBI/sipnet_unk.tar.gz
tar zxf sipnet_unk.tar.gz
rm sipnet_unk.tar.gz

cd sipnet_unk
make
sudo cp sipnet /usr/local/bin/sipnet.runk

34.2.2.2 SIPNET testrun

cd
curl -o testrun.sipnet.tar.gz http://isda.ncsa.illinois.edu/~kooper/EBI/testrun.sipnet.tar.gz
tar zxf testrun.sipnet.tar.gz
rm testrun.sipnet.tar.gz
cd testrun.sipnet
sipnet.runk

34.2.3 BioCro

34.2.3.1 Installation

# Public
echo 'devtools::install_github("ebimodeling/biocro")' | R --vanilla
# Development:
echo 'devtools::install_github("ebimodeling/biocro-dev")' | R --vanilla

BioCro Developers: request from [@dlebauer on GitHub](https://github.com/dlebauer)

34.2.4 Linkages

34.2.4.1 Installation

# Public
echo 'devtools::install_github("araiho/linkages_package")' | R --vanilla

34.2.5 DALEC

34.2.5.1 Installation

cd
curl -o dalec_EnKF_pub.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/dalec_EnKF_pub.tgz
tar zxf dalec_EnKF_pub.tgz
rm dalec_EnKF_pub.tgz

cd dalec_EnKF_pub
make dalec_EnKF
make dalec_seqMH
sudo cp dalec_EnKF dalec_seqMH /usr/local/bin

34.2.6 LINKAGES

34.2.6.1 Installation

#FORTRAN VERSION
cd
git clone https://github.com/araiho/Linkages.git
cd Linkages
gfortran -o linkages linkages.f
sudo cp linkages /usr/local/bin/linkages.git

#R Version
git clone https://github.com/araiho/linkages_package.git
R CMD INSTALL --no-multiarch --with-keep.source linkages_package

34.2.7 CLM 4.5

The version of CLM installed on PEcAn is the ORNL branch provided by Dan Ricciuto. This version includes Dan’s point-level CLM processing scripts

Download the code (~300M compressed), input data (1.7GB compressed and expands to 14 GB), and a few misc inputs.

mkdir models
cd models
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm4_5_1_r085.tar.gz
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/ccsm_inputdata.tar.gz
tar -xvzf clm4_5*
tar -xvzf ccsm_inputdata.tar.gz

#Parameter file:
mkdir /home/carya/models/ccsm_inputdata/lnd/clm2/paramdata
cd  /home/carya/models/ccsm_inputdata/lnd/clm2/paramdata
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm_params.c130821.nc
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm_params.c140423.nc

#Domain file:
cd /home/carya/models/ccsm_inputdata/share/domains/domain.clm/
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/domain.lnd.1x1pt_US-UMB_navy.nc

#Aggregated met data file:
cd /home/carya/models/ccsm_inputdata/atm/datm7/CLM1PT_data/1x1pt_US-UMB
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/all_hourly.nc 

## lightning database
cd /home/carya/models/ccsm_inputdata/atm/datm7/NASA_LIS/
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clmforc.Li_2012_climo1995-2011.T62.lnfm_Total_c140423.nc

## surface data
cd /home/carya/models/ccsm_inputdata/lnd/clm2/surfdata
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/surfdata_360x720cru_simyr1850_c130927.nc
cd /home/carya/models/ccsm_inputdata/lnd/clm2/surfdata_map
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc_new
mv surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc_new surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc

Required libraries

sudo apt-get install mercurial csh tcsh subversion cmake

sudo ln -s /usr/bin/make /usr/bin/gmake

Compile and build default inputs

cd ~/carya/models/clm4_5_1_r085/scripts
python runCLM.py --site US-UMB ––compset I1850CLM45CN --mach ubuntu --ccsm_input /home/carya/models/ccsm_inputdata --tstep 1 --nopointdata --coldstart --cpl_bypass --clean_build

34.2.7.1 CLM Test Run

You will see a new directory in scripts: US-UMB_I1850CLM45CN Enter this directory and run (you shouldn’t have to do this normally, but there is a bug with the python script and doing this ensures all files get to the right place):

./US-UMB_I1850CLM45CN.build

Next you are ready to go to the run directory:

/home/carya/models/clm4_5_1_r085/run/US-UMB_I1850CLM45CN/run

Open to edit file: datm.streams.txt.CLM1PT.CLM_USRDAT and check file paths such that all paths start with /home/carya/models/ccsm_inputdata

From this directory, launch the executable that resides in the bld directory:

/home/carya/clm4_5_1_r085/run/US-UMB_I1850CLM45CN/bld/cesm.exe

not sure this was the right location, but wherever the executable is

You should begin to see output files that look like this: US-UMB_I1850CLM45CN.clm2.h0.yyyy-mm.nc (yyyy is year, mm is month) These are netcdf files containing monthly averages of lots of variables.

The lnd_in file in the run directory can be modified to change the output file frequency and variables.

34.3 CLM-FATES

Prerequisites

sudo apt-get upgrade libnetcdf-dev
sudo apt-get install subversion
sudo apt-get install csh
sudo apt-get install cmake
sudo ln -s /usr/bin/make /usr/bin/gmake
sudo rm /bin/sh
sudo ln -s /bin/bash /bin/sh

wget https://github.com/Unidata/netcdf-fortran/archive/v4.4.4.tar.gz
cd netcdf-4.4.4
./configure
make
sudo make install

you might need to mess around with installing netcdf and netcdf-fortran to get a version FATES likes…

Get code from Github (currently private) and go to cime/scripts directory

git clone git@github.com:NGEET/ed-clm.git
cd ed-clm/cime/scripts/

Within CLM-FATES, to be able to build an executable we need to create a reference run. We’ll also use this reference run to grab defaults from, so we’ll be registering the location of both the reference case (location of executable, scripts, etc) and the reference inputs with the PEcAn database. To begin, copy reference run script from pecan

cp ~/pecan/models/fates/inst/create_1x1_ref_case.sh .

Edit reference case script to set NETCDF_HOME, CROOT (reference run case), DIN_LOC_ROOT (reference run inputs). Also, make sure DIN_LOC_ROOT exists as FATES will not create it itself. Then run the script

./create_1x1_ref_case.sh

Be aware that this script WILL ask you for your password on the NCAR server to download the reference case input data (the guest password may work, haven’t tried this). If it gives an error at the pio stage check the log, but the most likely error is it being unable to find a version of netcdf it likes.

Once FATES is installed, set the whole reference case directory as the Model path (leave filename blank) and set the whole inputs directory as an Input with format clm_defaults.

34.4 PEcAn Installation

PEcAn is the software package that ties all the pieces together.

34.5 Installing PEcAn

Download, compile and install PEcAn

# download pecan
cd
git clone https://github.com/PecanProject/pecan.git

# compile pecan
cd pecan
./scripts/build.sh --dependencies

Following will run a small script to setup some hooks to prevent people from using the pecan demo user account to check in any code.

# prevent pecan user from checking in code
./scripts/create-hooks.sh

34.6 Installing BETY

34.6.1 Install Postgres

See OS-specific instructions for installing Postgres + PostGIS * Ubuntu * OSX * RedHat / CentOS

34.6.2 Install Database + Data

• note To install BETYdb without PEcAn, first download the load.bety.sh script

# install database (code assumes password is bety)
sudo -u postgres createuser -d -l -P -R -S bety
sudo -u postgres createdb -O bety bety
sudo -u postgres ./scripts/load.bety.sh -c YES -u YES -r 0
sudo -u postgres ./scripts/load.bety.sh -r 1
sudo -u postgres ./scripts/load.bety.sh -r 2

# configure for PEcAn web app (change password if needed)
cp web/config.example.php web/config.php 

# add models to database (VM only)
./scripts/add.models.sh

# add data to database
./scripts/add.data.sh

# create outputs folder
mkdir ~/output
chmod 777 ~/output

34.6.3 Installing BETYdb Web Application

There are two flavors of BETY, PHP and RUBY. The PHP version allows for a minimal interaction with the database while the RUBY version allows for full interaction with the database.

34.6.3.1 PHP version

The php version comes with PEcAn and is already configured.

34.6.3.2 RUBY version

The RUBY version requires a few extra packages to be installed first.

Next we install the web app.

# install bety
cd
git clone https://github.com/PecanProject/bety.git

# install gems
cd bety
sudo gem2.0 install bundler
bundle install --without development:test:javascript_testing:debug

and configure BETY

# create folders for upload folders
mkdir paperclip/files paperclip/file_names
chmod 777 paperclip/files paperclip/file_names

# create folder for log files
mkdir log
touch log/production.log
chmod 0666 log/production.log

# fix configuration for vm
cp config/additional_environment_vm.rb config/additional_environment.rb
chmod go+w public/javascripts/cache/

# setup bety database configuration
cat > config/database.yml << EOF
production:
  adapter: postgis
  encoding: utf-8
  reconnect: false
  database: bety
  pool: 5
  username: bety
  password: bety
EOF

# setup login tokens
cat > config/initializers/site_keys.rb << EOF
REST_AUTH_SITE_KEY         = 'thisisnotasecret'
REST_AUTH_DIGEST_STRETCHES = 10
EOF

34.6.4 PEcAn Testrun

Do the run, this assumes you have installed the BETY database, sites tar file and sipnet.

# create folder
cd
mkdir testrun.pecan
cd testrun.pecan

# copy example of pecan workflow and configuration file
cp ../pecan/tests/pecan32.sipnet.xml pecan.xml
cp ../pecan/scripts/workflow.R workflow.R

# exectute workflow
rm -rf pecan
./workflow.R pecan.xml

NB: pecan.xml is configured for the virtual machine, you will need to change the field from ‘/home/carya/’ to wherever you installed your ‘sites’, usually $HOME