22 PEcAn Setup

22.1 Installing the PEcAn VM

We recommend that new users download the PEcAn VM instead of creating from one from scratch.

Additional Setup - AWS - Shiny - Thredds

22.2 Install PEcAn by hand

These instructions are provided to document how to install PEcAn on different Operating Systems.

Prerequisites
OS specific installations
Install BETY
Install models
Clone and Compile PEcAn source code
Test Run

22.2.1 Installation Prerequisites

Please review the following topics before starting PEcAn installation

22.2.1.1 Creating a Virtual Machine

First create virtual machine

# ----------------------------------------------------------------------
# CREATE VM USING FOLLOWING:
# - VM NAME  = PEcAn
# - CPU      = 2
# - MEMORY   = 2GB 
# - DISK     = 100GB
# - HOSTNAME = pecan
# - FULLNAME = PEcAn Demo User
# - USERNAME = xxxxxxx
# - PASSWORD = yyyyyyy
# - PACKAGE  = openssh
# ----------------------------------------------------------------------

To enable tunnels run the following on the host machine:

VBoxManage modifyvm "PEcAn" --natpf1 "ssh,tcp,,6422,,22"
VBoxManage modifyvm "PEcAn" --natpf1 "www,tcp,,6480,,80"

Make sure machine is up to date.

UBUNTU

sudo apt-get update
sudo apt-get -y dist-upgrade
sudo reboot

CENTOS/REDHAT

sudo yum -y update
sudo reboot

Install compiler and other packages needed and install the tools.

UBUNTU

sudo apt-get -y install build-essential linux-headers-server dkms

CENTOS/REDHAT

sudo yum -y groupinstall "Development Tools"
sudo yum -y install wget

Install Virtual Box additions for better integration

sudo mount /dev/cdrom /mnt
sudo /mnt/VBoxLinuxAdditions.run
sudo umount /mnt
sudo usermod -a -G vboxsf carya

Finishing up the machine

Add a message to the login:

sudo -s
export PORT=$( hostname | sed 's/pecan//' )
cat > /etc/motd << EOF
PEcAn version 1.4.3

For more information about:
Pecan    - http://pecanproject.org
BETY     - http://www.betydb.org

For a list of all models currently navigate [here](../users_guide/basic_users_guide/models_table.md)


You can access this system using a webbrowser at
 http://<hosting machine>:${PORT}80/
or using SSH at
 ssh -l carya -p ${PORT}22 <hosting machine>
where <hosting machine> is the machine where the VM runs on.
EOF
exit

Finishing up

Script to clean the VM and remove as much as possible history cleanvm.sh

wget -O ~/cleanvm.sh http://isda.ncsa.uiuc.edu/~kooper/EBI/cleanvm.sh
chmod 755 ~/cleanvm.sh

Make sure machine has SSH keys rc.local

sudo wget -O /etc/rc.local http://isda.ncsa.illinois.edu/~kooper/EBI/rc.local

Change the resolution of the console

sudo sed -i -e 's/#GRUB_GFXMODE=640x480/GRUB_GFXMODE=1024x768/' /etc/default/grub
sudo update-grub

Once all done, stop the virtual machine

history -c && ${HOME}/cleanvm.sh

22.2.1.2 Installing data for PEcAn

PEcAn assumes some of the data to be installed on the machine. This page will describe how to install this data.

22.2.1.2.1 Site Information

These are large-ish files that contain data used with ED2 and SIPNET

rm -rf sites
curl -o sites.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/sites.tgz
tar zxf sites.tgz
sed -i -e "s#/home/kooper/Projects/EBI#${PWD}#" sites/*/ED_MET_DRIVER_HEADER
rm sites.tgz

rm -rf inputs
curl -o inputs.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/inputs.tgz
tar zxf inputs.tgz
rm inputs.tgz

22.2.1.2.2 FIA database

FIA database is large and will add an extra 10GB to the installation.

# download and install database
curl -o fia5data.psql.gz http://isda.ncsa.illinois.edu/~kooper/EBI/fia5data.psql.gz
dropdb --if-exists fia5data
createdb -O bety fia5data
gunzip fia5data.psql.gz
psql -U bety -d fia5data < fia5data.psql
rm fia5data.psql

22.2.1.2.3 Flux Camp

Following will install the data for flux camp (as well as the demo script for PEcAn).

cd
curl -o plot.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/plot.tgz
tar zxf plot.tgz
rm plot.tgz

22.2.1.2.4 Harvard for ED tutorial

Add datasets and runs

curl -o Santarem_Km83.zip http://isda.ncsa.illinois.edu/~kooper/EBI/Santarem_Km83.zip
unzip -d sites Santarem_Km83.zip
sed -i -e "s#/home/pecan#${HOME}#" sites/Santarem_Km83/ED_MET_DRIVER_HEADER
rm Santarem_Km83.zip

curl -o testrun.s83.zip http://isda.ncsa.illinois.edu/~kooper/EBI/testrun.s83.zip
unzip testrun.s83.zip
sed -i -e "s#/home/pecan#${HOME}#" testrun.s83/ED2IN
rm testrun.s83.zip

curl -o ed2ws.harvard.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/ed2ws.harvard.tgz
tar zxf ed2ws.harvard.tgz
mkdir ed2ws.harvard/analy ed2ws.harvard/histo
sed -i -e "s#/home/pecan#${HOME}#g" ed2ws.harvard/input_harvard/met_driver/HF_MET_HEADER ed2ws.harvard/ED2IN ed2ws.harvard/*.r
rm ed2ws.harvard.tgz

curl -o testrun.PDG.zip http://isda.ncsa.illinois.edu/~kooper/EBI/testrun.PDG.zip
unzip testrun.PDG.zip
sed -i -e "s#/home/pecan#${HOME}#" testrun.PDG/Met/PDG_MET_DRIVER testrun.PDG/Template/ED2IN
sed -i -e 's#/n/scratch2/moorcroft_lab/kzhang/PDG/WFire_Pecan/##' testrun.PDG/Template/ED2IN
rm testrun.PDG.zip

curl -o create_met_driver.tar.gz http://isda.ncsa.illinois.edu/~kooper/EBI/create_met_driver.tar.gz
tar zxf create_met_driver.tar.gz
rm create_met_driver.tar.gz

22.2.2 OS Specific Installations

Ubuntu
CentOS
OSX
RedHat Check this vs. CentOS … ?

22.2.2.1 Ubuntu

These are specific notes for installing PEcAn on Ubuntu (14.04) and will be referenced from the main installing PEcAn page. You will at least need to install the build environment and Postgres sections. If you want to access the database/PEcAn using a web browser you will need to install Apache. To access the database using the BETY interface, you will need to have Ruby installed.

This document also contains information on how to install the Rstudio server edition as well as any other packages that can be helpful.

22.2.2.1.1 Install build environment

sudo -s

# point to latest R
echo "deb http://cran.rstudio.com/bin/linux/ubuntu `lsb_release -s -c`/" > /etc/apt/sources.list.d/R.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9

# update package list
apt-get -y update

# install packages needed for PEcAn
apt-get -y install build-essential gfortran git r-base-core r-base-dev jags liblapack-dev libnetcdf-dev netcdf-bin bc libcurl4-gnutls-dev curl udunits-bin libudunits2-dev libgmp-dev python-dev libgdal1-dev libproj-dev expect

# install packages needed for ED2
apt-get -y install openmpi-bin libopenmpi-dev

# install requirements for DALEC
apt-get -y install libgsl0-dev

# install packages for webserver
apt-get -y install apache2 libapache2-mod-php5 php5

# install packages to compile docs
apt-get -y install texinfo texlive-latex-base texlive-latex-extra texlive-fonts-recommended

# install devtools
echo 'install.packages("devtools", repos="http://cran.rstudio.com/")' | R --vanilla

# done as root
exit

22.2.2.1.2 Install Postgres

Documentation: http://trac.osgeo.org/postgis/wiki/UsersWikiPostGIS21UbuntuPGSQL93Apt

sudo -s

# point to latest PostgreSQL
echo "deb http://apt.postgresql.org/pub/repos/apt `lsb_release -s -c`-pgdg main" > /etc/apt/sources.list.d/pgdg.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -

# update package list
apt-get -y update

# install packages for postgresql (using a newer version than default)
apt-get -y install libdbd-pgsql postgresql postgresql-client libpq-dev postgresql-9.4-postgis-2.1 postgresql-9.4-postgis-2.1-scripts

# install following if you want to run pecan through the web
apt-get -y install php5-pgsql

# enable bety user to login with trust by adding the following lines after
# the ability of postgres user to login in /etc/postgresql/9.4/main/pg_hba.conf
local   all             bety                                    trust
host    all             bety            127.0.0.1/32            trust
host    all             bety            ::1/128                 trust

# Once done restart postgresql
/etc/init.d/postgresql restart

exit

To install the BETYdb database .. ##### Apache Configuration PEcAn

# become root
sudo -s

# get index page
rm /var/www/html/index.html
ln -s ${HOME}/pecan/documentation/index_vm.html /var/www/html/index.html

# setup a redirect
cat > /etc/apache2/conf-available/pecan.conf << EOF
Alias /pecan ${HOME}/pecan/web
<Directory ${HOME}/pecan/web>
  DirectoryIndex index.php
  Options +ExecCGI
  Require all granted
</Directory>
EOF
a2enconf pecan
/etc/init.d/apache2 restart

# done as root
exit

22.2.2.1.3 Apache Configuration BETY

sudo -s

# install all ruby related packages
apt-get -y install ruby2.0 ruby2.0-dev libapache2-mod-passenger 

# link static content
ln -s ${HOME}/bety/public /var/www/html/bety

# setup a redirect
cat > /etc/apache2/conf-available/bety.conf << EOF
RailsEnv production
RailsBaseURI /bety
PassengerRuby /usr/bin/ruby2.0
<Directory /var/www/html/bety>
  Options +FollowSymLinks
  Require all granted
</Directory>
EOF
a2enconf bety
/etc/init.d/apache2 restart

22.2.2.1.4 Rstudio-server

NOTE This will allow anybody to login to the machine through the rstudio interface and run any arbitrary code. The login used however is the same as the system login/password.

Based on version of ubuntu 32/64 use either of the following

32bit only

wget http://download2.rstudio.org/rstudio-server-0.98.1103-i386.deb

64bit only

wget http://download2.rstudio.org/rstudio-server-0.98.1103-amd64.deb

# bceome root
sudo -s

# install required packages
apt-get -y install libapparmor1 apparmor-utils libssl0.9.8

# install rstudio
dpkg -i rstudio-server-*
rm rstudio-server-*
echo "www-address=127.0.0.1" >> /etc/rstudio/rserver.conf
echo "r-libs-user=~/R/library" >> /etc/rstudio/rsession.conf
rstudio-server restart

# setup rstudio forwarding in apache
a2enmod proxy_http
cat > /etc/apache2/conf-available/rstudio.conf << EOF
ProxyPass        /rstudio/ http://localhost:8787/
ProxyPassReverse /rstudio/ http://localhost:8787/
RedirectMatch permanent ^/rstudio$ /rstudio/
EOF
a2enconf rstudio
/etc/init.d/apache2 restart

# all done, exit root
exit

22.2.2.1.5 Additional packages

HDF5 Tools, netcdf, GDB and emacs

sudo apt-get -y install hdf5-tools cdo nco netcdf-bin ncview gdb emacs ess nedit

22.2.2.2 CentOS/RedHat

These are specific notes for installing PEcAn on CentOS (7) and will be referenced from the main installing PEcAn page. You will at least need to install the build environment and Postgres sections. If you want to access the database/PEcAn using a web browser you will need to install Apache. To access the database using the BETY interface, you will need to have Ruby installed.

This document also contains information on how to install the Rstudio server edition as well as any other packages that can be helpful.

22.2.2.2.1 Install build environment

sudo -s

# install packages needed for PEcAn
yum -y groupinstall 'Development Tools' 
yum -y install git netcdf-fortran-openmpi-devel R bc curl libxml2-devel openssl-devel ed  udunits2 udunits2-devel netcdf netcdf-devel gmp-devel python-devel gdal-devel proj-devel proj-epsg expect

# jags
yum -y install http://download.opensuse.org/repositories/home:/cornell_vrdc/CentOS_7/x86_64/jags3-3.4.0-54.1.x86_64.rpm
yum -y install http://download.opensuse.org/repositories/home:/cornell_vrdc/CentOS_7/x86_64/jags3-devel-3.4.0-54.1.x86_64.rpm

# fix include folder for udunits2
ln -s /usr/include/udunits2/* /usr/include/

# install packages needed for ED2
yum -y install environment-modules openmpi-bin libopenmpi-dev

# install requirements for DALEC
yum -y install gsl-devel

# install packages for webserver
yum -y install httpd php
systemctl enable httpd
systemctl start httpd
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --reload

# install packages to compile docs
#apt-get -y install texinfo texlive-latex-base texlive-latex-extra texlive-fonts-recommended

# install devtools
echo 'install.packages("devtools", repos="http://cran.rstudio.com/")' | R --vanilla

# done as root
exit

echo "module load mpi" >> ~/.bashrc
module load mpi

22.2.2.2.2 Install Postgres

sudo -s

# point to latest PostgreSQL
yum install -y epel-release 
yum -y install http://yum.postgresql.org/9.4/redhat/rhel-7-x86_64/pgdg-centos94-9.4-1.noarch.rpm

# install packages for postgresql (using a newer version than default)
yum -y install postgresql94-server postgresql94-contrib postgis2_94 postgresql94-devel

# install following if you want to run pecan through the web
yum -y install php-pgsql

# enable bety user to login with trust by adding the following lines after
# the ability of postgres user to login in /var/lib/pgsql/9.4/data/pg_hba.conf
local   all             bety                                    trust
host    all             bety            127.0.0.1/32            trust
host    all             bety            ::1/128                 trust

# Create database
/usr/pgsql-9.4/bin/postgresql94-setup initdb

# Enable postgres
systemctl enable postgresql-9.4
systemctl start postgresql-9.4

exit

22.2.2.2.3 Apache Configuration PEcAn

# become root
sudo -s

# get index page
rm /var/www/html/index.html
ln -s /home/carya/pecan/documentation/index_vm.html /var/www/html/index.html

# fix selinux context (does this need to be done after PEcAn is installed?)
chcon -R -t httpd_sys_content_t /home/carya/pecan /home/carya/output

# setup a redirect
cat > /etc/httpd/conf.d/pecan.conf << EOF
Alias /pecan /home/carya/pecan/web
<Directory /home/carya/pecan/web>
  DirectoryIndex index.php
  Options +ExecCGI
  Require all granted
</Directory>
EOF
a2enconf pecan
/etc/init.d/apache2 restart

# done as root
exit

22.2.2.2.4 Apache Configuration BETY

sudo -s

# install all ruby related packages
sudo curl --fail -sSLo /etc/yum.repos.d/passenger.repo https://oss-binaries.phusionpassenger.com/yum/definitions/el-passenger.repo
yum -y install ruby ruby-devel mod_passenger

# link static content
ln -s /home/carya/bety/public /var/www/html/bety

# fix GemFile
echo 'gem "test-unit"' >> bety/Gemlile

# fix selinux context (does this need to be done after bety is installed?)
chcon -R -t httpd_sys_content_t /home/carya/bety

# setup a redirect
cat > /etc/httpd/conf.d/bety.conf << EOF
RailsEnv production
RailsBaseURI /bety
PassengerRuby /usr/bin/ruby
<Directory /var/www/html/bety>
  Options +FollowSymLinks
  Require all granted
</Directory>
EOF
systemctl restart httpd

22.2.2.2.5 Rstudio-server

NEED FIXING

NOTE This will allow anybody to login to the machine through the rstudio interface and run any arbitrary code. The login used however is the same as the system login/password.

Based on version of ubuntu 32/64 use either of the following

32bit only

wget http://download2.rstudio.org/rstudio-server-0.98.1103-i386.deb

64bit only

wget http://download2.rstudio.org/rstudio-server-0.98.1103-amd64.deb

# bceome root
sudo -s

# install required packages
apt-get -y install libapparmor1 apparmor-utils libssl0.9.8

# install rstudio
dpkg -i rstudio-server-*
rm rstudio-server-*
echo "www-address=127.0.0.1" >> /etc/rstudio/rserver.conf
echo "r-libs-user=~/R/library" >> /etc/rstudio/rsession.conf
rstudio-server restart

# setup rstudio forwarding in apache
a2enmod proxy_http
cat > /etc/apache2/conf-available/rstudio.conf << EOF
ProxyPass        /rstudio/ http://localhost:8787/
ProxyPassReverse /rstudio/ http://localhost:8787/
RedirectMatch permanent ^/rstudio$ /rstudio/
EOF
a2enconf rstudio
/etc/init.d/apache2 restart

# all done, exit root
exit

22.2.2.2.6 Additional packages

NEED FIXING

HDF5 Tools, netcdf, GDB and emacs

sudo apt-get -y install hdf5-tools cdo nco netcdf-bin ncview gdb emacs ess nedit

22.2.2.3 Mac OSX

These are specific notes for installing PEcAn on Mac OSX and will be referenced from the main installing PEcAn page. You will at least need to install the build environment and Postgres sections. If you want to access the database/PEcAn using a web browser you will need to install Apache. To access the database using the BETY interface, you will need to have Ruby installed.

This document also contains information on how to install the Rstudio server edition as well as any other packages that can be helpful.

22.2.2.3.1 Install build environment

# install R
# download from http://cran.r-project.org/bin/macosx/

# install gfortran 
# download from http://cran.r-project.org/bin/macosx/tools/

# install OpenMPI
curl -o openmpi-1.6.3.tar.gz http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.3.tar.gz
tar zxf openmpi-1.6.3.tar.gz
cd openmpi-1.6.3
./configure --prefix=/usr/local
make all
sudo make install
cd ..

# install szip
curl -o szip-2.1-MacOSX-intel.tar.gz ftp://ftp.hdfgroup.org/lib-external/szip/2.1/bin/szip-2.1-MacOSX-intel.tar.gz
tar zxf szip-2.1-MacOSX-intel.tar.gz
sudo mv szip-2.1-MacOSX-intel /usr/local/szip

# install HDF5

curl -o hdf5-1.8.11.tar.gz http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.11.tar.gz
tar zxf hdf5-1.8.11.tar.gz
cd hdf5-1.8.11
sed -i -e 's/-O3/-O0/g' config/gnu-flags 
./configure --prefix=/usr/local/hdf5 --enable-fortran --enable-cxx --with-szlib=/usr/local/szip
make
# make check
sudo make install
# sudo make check-install
cd ..

22.2.2.3.2 Install Postgres

For those on a Mac I use the following app for postgresql which has postgis already installed (http://postgresapp.com/)

To get postgis run the following commands in psql:

##### Enable PostGIS (includes raster)
CREATE EXTENSION postgis;
##### Enable Topology
CREATE EXTENSION postgis_topology;
##### fuzzy matching needed for Tiger
CREATE EXTENSION fuzzystrmatch;
##### Enable US Tiger Geocoder
CREATE EXTENSION postgis_tiger_geocoder;

To check your postgis run the following command again in psql: SELECT PostGIS_full_version();

22.2.2.3.3 Additional installs

22.2.2.3.3.1 Install JAGS

Download JAGS from http://sourceforge.net/projects/mcmc-jags/files/JAGS/3.x/Mac%20OS%20X/JAGS-Mavericks-3.4.0.dmg/download

22.2.2.3.3.2 Install udunits

Installing udunits-2 on MacOSX is done from source.

download most recent version of Udunits here
instructions for compiling from source

curl -o udunits-2.1.24.tar.gz ftp://ftp.unidata.ucar.edu/pub/udunits/udunits-2.1.24.tar.gz
tar zxf udunits-2.1.24.tar.gz
cd udunits-2.1.24
./configure
make
sudo make install

22.2.2.3.4 Apache Configuration

Mac does not support pdo/postgresql by default. The easiest way to install is use: http://php-osx.liip.ch/

To enable pecan to run from your webserver.

cat > /etc/apache2/others/pecan.conf << EOF
Alias /pecan ${PWD}/pecan/web
<Directory ${PWD}/pecan/web>
  DirectoryIndex index.php
  Options +All
  Require all granted
</Directory>
EOF

22.2.2.3.5 Ruby

The default version of ruby should work. Or use JewelryBox.

22.2.2.3.6 Rstudio Server

For the mac you can download Rstudio Desktop.

22.2.2.4 CentOS / RHEL

These are specific notes for installing PEcAn on RedHat/CentOS and will be referenced from the main installing PEcAn page. You will at least need to install the build environment and Postgres sections. If you want to access the database/PEcAn using a web browser you will need to install Apache. To access the database using the BETY interface, you will need to have Ruby installed.

This document also contains information on how to install the Rstudio server edition as well as any other packages that can be helpful.

22.2.2.4.1 Install build environment

22.2.2.4.1.1 Install and configure PostgreSQL, udunits2, NetCDF

Reference: centoshelp.org

yum install -y epel-release 
yum -y install http://yum.postgresql.org/9.4/redhat/rhel-7-x86_64/pgdg-centos94-9.4-1.noarch.rpm
yum -y install git R postgresql94-server postgresql94-contrib postgis2_94 udunits2 netcdf
/usr/pgsql-9.4/bin/postgresql94-setup initdb
systemctl enable postgresql-9.4
systemctl start postgresql-9.4

22.2.2.4.1.2 Install and start Apache

yum -y install httpd
systemctl enable httpd
systemctl start httpd

22.2.2.4.1.3 Install PHP

sudo yum -y install php php-pgsql

22.2.2.4.1.4 Install ruby-netcdf gem

cd $RUBY_APPLICATION_HOME
export $NETCDF_URL=http://www.gfd-dennou.org/arch/ruby/products/ruby-netcdf/release/ruby-netcdf-0.6.6.tar.gz
export $NETCDF_DIR=/usr/local/netcdf
gem install narray
export NARRAY_DIR="$(ls $GEM_HOME/gems | grep 'narray-')"
export NARRAY_PATH="$GEM_HOME/gems/$NARRAY_DIR"
cd $MY_RUBY_HOME/bin
wget $NETCDF_URL -O ruby-netcdf.tgz
tar zxf ruby-netcdf.tgz && cd ruby-netcdf-0.6.6/
ruby -rubygems extconf.rb --with-narray-include=$NARRAY_PATH --with-netcdf-dir=/usr/local/netcdf-4.3.0
sed -i 's|rb/$|rb|' Makefile
make
make install
cd ../ && sudo rm -rf ruby-netcdf*

cd $RUBY_APPLICATION
bundle install --without development

22.2.2.4.2 Apache Configuration

22.2.2.4.3 Install and configure Rstudio-server

based on Rstudio Server documentation

add PATH=$PATH:/usr/sbin:/sbin to /etc/profile

   cat "PATH=$PATH:/usr/sbin:/sbin; export PATH" >> /etc/profile

add rstudio.conf to /etc/httpd/conf.d/

   wget https://gist.github.com/dlebauer/6921889/raw/d1e0f945228e5519afa6223d6f49d6e0617262bd/rstudio.conf
   sudo mv rstudio.conf /httpd/conf.d/

download and install server:

   wget http://download2.rstudio.org/rstudio-server-0.97.551-i686.rpm
   sudo yum install --nogpgcheck rstudio-server-0.97.551-i686.rpm

restart server sudo httpd restart
now you should be able to access http://<server>/rstudio

22.2.2.4.4 Install Postgres

See documentation under the BETYdb Wiki

22.2.3 Installing BETY

22.2.3.1 Install Postgres

See OS-specific instructions for installing Postgres + PostGIS * Ubuntu * OSX * RedHat / CentOS

22.2.3.2 Install Database + Data

note To install BETYdb without PEcAn, first download the load.bety.sh script

# install database (code assumes password is bety)
sudo -u postgres createuser -d -l -P -R -S bety
sudo -u postgres createdb -O bety bety
sudo -u postgres ./scripts/load.bety.sh -c YES -u YES -r 0
sudo -u postgres ./scripts/load.bety.sh -r 1
sudo -u postgres ./scripts/load.bety.sh -r 2

# configure for PEcAn web app (change password if needed)
cp web/config.example.php web/config.php 

# add models to database (VM only)
./scripts/add.models.sh

# add data to database
./scripts/add.data.sh

# create outputs folder
mkdir ~/output
chmod 777 ~/output

22.2.3.3 Installing BETYdb Web Application

There are two flavors of BETY, PHP and RUBY. The PHP version allows for a minimal interaction with the database while the RUBY version allows for full interaction with the database.

22.2.3.3.1 PHP version

The php version comes with PEcAn and is already configured.

22.2.3.3.2 RUBY version

The RUBY version requires a few extra packages to be installed first.

Next we install the web app.

# install bety
cd
git clone https://github.com/PecanProject/bety.git

# install gems
cd bety
sudo gem2.0 install bundler
bundle install --without development:test:javascript_testing:debug

and configure BETY

# create folders for upload folders
mkdir paperclip/files paperclip/file_names
chmod 777 paperclip/files paperclip/file_names

# create folder for log files
mkdir log
touch log/production.log
chmod 0666 log/production.log

# fix configuration for vm
cp config/additional_environment_vm.rb config/additional_environment.rb
chmod go+w public/javascripts/cache/

# setup bety database configuration
cat > config/database.yml << EOF
production:
  adapter: postgis
  encoding: utf-8
  reconnect: false
  database: bety
  pool: 5
  username: bety
  password: bety
EOF

# setup login tokens
cat > config/initializers/site_keys.rb << EOF
REST_AUTH_SITE_KEY         = 'thisisnotasecret'
REST_AUTH_DIGEST_STRETCHES = 10
EOF

22.2.4 Install Models

This page contains instructions on how to download and install ecosystem models that have been or are being coupled to PEcAn. These instructions have been tested on the PEcAn unbuntu VM. Commands may vary on other operating systems. Also, some model downloads require permissions before downloading, making them unavailable to the general public. Please contact the PEcAn team if you would like access to a model that is not already installed on the default PEcAn VM.

22.2.4.1 BioCro

# Public
echo 'devtools::install_github("ebimodeling/biocro")' | R --vanilla
# Development:
echo 'devtools::install_github("ebimodeling/biocro-dev")' | R --vanilla

BioCro Developers: request from [@dlebauer on GitHub](https://github.com/dlebauer)

22.2.4.2 CLM 4.5

The version of CLM installed on PEcAn is the ORNL branch provided by Dan Ricciuto. This version includes Dan’s point-level CLM processing scripts

Download the code (~300M compressed), input data (1.7GB compressed and expands to 14 GB), and a few misc inputs.

mkdir models
cd models
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm4_5_1_r085.tar.gz
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/ccsm_inputdata.tar.gz
tar -xvzf clm4_5*
tar -xvzf ccsm_inputdata.tar.gz

#Parameter file:
mkdir /home/carya/models/ccsm_inputdata/lnd/clm2/paramdata
cd  /home/carya/models/ccsm_inputdata/lnd/clm2/paramdata
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm_params.c130821.nc
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm_params.c140423.nc

#Domain file:
cd /home/carya/models/ccsm_inputdata/share/domains/domain.clm/
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/domain.lnd.1x1pt_US-UMB_navy.nc

#Aggregated met data file:
cd /home/carya/models/ccsm_inputdata/atm/datm7/CLM1PT_data/1x1pt_US-UMB
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/all_hourly.nc 

## lightning database
cd /home/carya/models/ccsm_inputdata/atm/datm7/NASA_LIS/
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clmforc.Li_2012_climo1995-2011.T62.lnfm_Total_c140423.nc

## surface data
cd /home/carya/models/ccsm_inputdata/lnd/clm2/surfdata
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/surfdata_360x720cru_simyr1850_c130927.nc
cd /home/carya/models/ccsm_inputdata/lnd/clm2/surfdata_map
wget ftp://nacp.ornl.gov/synthesis/2008/firenze/site/clm/surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc_new
mv surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc_new surfdata_1x1pt_US-UMB_I1850CLM45CN_simyr1850.nc

Required libraries

sudo apt-get install mercurial csh tcsh subversion cmake

sudo ln -s /usr/bin/make /usr/bin/gmake

Compile and build default inputs

cd ~/carya/models/clm4_5_1_r085/scripts
python runCLM.py --site US-UMB ––compset I1850CLM45CN --mach ubuntu --ccsm_input /home/carya/models/ccsm_inputdata --tstep 1 --nopointdata --coldstart --cpl_bypass --clean_build

22.2.4.2.1 CLM Test Run

You will see a new directory in scripts: US-UMB_I1850CLM45CN Enter this directory and run (you shouldn’t have to do this normally, but there is a bug with the python script and doing this ensures all files get to the right place):

./US-UMB_I1850CLM45CN.build

Next you are ready to go to the run directory:

/home/carya/models/clm4_5_1_r085/run/US-UMB_I1850CLM45CN/run

Open to edit file: datm.streams.txt.CLM1PT.CLM_USRDAT and check file paths such that all paths start with /home/carya/models/ccsm_inputdata

From this directory, launch the executable that resides in the bld directory:

/home/carya/clm4_5_1_r085/run/US-UMB_I1850CLM45CN/bld/cesm.exe

not sure this was the right location, but wherever the executable is

You should begin to see output files that look like this: US-UMB_I1850CLM45CN.clm2.h0.yyyy-mm.nc (yyyy is year, mm is month) These are netcdf files containing monthly averages of lots of variables.

The lnd_in file in the run directory can be modified to change the output file frequency and variables.

22.2.4.3 DALEC

cd
curl -o dalec_EnKF_pub.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/dalec_EnKF_pub.tgz
tar zxf dalec_EnKF_pub.tgz
rm dalec_EnKF_pub.tgz

cd dalec_EnKF_pub
make dalec_EnKF
make dalec_seqMH
sudo cp dalec_EnKF dalec_seqMH /usr/local/bin

22.2.4.4 ED2

22.2.4.4.1 ED2.2 r46 (used in PEcAn manuscript)

# ----------------------------------------------------------------------
# Get version r46 with a few patches for ubuntu
cd
curl -o ED.r46.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r46.tgz
tar zxf ED.r46.tgz
rm ED.r46.tgz
# ----------------------------------------------------------------------
# configure and compile ed
cd ~/ED.r46/ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.r46

Perform a test run using pre configured ED settings for ED2.2 r46

# ----------------------------------------------------------------------
# Create sample run
cd
mkdir testrun.ed.r46
cd testrun.ed.r46
curl -o ED2IN http://isda.ncsa.illinois.edu/~kooper/EBI/ED2IN.r46
sed -i -e "s#\$HOME#$HOME#" ED2IN
curl -o config.xml  http://isda.ncsa.illinois.edu/~kooper/EBI/config.r46.xml
# execute test run
time ed2.r46

22.2.4.4.2 ED 2.2 r82

cd
curl -o ED.r82.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r82.tgz
tar zxf ED.r82.tgz
rm ED.r82.tgz

cd ED.r82
curl -o ED.r82.patch http://isda.ncsa.illinois.edu/~kooper/EBI/ED.r82.patch
patch -p1 < ED.r82.patch
cd ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.r82

Perform a test run using pre configured ED settings for ED2.2 r82

cd
mkdir testrun.ed.r82
cd testrun.ed.r82
curl -o ED2IN http://isda.ncsa.illinois.edu/~kooper/EBI/ED2IN.r82
sed -i -e "s#\$HOME#$HOME#" ED2IN
curl -o config.xml  http://isda.ncsa.illinois.edu/~kooper/EBI/config.r82.xml
# execute test run
time ed2.r82

22.2.4.4.3 ED 2.2 bleeding edge

cd
git clone https://github.com/EDmodel/ED2.git

cd ED2/ED/build/bin
curl -o include.mk.VM http://isda.ncsa.illinois.edu/~kooper/EBI/include.mk.opt.`uname -s`
./generate_deps.sh
make OPT=VM
sudo cp ../ed_2.1-VM /usr/local/bin/ed2.git

22.2.4.5 CLM-FATES

Prerequisites

sudo apt-get upgrade libnetcdf-dev
sudo apt-get install subversion
sudo apt-get install csh
sudo apt-get install cmake
sudo ln -s /usr/bin/make /usr/bin/gmake
sudo rm /bin/sh
sudo ln -s /bin/bash /bin/sh

wget https://github.com/Unidata/netcdf-fortran/archive/v4.4.4.tar.gz
cd netcdf-4.4.4
./configure
make
sudo make install

you might need to mess around with installing netcdf and netcdf-fortran to get a version FATES likes…

Get code from Github (currently private) and go to cime/scripts directory

git clone git@github.com:NGEET/ed-clm.git
cd ed-clm/cime/scripts/

Within CLM-FATES, to be able to build an executable we need to create a reference run. We’ll also use this reference run to grab defaults from, so we’ll be registering the location of both the reference case (location of executable, scripts, etc) and the reference inputs with the PEcAn database. To begin, copy reference run script from pecan

cp ~/pecan/models/fates/inst/create_1x1_ref_case.sh .

Edit reference case script to set NETCDF_HOME, CROOT (reference run case), DIN_LOC_ROOT (reference run inputs). Also, make sure DIN_LOC_ROOT exists as FATES will not create it itself. Then run the script

./create_1x1_ref_case.sh

Be aware that this script WILL ask you for your password on the NCAR server to download the reference case input data (the guest password may work, haven’t tried this). If it gives an error at the pio stage check the log, but the most likely error is it being unable to find a version of netcdf it likes.

Once FATES is installed, set the whole reference case directory as the Model path (leave filename blank) and set the whole inputs directory as an Input with format clm_defaults.

22.2.4.6 GDAY

Navigate to a directory you would like to store GDAY and run the following:

git clone https://github.com/mdekauwe/GDAY.git

cd GDAY

cd src

make

gday is your executable.

22.2.4.7 JULES

INSTALL STEPS: 1) Download JULES and FCM JULES: Model requires registration to download. Not to be put on PEcAn VM Registration: https://jules.jchmr.org/software-and-documentation Documentation: http://jules-lsm.github.io/vn4.2/index.html

FCM:

    https://github.com/metomi/fcm/
    wget https://github.com/metomi/fcm/archive/2015.05.0.tar.gz

edit makefile ```bash open etc/fcm-make/make.cfg

set JULES_NETCDF = actual instead of dummy set path (e.g. /usr/) and lib_path /lib64 to netCDF libraries ```

compile JULES

cd etc/fcm-make/
{path.to.fcm}/fcm make -f etc/fcm-make/make.cfg --new

UBUNTU VERSION: installed without having to add any perl libraries
#perl stuff that I had to install on pecan2 not PEcAN VM
sudo yum install perl-Digest-SHA
sudo yum install perl-Time-modules
sudo yum install cpan
curl -L http://cpanmin.us | perl - --sudo App::cpanminus
sudo cpanm Time/Piece.pm
sudo cpanm IO/Uncompress/Gunzip.pm

Executable is under build/bin/jules.exe

Example rundir: examples/point_loobos

22.2.4.8 LINKAGES

22.2.4.8.1 R Installation

# Public
echo 'devtools::install_github("araiho/linkages_package")' | R --vanilla

22.2.4.8.2 FORTRAN VERSION

#FORTRAN VERSION
cd
git clone https://github.com/araiho/Linkages.git
cd Linkages
gfortran -o linkages linkages.f
sudo cp linkages /usr/local/bin/linkages.git

22.2.4.9 LPJ-GUESS

Instructions to download source code

Go to LPJ-GUESS website for instructions to access code.

22.2.4.10 MAESPA

Navigate to a directory you would like store MAESPA and run the following:

git clone https://bitbucket.org/remkoduursma/maespa.git

cd maespa

make

maespa.out is your executable. Example input files can be found in the inpufiles directory. Executing measpa.out from within one of the example directories will produce output.

MAESPA developers have also developed a wrapper package called Maeswrap. The usual R package installation method install.packages may present issues with downloading an unpacking a dependency package called rgl. Here are a couple of solutions:

22.2.4.10.1 Solution 1

### From the Command Line
sudo apt-get install r-cran-rgl

then from within R

install.packages("Maeswrap")

22.2.4.10.2 Solution 2

### From the Command line 
sudo apt-get install libglu1-mesa-dev

then from within R

install.packages("Maeswrap")

22.2.4.11 SIPNET

cd
curl -o sipnet_unk.tar.gz http://isda.ncsa.illinois.edu/~kooper/EBI/sipnet_unk.tar.gz
tar zxf sipnet_unk.tar.gz
rm sipnet_unk.tar.gz

cd sipnet_unk
make
sudo cp sipnet /usr/local/bin/sipnet.runk

22.2.4.11.1 SIPNET testrun

cd
curl -o testrun.sipnet.tar.gz http://isda.ncsa.illinois.edu/~kooper/EBI/testrun.sipnet.tar.gz
tar zxf testrun.sipnet.tar.gz
rm testrun.sipnet.tar.gz
cd testrun.sipnet
sipnet.runk

22.2.5 Download and Compile PEcAn

Set R_LIBS_USER

CRAN Reference

# point R to personal lib folder
echo 'export R_LIBS_USER=${HOME}/R/library' >> ~/.profile
source ~/.profile
mkdir -p ${R_LIBS_USER}

22.2.5.1 Download, compile and install PEcAn from GitHub

# download pecan
cd
git clone https://github.com/PecanProject/pecan.git

# compile pecan
cd pecan
./scripts/build.sh --dependencies

Following will run a small script to setup some hooks to prevent people from using the pecan demo user account to check in any code.

# prevent pecan user from checking in code
./scripts/create-hooks.sh

22.2.6 PEcAn Testrun

Do the run, this assumes you have installed the BETY database, sites tar file and SIPNET.

# create folder
cd
mkdir testrun.pecan
cd testrun.pecan

# copy example of pecan workflow and configuration file
cp ../pecan/tests/pecan32.sipnet.xml pecan.xml
cp ../pecan/scripts/workflow.R workflow.R

# exectute workflow
rm -rf pecan
./workflow.R pecan.xml

NB: pecan.xml is configured for the virtual machine, you will need to change the field from ‘/home/carya/’ to wherever you installed your ‘sites’, usually $HOME

22.2.7 PEcAn Customizations

22.2.7.1 PalEON version of PEcAn

Install the following packages: gnome2, xfce4, firefox

sudo apt-get -y install gdm gnome-shell xfce4 firefox

Install Rstudio-desktop (+ libjpeg62 seems to be a dependency)

PROC=$( uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/' )
wget http://download1.rstudio.org/rstudio-0.98.507-${PROC}.deb
sudo apt-get install libjpeg62
sudo dpkg -i rstudio-*.deb
rm rstudio-*.deb

Additional pieces of software

wget http://chrono.qub.ac.uk/blaauw/LinBacon_2.2.zip
unzip LinBacon_2.2.zip
rm LinBacon_2.2.zip

wget http://chrono.qub.ac.uk/blaauw/clam.zip
unzip clam.zip
rm clam.zip

Additional R packages:

cat << EOF | R --vanilla
list.of.packages <- c('R2jags', 'RCurl', 'RJSONIO', 'reshape2', 'plyr', 'fields',
                      'maps', 'maptools', 'ggplot2', 'mvtnorm', 'devtools')
new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])]
if(length(new.packages)) {
  print("installing : ")
  print(new.packages)
  install.packages(new.packages, repos="http://cran.rstudio.com/")
}

require(devtools)
install_github("neotoma", "ropensci")
EOF

22.3 AWS Setup

22.3.1 Porting VM to AWS

The following are Mike’s rough notes from a first attempt to port the PEcAn VM to the AWS. This was done on a Mac

These notes are based on following the instructions here

22.3.1.1 Convert PEcAn VM

AWS allows upload of files as VMDK but the default PEcAn VM is in OVA format

If you haven’t done so already, download the PEcAn VM
Split the OVA file into OVF and VMDK files

tar xf <ovafile>

22.3.1.2 Set up an account on AWS

After you have an account you need to set up a user and save your access key and secret key

In my case I created a user named ‘carya’

Note: the key that ended up working had to be made at https://console.aws.amazon.com/iam/home#security_credential, not the link above.

22.3.1.3 Install EC2 command line tools

wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip

sudo mkdir /usr/local/ec2

sudo unzip ec2-api-tools.zip -d /usr/local/ec2

If need be, download and install JDK

export JAVA_HOME=$(/usr/libexec/java_home)

export EC2_HOME=/usr/local/ec2/ec2-api-tools-<version>

export PATH=$PATH:$EC2_HOME/bin

Then set your user credentials as environment variables:

export AWS_ACCESS_KEY=xxxxxxxxxxxxxx

export AWS_SECRET_KEY=xxxxxxxxxxxxxxxxxxxxxx

Note: you may want to add all the variables set in the above EXPORT commands above into your .bashrc or equivalent.

22.3.1.4 Create an AWS S3 ‘bucket’ to upload VM to

Go to https://console.aws.amazon.com/s3 and click “Create Bucket”

In my case I named the bucket ‘pecan’

22.3.1.5 Upload

In the code below, make sure to change the PEcAn version, the name of the bucket, and the name of the region. Make sure that the architecture matches the version of PEcAn you downloaded (i386 for 32 bit, x86_64 for 64 bit).

Also, you may want to choose a considerably larger instance type. The one chosen below is that corresponding to the AWS Free Tier

ec2-import-instance PEcAn32bit_1.2.6-disk1.vmdk --instance-type t2.micro --format VMDK --architecture i386 --platform Linux --bucket pecan --region us-east-1 --owner-akid $AWS_ACCESS_KEY --owner-sak $AWS_SECRET_KEY

Make sure to note the ID of the image since you’ll need it to check the VM status. Once the image is uploaded it will take a while (typically about an hour) for Amazon to convert the image to one it can run. You can check on this progress by running

ec2-describe-conversion-tasks <image.ID>

22.3.1.6 Configuring the VM

On the EC2 management webpage, https://console.aws.amazon.com/ec2, if you select Instances on the left hand side (LHS) you should be able to see your new PEcAn image as an option under Launch Instance.

Before launching, you will want to update the firewall to open up additional ports that PEcAn needs – specifically port 80 for the webpage. Port 22 (ssh/sftp) should be open by default. Under “Security Groups” select “Inbound” then “Edit” and then add “HTTP”.

Select “Elastic IPs” on the LHS, and “Allocate New Address” in order to create a public IP for your VM.

Next, select “Network Interfaces” on the LHS and then under Actions select “Associate Addresses” then choose the Elastic IP you just created.

22.3.2 Set up multiple instances (optional)

For info on setting up multiple instances with load balancing see: http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/gs-ec2VPC.html

Select “Load Balancers” on the LHS, click on “Create Load Balancer”, follow Wizard keeping defaults.

To be able to launch multiple VMs: Under “Instances” convert VM to an Image. When done, select Launch, enable multiple instances, and associate with the previous security group. Once running, go back to “Load Balancers” and add the instances to the load balancer. Each instance can be accessed individually by it’s own public IP, but external users should access the system more generally via the Load Balancers DNS.

22.3.2.1 Booting the VM

Return to “Instances” using the menu on the LHS.

To boot the VM select “Actions” then “Instance State” then “Start”. In the future, once you have the VM loaded and configured this last step is the only one you will need to repeat to turn your VM on and off.

The menu provided should specify the Public IP where the VM has launched

22.4 Shiny Setup

Installing and configuring Shiny for PEcAn authors - Alexey Shiklomanov - Rob Kooper

NOTE: Instructions are only tested for CentOS 6.5 and Ubuntu 16.04 NOTE: Pretty much every step here requires root access.

22.4.1 Install the Shiny R package and Shiny server

Follow the instructions on the Shiny download page for the operating system you are using.

22.4.2 Modify the shiny configuration file

The Shiny configuration file is located in /etc/shiny-server/shiny-server.conf. Comment out the entire file and add the following, replacing <username> with your user name and <location> with the URL location you want for your app. This will allow you to run Shiny apps from your web browser at https://your.server.edu/shiny/your-location

run as shiny;
server {
    listen 3838;
    location /<location>/ {
        run as <username>;
        site_dir /path/to/your/shiny/app;
        log_dir /var/log/shiny-server;
        directory_index on;
    }
}

For example, my configuration on the old test-pecan looks like this.

run as shiny;
server {
    listen 3838;
    location /ashiklom/ {
        run as ashiklom;
        site_dir /home/ashiklom/fs-data/pecan/shiny/;
        log_dir /var/log/shiny-server;
        directory_index on;
    }
}

…and I can access my Shiny apps at, for instance, https://test-pecan.bu.edu/shiny/ashiklom/workflowPlots.

You can add as many location <loc> { ... } fields as you would like.

run as shiny;
server {
    listen 3838;
    location /ashiklom/ {
        ...
    }
    location /bety/ {
        ...
    }
}

If you change the configuration, for example to add a new location, you will need to restart Shiny server. If you are setting up a new instance of Shiny, skip this step and continue with the guide, since there are a few more steps to get Shiny working. If there is an instance of Shiny already running, you can restart it with:

## On CentOS
sudo service shiny-server stop
sudo service shiny-server start

## On Ubuntu
sudo systemctl stop shiny-server.service
sudo systemctl start shiny-server.service

22.4.3 Set the Apache proxy

Create a file with the following name, based on the version of the operating system you are using:

Ubuntu 16.04 (pecan1, pecan2, test-pecan) – /etc/apache2/conf-available/shiny.conf
CentOS 6.5 (psql-pecan) – /etc/httpd/conf.d/shiny.conf

Into this file, add the following:

ProxyPass           /shiny/ http://localhost:3838/
ProxyPassReverse    /shiny/ http://localhost:3838/
RedirectMatch permanent ^/shiny$ /shiny/

22.4.3.1 Ubuntu only: Enable the new shiny configuration

sudo a2enconf shiny

This will create a symbolic link to the newly created shiny.conf file inside the /etc/apache2/conf-enabled directory. You can do ls -l /etc/apache2/conf-enabled to confirm that this worked.

22.4.4 Enable and start the shiny server, and restart apache

22.4.4.1 On CentOS

sudo ln -s /opt/shiny-server/config/init.d/redhat/shiny-server /etc/init.d
sudo service shiny-server stop
sudo service shiny-server start
sudo service httpd restart

You can check that Shiny is running with service shiny-server status.

22.4.4.2 On Ubuntu

Enable the Shiny server service. This will make sure Shiny runs automatically on startup.

sudo systemctl enable shiny-server.service

Restart Apache.

sudo apachectl restart

Start the Shiny server.

sudo systemctl start shiny-server.service

If there are problems, you can stop the shiny-server.service with…

sudo systemctl stop shiny-server.service

…and then use start again to restart it.

22.4.5 Troubleshooting

Refer to the log files for shiny (/var/log/shiny-server.log) and httpd (on CentOS, /var/log/httpd/error-log; on Ubuntu, /var/log/apache2/error-log).

22.4.6 Further reading

Shiny server configuration reference

22.5 Thredds Setup

Installing and configuring Thredds for PEcAn authors - Rob Kooper

NOTE: Instructions are only tested for Ubuntu 16.04 on the VM, if you have instructions for CENTOS/RedHat please update this documentation NOTE: Pretty much every step here requires root access.

22.5.1 Install the Tomcat 8 and Thredds webapp

The Tomcat 8 server can be installed from the default Ubuntu repositories. The thredds webapp will be downloaded and installed from unidata.

22.5.2 Ubuntu

First step is to install Tomcat 8 and configure it. The flag -Dtds.content.root.path should point to the location of where the thredds folder is located. This needs to be writeable by the user for tomcat. -Djava.security.egd is a special flag to use a different random number generator for tomcat. The default would take to long to generate a random number.

apt-get -y install tomcat8 openjdk-8-jdk
echo JAVA_OPTS=\"-Dtds.content.root.path=/home/carya \${JAVA_OPTS}\" >> /etc/default/tomcat8
echo JAVA_OPTS=\"-Djava.security.egd=file:/dev/./urandom \${JAVA_OPTS}\" >> /etc/default/tomcat8
service tomcat8 restart

Next is to install the webapp.

mkdir /home/carya/thredds
chmod 777 /home/carya/thredds

wget -O /var/lib/tomcat8/webapps/thredds.war ftp://ftp.unidata.ucar.edu/pub/thredds/4.6/current/thredds.war

Finally we configure Apache to prox the thredds server

cat > /etc/apache2/conf-available/thredds.conf << EOF
ProxyPass        /thredds/ http://localhost:8080/thredds/
ProxyPassReverse /thredds/ http://localhost:8080/thredds/
RedirectMatch permanent ^/thredds$ /thredds/
EOF
a2enmod proxy_http
a2enconf thredds
service apache2 reload

22.5.3 Customize the Thredds server

To customize the thredds server for your installation edit the file in /home/carya/thredds/threddsConfig.xml. For example the following file is included in the VM.

<?xml version="1.0" encoding="UTF-8"?>
<threddsConfig>

  <!-- all options are commented out in standard install - meaning use default values -->
  <!-- see http://www.unidata.ucar.edu/software/thredds/current/tds/reference/ThreddsConfigXMLFile.html -->
  <serverInformation>
    <name>PEcAn</name>
    <logoUrl>/pecan/images/pecan_small.jpg</logoUrl>
    <logoAltText>PEcAn</logoAltText>

    <abstract>Scientific Data</abstract>
    <keywords>meteorology, atmosphere, climate, ocean, earth science</keywords>
    
    <contact>
      <name>Rob Kooper</name>
      <organization>NCSA</organization>
      <email>kooper@illinois.edu</email>
      <!--phone></phone-->
    </contact>
    <hostInstitution>
      <name>PEcAn</name>
      <webSite>http://www.pecanproject.org/</webSite>
      <logoUrl>/pecan/images/pecan_small.jpg</logoUrl>
      <logoAltText>PEcAn Project</logoAltText>
    </hostInstitution>
  </serverInformation>

  <!--
  The <catalogRoot> element:
  For catalogs you don't want visible from the /thredds/catalog.xml chain
  of catalogs, you can use catalogRoot elements. Each catalog root config
  catalog is crawled and used in configuring the TDS.

  <catalogRoot>myExtraCatalog.xml</catalogRoot>
  <catalogRoot>myOtherExtraCatalog.xml</catalogRoot>
  -->

  <!--
   * Setup for generated HTML pages.
   *
   * NOTE: URLs may be absolute or relative, relative URLs must be relative
   * to the webapp URL, i.e., http://server:port/thredds/.
    -->
  <htmlSetup>
    <!--
     * CSS documents used in generated HTML pages.
     * The CSS document given in the "catalogCssUrl" element is used for all pages
     * that are HTML catalog views. The CSS document given in the "standardCssUrl"
     * element is used in all other generated HTML pages.
     * -->
    <standardCssUrl>tds.css</standardCssUrl>
    <catalogCssUrl>tdsCat.css</catalogCssUrl>
    <openDapCssUrl>tdsDap.css</openDapCssUrl>

    <!--
     * The Google Analytics Tracking code you would like to use for the
     * webpages associated with THREDDS. This will not track WMS or DAP
     * requests for data, only browsing the catalog.
    -->
    <googleTrackingCode></googleTrackingCode>

  </htmlSetup>
  
  <!-- 
    The <TdsUpdateConfig> element controls if and how the TDS checks
    for updates. The default is for the TDS to check for the current
    stable and development release versions, and to log that information
    in the TDS serverStartup.log file as INFO entries.

  <TdsUpdateConfig>
     <logVersionInfo>true</logVersionInfo>
  </TdsUpdateConfig>
  -->
   
  <!--
   The <CORS> element controls Cross-Origin Resource Sharing (CORS).
   CORS is a way to allow a website (such as THREDDS) to open up access
   to resources to web pages and applications running on a different domain.
   One example would be allowing a web-application to use fonts from
   a separate host. For TDS, this can allow a javascript app running on a
   different site to access data on a THREDDS server.
   For more information see: https://en.wikipedia.org/wiki/Cross-origin_resource_sharing
   The elements below represent defaults. Only the <enabled> tag is required
   to enable CORS. The default allowed origin is '*', which allows sharing
   to any domain.
  <CORS>
    <enabled>false</enabled>
    <maxAge>1728000</maxAge>
    <allowedMethods>GET</allowedMethods>
    <allowedHeaders>Authorization</allowedHeaders>
    <allowedOrigin>*</allowedOrigin>
  </CORS>
  -->

  <!--
   The <CatalogServices> element:
   - Services on local TDS served catalogs are always on.
   - Services on remote catalogs are set with the allowRemote element
   below. They are off by default (recommended).
   -->
  <CatalogServices>
    <allowRemote>false</allowRemote>
  </CatalogServices>

  <!--
  Configuring the CDM (netcdf-java library)
  see http://www.unidata.ucar.edu/software/netcdf-java/reference/RuntimeLoading.html

  <nj22Config>
    <ioServiceProvider class="edu.univ.ny.stuff.FooFiles"/>
    <coordSysBuilder convention="foo" class="test.Foo"/>
    <coordTransBuilder name="atmos_ln_sigma_coordinates" type="vertical" class="my.stuff.atmosSigmaLog"/>
    <typedDatasetFactory datatype="Point" class="gov.noaa.obscure.file.Flabulate"/>
  </nj22Config>
  -->

  <!--
  CDM uses the DiskCache directory to store temporary files, like uncompressed files.
  <DiskCache>
    <alwaysUse>false</alwaysUse>
    <scour>1 hour</scour>
    <maxSize>1 Gb</maxSize>
  </DiskCache>
  -->

  <!--
  Caching open NetcdfFile objects.
  default is to allow 50 - 100 open files, cleanup every 11 minutes
  <NetcdfFileCache>
    <minFiles>50</minFiles>
    <maxFiles>100</maxFiles>
    <scour>11 min</scour>
  </NetcdfFileCache>
  -->

  <!--
  The <HTTPFileCache> element:
  allow 10 - 20 open datasets, cleanup every 17 minutes
  used by HTTP Range requests.
  <HTTPFileCache>
    <minFiles>10</minFiles>
    <maxFiles>20</maxFiles>
    <scour>17 min</scour>
  </HTTPFileCache>
  -->

  <!--
  Writing GRIB indexes.
  <GribIndexing>
    <setExtendIndex>false</setExtendIndex>
    <alwaysUseCache>false</alwaysUseCache>
  </GribIndexing>
  -->

  <!--
  Persist joinNew aggregations to named directory. scour every 24 hours, delete stuff older than 90 days
  <AggregationCache>
    <scour>24 hours</scour>
    <maxAge>90 days</maxAge>
    <cachePathPolicy>NestedDirectory</cachePathPolicy>
  </AggregationCache>
  -->

  <!--
  How to choose the template dataset for an aggregation. latest, random, or penultimate
  <Aggregation>
    <typicalDataset>penultimate</typicalDataset>
  </Aggregation>
  -->

  <!--
  The Netcdf Subset Service is off by default.
  <NetcdfSubsetService>
    <allow>false</allow>
    <scour>10 min</scour>
    <maxAge>-1 min</maxAge>
  </NetcdfSubsetService>
  -->

  <!--
  <Opendap>
    <ascLimit>50</ascLimit>
    <binLimit>500</binLimit>
    <serverVersion>opendap/3.7</serverVersion>
  </Opendap>
    -->
  
  <!--
  The WCS Service is off by default.
  Also, off by default (and encouraged) is operating on a remote dataset.
  <WCS>
    <allow>false</allow>
    <allowRemote>false</allowRemote>
    <scour>15 min</scour>
    <maxAge>30 min</maxAge>
  </WCS>
  -->

  <!--
  <WMS>
    <allow>false</allow>
    <allowRemote>false</allowRemote>
    <maxImageWidth>2048</maxImageWidth>
    <maxImageHeight>2048</maxImageHeight>
  </WMS>
  -->

  <!--
  <NCISO>
    <ncmlAllow>false</ncmlAllow>
    <uddcAllow>false</uddcAllow>
    <isoAllow>false</isoAllow>
  </NCISO>
  -->

  <!-- CatalogGen service is off by default.
  <CatalogGen>
    <allow>false</allow>
  </CatalogGen>
   -->

  <!-- DLwriter service is off by default.
       As is support for operating on remote catalogs.
  <DLwriter>
    <allow>false</allow>
    <allowRemote>false</allowRemote>
  </DLwriter>
   -->

  <!-- DqcService is off by default.
  <DqcService>
    <allow>false</allow>
  </DqcService>
   -->

  <!--
   Link to a Viewer application on the HTML page:
   <Viewer>my.package.MyViewer</Viewer>
   -->

   <!--
   Add a DataSource - essentially an IOSP with access to Servlet request parameters
   <datasetSource>my.package.DatsetSourceImpl</datasetSource>
   -->

  <!--
   set FeatureCollection logging
  <FeatureCollection>
     <RollingFileAppender>
       <MaxFileSize>1 MB</MaxFileSize>
       <MaxBackups>5</MaxBackups>
       <Level>INFO</Level>
     </RollingFileAppender>
  </FeatureCollection>
  -->

  <!--
    Configure how the NetCDF-4 C library is discovered and used.
    libraryPath: The directory in which the native library is installed.
    libraryName: The name of the native library. This will be used to locate the proper .DLL, .SO, or .DYLIB file
      within the libraryPath directory.
    useForReading: By default, the native library is only used for writing NetCDF-4 files; a pure-Java layer is
      responsible for reading them. However, if this property is set to true, then it will be used for reading
      NetCDF-4 (and HDF5) files as well.
  -->
  <!--
  <Netcdf4Clibrary>
    <libraryPath>/usr/local/lib</libraryPath>
    <libraryName>netcdf</libraryName>
    <useForReading>false</useForReading>
  </Netcdf4Clibrary>
  -->
</threddsConfig>

22.5.4 Update the catalog

For example to update the catalog with the latest data, run the following command from the root crontab. This cronjob will also synchronize the database with remote servers and dump your database (by default in /home/carya/dump)

0 * * * * /home/carya/pecan/scripts/cron.sh -o /home/carya/dump

22.5.5 Troubleshooting

Refer to the log files for Tomcat (/var/log/tomcat8/*) and Thredds (/home/carya/thredds/logs).

22.5.6 Further reading

Thredds reference