6 Archived Setup Documentation

These pages are being kept temporarily for reference. Much of what is here is superceded by the preceding setup documentation.

6.1 Setting up a RedHat or CentOS Server

6.1.1 Overview

BETYdb runs on RedHat, CentOS, Ubuntu and OSX; instructions for installing on these systems can be found in the PEcAn documentation.

The original BETYdb (betydb.org) runs on a Red Hat Enterprise Linux version 5.8 Server. To simulate this environment, we have set up a CentOS 5.8 server at pecandev.igb.illinois.edu for testing.

This documentation is aimed at installing BETYdb on a production RedHat, CentOS or similar operating system.

These instructions have been tested and refined on our production (ebi-forecast.igb.illinois.edu) and development (pecandev.igb.illinois.edu) servers. See the Installing BETY section of the PEcAn documentation for more generic installation instructions.

If you have any questions about installing BETYdb … please submit an issue or send an email.

6.1.1.1 Create an netinstall of the CentOS ISO

6.1.1.2 Boot from CD and Install

6.1.1.3 Configuration

  1. Add new user adduser johndoe
  2. add user to root sudo su emacs /etc/sudoers

  3. add the line

    johndoe  ALL=(ALL)  ALL

6.1.1.4 Add new repository

instructions here: http://www.rackspace.com/knowledge_center/article/installing-rhel-epel-repo-on-centos-5x-or-6x

wget http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
sudo rpm -Uvh epel-release-5*.rpm

… move to PEcAn documentation page for build environment

https://pecanproject.github.io/pecan-documentation/master/pecan-setup.html#installing-bety

Remember to install R 3.0:

wget http://cran.us.r-project.org/src/base/R-3/R-3.0.1.tar.gz
tar xzf R-3.0.1.tar.gz
cd R-3.0.1
./configure
make
sudo make install

6.1.1.5 Site data installation

cd /usr/local/ebi

rm -rf sites
curl -o sites.tgz http://isda.ncsa.illinois.edu/~kooper/EBI/sites.tgz
tar zxf sites.tgz
sed -i -e "s#/home/kooper/projects/EBI#${PWD}#" sites/*/ED_MET_DRIVER_HEADER
rm sites.tgz

rm -rf inputs
wget http://isda.ncsa.illinois.edu/~kooper/EBI/inputs.tgz
tar zxf inputs.tgz
rm inputs.tgz

6.1.1.6 Database Creation

See the PEcAn documentation for additional information, e.g. on the scripts (load.bety.sh is in the PEcAn repo), running the rails and/or php front-ends

# install database (code assumes password is bety)
sudo -u postgres createuser -d -l -P -R -S bety
sudo -u postgres createdb -O bety bety
sudo -u postgres CREATE=YES REMOTESITE=0 scripts/load.bety.sh
REMOTESITE=1 scripts/load.bety.sh

Open up updatedb.sh and remove the two lines

#change to home
cd

If these are left in, the script will attempt to put the site data in ~/sites instead of /usr/local/ebi/sites.

#fetch and updates the bety database
./updatedb.sh

6.1.1.7 Ruby installation

The version of ruby available through yum is too low, so we have to use rvm

user$ \curl -L https://get.rvm.io | sudo bash -s stable
rvm install 1.9
rvm use 1.9

yum upgrade rubygem

yum install mysql-devel.x86_64
yum install ImageMagick-devel.x86_64
yum install rubygem-rails
yum install httpd-devel
wget http://www.sqlite.org/2013/sqlite-autoconf-3071700.tar.gz
tar -xzf sqlite-autoconf-3071700.tar.gz
cd sqlite-autoconf-3071700.tar.gz
./configure
make
make install

If you plan to run the JavaScript-based rspec tests, you will also need to install qt:

wget http://download.qt-project.org/official_releases/qt/4.8/4.8.6/qt-everywhere-opensource-src-4.8.6.tar.gz
tar -xf qt-everywhere-opensource-src-4.8.6.tar.gz
cd qt-everywhere-opensource-src-4.8.6
./configure --prefix=/usr/local/qt-4.8.6
make
make install

export PATH=/usr/local/qt-4.8.6/bin:$PATH
export PATH=/usr/local/ruby-1.8.3/bin:$PATH
# Before running the "gem" command, type "gem environment" and make sure the installation directory matches
# the installation directory used by the BetyDB Rail application.  If you are using RVM, it should suffice
# simply to switch the the root directory of the application, e.g. "cd /usr/local/ebi".
gem install capybara-webkit

Then all the ruby gems bety needs.

cd /usr/local/bety
gem install bundler
bundle install
# (If you didn't install qt and capybara-webkit, you will need to run "bundle install --without test_js" instead.)

Configuration for Bety:

cd /usr/local/ebi/bety

# create folders for upload folders
mkdir paperclip/files paperclip/file_names
chmod 777 paperclip/files paperclip/file_names

# create folder for log files
mkdir log
touch log/production.log
chmod 0666 log/production.log
touch log/test.log
chmod 0666 log/test.log

cat > config/database.yml << EOF
production:
  adapter: mysql2
  encoding: latin1
  reconnect: false
  database: bety
  pool: 5
  username: bety
  password: bety

test:
  adapter: mysql2
  encoding: latin1
  reconnect: false
  database: test
  pool: 5
  username: bety
  password: bety
EOF


# setup per-instance configuration
cp config/application.yml.template config/application.yml
# Be sure to edit the sample values in application.yml to provide values appropriate to your server.
# In particular, the actual value for rest_auth_site_key needs to be replaced with one matching the DB you are using.



# configure apache
ln -s /usr/local/ebi/bety/public /var/www/bety

cat > /etc/apache2/conf.d/bety << EOF
RailsEnv production
RailsBaseURI /bety
<Directory /var/www/bety>
   Options FollowSymLinks
   AllowOverride None
   Order allow,deny
   Allow from all
</Directory>
EOF

You may have to change your DocumentRoot in /etc/httpd/conf/httpd.conf from “/var/www/html” to “/var/www” if you get the error message ‘Passenger error #2 An error occurred while trying to access’/var/www/html/bety’: Cannot resolve possible symlink ‘/var/www/html/bety’: No such file or directory (2)’. Up next make apache2 and passenger play nicely:

rvmsudo passenger-install-apache2-module

If that fails, try

sudo -s
source `rvm gemdir`

Finally run the tests

cd /usr/local/eby/bety/
bundle exec rake db:test:prepare && bundle exec rake db:fixtures:load RAILS_ENV=test && bundle exec rspec spec/

[Note: If you are using RVM version 1.11 or later, you can omit the “bundle exec” portion of all rake and rspec commands. For example, the command above could just be typed as rake db:test:prepare && rake db:fixtures:load RAILS_ENV=test && rspec spec/]

6.2 Instructions for Installing and Configuring PostgreSQL

These instructions are for installation on Ubuntu. For other Linux-like systems, the steps are similar. Mainly the file locations and the installation command (apt-get here) will be different.

  1. Run sudo apt-get install postgresql.
  2. Create user bety: Run sudo -u postgres createuser -P bety. Type bety when prompted for a password. This is the default user the the BetyDB Rails application connects to the database as.
  3. [Optional:] If you wish to administer the database without switching to user postgres (so that you don’t have to prefix all your commands with sudo -u postgres) add yourself as a superuser: sudo -u postgres createuser -s <your_ubuntu_login_name>. This will allow you to run all PostgreSQL commands as yourself without a password.

  4. Edit the pg_hba.conf file to allow bety to log in using a password. On Ubuntu, this file is most likely located at /etc/postgresql/9.3/main/pg_hba.conf (assuming you are using version 9.3). In detail:

    sudo su # this allows accessing and editing the conf file
cd /etc/postgresql/9.3/main
cp pg_hba.conf pg_hba.conf-original # optional convenience if you need to start over
emacs pg_hba.conf # use your favorite editor
    In this file, you will see a line like this local all all peer Assuming this is a fresh install of PostgreSQL, you can give password access to user bety by adding the line local bety bety md5 immediately before this. Then save the file.
  5. In order for the configuration to take effect, you must reload the config files: sudo /etc/init.d/postgresql reload

  6. You can test that user bety exists and can log in by running the following:

    sudo -u postgres createdb -O bety bety
psql -U bety

6.2.0.1 Manually dumping and installing BETYdb

In this example, we are dumping BETY from “betyhost” to “myserver”

ssh betyhost
pg_dump -U postgres bety > bety_YYYYMMDD.sql
rsync bety_YYYYMMDD.sql myserver:
6.2.0.1.1 Create Copy of BETY
ssh myserver
createdb -U postgres bety_copy
6.2.0.1.2 Enable PostGIS
psql -U postgres
postgres=# \c bety_copy
postgres=# CREATE EXTENSION POSTGIS;

Also see Creating a new PostGIS-enabled database template.

6.2.0.1.3 Import database
psql -U postgres bety_copy < bety_YYYYMMDD.sql

6.3 Installing the BETYdb Rails Application

Note: This guide is aimed at Rails developers and testers. If you are a Pecan developer, you may want to use the notes in the PEcAn documentation instead of or in addition to the notes below.

quick start The install_pecan.sh script contains steps used to create a Virtual Machine on line 398 and dependencies for different OS’s on line 102

6.3.1 Prerequisites

  1. Git
  2. Ruby 2.1.5 (Anything later than version 1.9.3 will probably work, but 2.1.5 is the officially supported version.) If you are doing Rails development or if you are using Ruby for outside of BETYdb, you may want to install RVM so that you can easily switch between Rails versions and Gem sets.
  3. PostgreSQL with the PostGIS extension (see Instructions for Installing and Configuring PostgreSQL for information on installing and configuring PostgreSQL)
  4. Apache web server (optional; developers in particular can simply use the built-in Rails server)

In addition, the scripts below assume you have a working Bash shell. (Windows users might be able to use Cygwin or some other some other port of Linux tools.)

6.3.2 Installing the Rails Application

6.3.2.1 Installing the Rails code and Ruby Gems

Run these commands to get the Rails code and the Ruby Gems that it uses:


# This can be any place you have write permissions for, probably something under your home directory:
INSTALLATION_DIRECTORY=~/projects

# install bety
cd $INSTALLATION_DIRECTORY

# Developers who will be submitting Git pull requests should make a fork of bety.git on GitHub and then
# replace the URL below with the address of their own copy:
git clone https://github.com/PecanProject/bety.git

# install gems
cd bety
gem install bundler # not needed if you already have bundler
bundle install # Use the --without option to avoid installing certain groups of Gems; see below.
exit

[Note: If you can’t or don’t wish to install the capybara-webkit gem, you can comment it out in the Gemfile before running bundle install.

Note: If you receive “checking for pg_config… no” and the associated errors then you may need update build.pg using the “bundle config” command. For example, to update the bundle executable with the location of the pg_config command you can run: bundle config build.pg –with-pg-config=/usr/pgsql-9.4/bin/pg_config This assumes your pg_config is located at /usr/pgsql-9.4/bin/. Update this path as necessary for your local PostgreSQL/Postgis install]

6.3.2.1.1 Minimizing Gem Installation

Certain Ruby Gems are difficult or time-consuming to install on certain platforms, and if they are not essential to your work, you may wish to avoid installing them. (If this isn’t a concern, you may skip this section.)

If you look at the Gemfile in the root directory of the BETYdb Rails code, you will see the certain Gems are specified within group blocks; this means they are intended to be used only in certain contexts. If you don’t intend to use BETYdb within those contexts, you may safely use the --without option to bundle install to exclude the Gems used only in those contexts.

As an example, the passenger Gem is used only in the production environment. Therefore, it is in a production group within the Gemfile. If you run

bundle install --without=production

the bundler will skip installation of passenger.

Moreover, this is a “remembered” option: the next time you run bundle install, it will remember not to install production-only Gems even if you haven’t specified the --without option. Furthermore, this “remembered option” is also respected by WEBrick, the default Rails server, so it won’t complain that you didn’t install the passenger Gem.

As another example, the capybara-webkit Gem is difficult and time-consuming to install on some platforms, and unless you are running the RSpec tests, you can do without it. (In fact, even if you are running RSpec tests, most of the tests don’t use capybara-webkit, and for those that do, you can either skip them or tell them to use selenium-webdriver instead.)

capybara-webkit is in a group called javascript_testing, so to avoid installing it, run

bundle install --without=javascript_testing

To see what the remembered “without” options are, run

bundle config

You can also use bundle config to specify directly what groups Bundler should skip. For example, to tell Bundler to ignore all groups except the production group, pass a colon-separated list containing all of the other groups to bundle config --local without:

bundle config --local without development:test:javascript_testing:debug

To revert to installing everything when you run bundle install, remove the without setting from the configuration with

bundle config --delete without

6.3.2.2 Configuring Rails

Configure the BETYdb Rails application using the following commands:

cd $INSTALLATION_DIRECTORY/bety

# setup bety database configuration
cat > config/database.yml << EOF
development:
  adapter: postgis
  encoding: utf-8
  reconnect: false
  database: bety
  pool: 5
  username: bety
  password: bety
EOF

# Optional: Override some of the default configuration settings given in config/defaults.yml.
cp config/application.yml.template config/application.yml
# Read the comments in this file and set the variable values you are interested in; delete the other settings.

6.3.2.3 Installing the Database

Note To join the distributed network of databases, see the chapter Distributed instances of BETYdb.

In the script directory of the bety Rails installation, find and run the update-betydb.sh script:

./update-betydb.sh

This script is a wrapper script for the script load.bety.sh from the Pecan project. The latter can be downloaded by running update-betydb.sh without options. Use the -h option for more information.

6.3.3 Updating / Syncing the database

See instructions Updating the BETY database.

6.3.4 Starting the BETYdb Rails Web Application

  1. cd to the bety directory, the directory you cloned the Rails code to.
  2. Run rails s.
  3. You should now be able to visit the web application at http://localhost:3000.
  4. To log in, use Login: carya, Password: illinois

6.3.5 Logrotation

To prevent the log files from growing to large it is recommended to use logrotation. This will rotate the logs (for example every week) and append .1 etc. to the logfiles. The following can be used on an Ubuntu system.

Edit /etc/logrotate.conf and add the following snippet at the bottom (replacing /home/bety/bety with the actual path to the installation of bety):

/home/bety/bety/log/*.log {
  daily
  missingok
  rotate 7
  compress
  delaycompress
  notifempty
  copytruncate
}

Once this installed you can force a logrotate to happen (or wait till Sunday) by using: sudo /usr/sbin/logrotate -f /etc/logrotate.conf