Installation Instructions for Ubuntu 12.04¶
Defaults¶
This document sets up the Indivo backend server and Indivo UI server on the same machine. The backend server runs on port 8000, the UI server on port 80 by default. This is configurable by changing the appropriate port numbers in the instructions below. For databases, the instructions below assume indivo for the user and database name
Pre-Requisites¶
This documents the installation of a complete Indivo X server and user interface. For concreteness, we show all of the exact instructions needed when installing on Ubuntu Linux 12.04. We welcome variants of these explicit instructions from folks installing on other operating systems, and are happy to post them alongside these instructions.
Note
We recommend you do this by sudo’ing from a non-root user. If you would like to do this as root make sure you create at least one non-root user with:
useradd -m {USER}
otherwise the default locale will not be set. This issue is most common on a new OS build.
Install the following system packages:
apt-get install libxslt-dev libxml2-dev python-dev git
Database Install¶
Postgres¶
Install¶
Install the PostgreSQL meta package (currently PostgreSQL 9.1 on 12.04) and header files for libpq5:
apt-get install postgresql libpq-dev
Setup¶
Note
the instructions below assume you have named your database indivo and will create a database user named indivo
Create a PostgreSQL user for your Indivo service, e.g. “indivo” and setup a password:
su postgres
createuser --superuser indivo
psql
postgres=# \password indivo
postgres=# \q
Create the Database and make the Indivo user its owner:
createdb -U indivo -O indivo indivo
There are two ways to authenticate to PostgreSQL: use your Unix credentials, or use a separate username and password. We strongly recommend the latter, and our instructions are tailored appropriately. If you know how to use PostgreSQL and want to use Unix-logins, go for it, just remember that when you use Apache, it will usually try to log in using its username, www-data.
in /etc/postgresql/9.1/main/pg_hba.conf, find the line that reads:
local all all peer
This should be the second uncommented line in your default config. Alter it to be:
local all all md5
You will need to restart PostgreSQL:
service postgresql restart
More Information¶
See the Django PostgreSQL notes
MySQL¶
Install¶
The Mysql server (only tested with v5.0+):
sudo apt-get install mysql-server libmysqlclient-dev
When prompted, enter a password for the root user.
The MySQLdb python binding (v1.2.1p2 or greater required for django):
sudo apt-get install python-mysqldb
Setup¶
Change the default storage engine to InnoDB, which supports transactions. In /etc/mysql/my.cnf, find the line reading [mysqld]. Directly underneath that line, add:
default-storage-engine = innodb character-set-server = utf8
Then restart mysql with:
sudo service mysql restart
Create the Indivo User by logging in (using the password you set up for the root user during install):
mysql -uroot -p
And then running the following commands:
> CREATE USER 'indivo'@'%' IDENTIFIED BY 'YOURPASSWORD'; # Replace YOURPASSWORD with a password for the new user > GRANT ALL PRIVILEGES ON *.* TO 'indivo'@'%' WITH GRANT OPTION; > exit
Create the Indivo Database:
mysqladmin -u indivo -p create indivo
Authenticating with the password you set up for the indivo user.
Idiosyncracies¶
Date formatting doesn’t work quite the same as it does on the other backends. Specifically:
- “Week of the Year” (00-53), which normally counts weeks as increments of 7 days starting at Jan 01, in mysql counts week 0 as anything before the first Sunday of the year, and after that counts weeks in increments of 7 days, starting on Sunday.
- “Day of the Week”, which is normally indexed from 1 to 7, starting on Sunday, on mysql is indexed from 0 to 6, starting on Sunday.
More Information¶
See the Django MySQL notes
Oracle¶
Install¶
If you do not have a supported installation of Oracle already, the odds are good that you shouldn’t be running Indivo on Oracle. Also, Oracle doesn’t play nicely with Debian Linux, so you also probably shouldn’t be setting it up on Ubuntu. These installation instructions assume that you have a running instance of Oracle on another machine, and describe how to connect to it from an Indivo instance running on Ubuntu.
You’ll need an installation of Oracle against which to bind the Python drivers. You can use Oracle XE (express edition), which is free and based on Oracle 11i. We used the following installation instructions.
Note
These instructions only work for 32-bit Linux. For 64-bit versions, Oracle doesn’t offer a solution.
Get the Python Oracle driver, cx_Oracle, with installation instructions explained here.
Setup¶
Set up your Oracle user on the remote system. From the Django docs, you’ll need to insure that you have access to your Oracle instance as a user with the following privileges:
- CREATE TABLE
- CREATE SEQUENCE
- CREATE PROCEDURE
- CREATE TRIGGER
To run Indivo’s test suite, the user needs these additional privileges:
- CREATE USER
- DROP USER
- CREATE TABLESPACE
- DROP TABLESPACE
- CONNECT WITH ADMIN OPTION
- RESOURCE WITH ADMIN OPTION
Make sure your environment variables are set properly as described in the install instructions for cx_Oracle. Importantly:
- set ORACLE_HOME to the home directory for oracle, /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/ by default
- set LD_LIBRARY_PATH to $ORACLE_HOME/lib
- add $ORACLE_HOME/bin to your $PATH variable.
If you intend on running Indivo on Apache, the Apache user will also need access to these environment variables. You can set this up by editing /etc/apache2/envvars and adding the above variable declarations.
Test that cx_Oracle has been installed. If the following command exits silently, your setup is correct:
python -c "import cx_Oracle"
More Information¶
See the Django Oracle notes
Indivo Server¶
Get the Code¶
From A Packaged Release¶
- Download the latest release of Indivo X from our tags page and untar into indivo_server/. Do not change this directory name–it will break the django settings file.
From Github¶
From the commandline, run:
cd /desired/install/directory git clone --recursive git://github.com/chb/indivo_server.git
If you want to run the stable version of Indivo, you’re done. If you want to use a tagged release, you can list them with:
git tag -n1
and checkout your desired release and update its submodules with:
git checkout {TAGNAME} git submodule init git submodule update
where tagname might be (i.e., for version 2.0) v2.0.0.
Configuration¶
Copy indivo/settings.py.default to indivo/settings.py, and open it up. Make sure to look at the ‘Required Setup’ settings, and examine ‘Advanced Setup’ if you are interested. As an absolute minimum, update the following:
set SECRET_KEY to a unique value, and don’t share it with anybody
set SITE_URL_PREFIX to the URL where your server is running, including port number e.g. https://pchr.acme.com:8443
Edit the ‘default’ database under DATABASES, and:
- set ENGINE to the database backend you are using, prefixed by ‘django.db.backends.’. Supported options are ‘postgresql_psycopg2’, ‘mysql’, and ‘oracle’.
- set NAME to the name you would like to use for your database. If you followed the database setup instructions above, you should leave this as ‘indivo’.
- set USER to the username you chose, in this documentation indivo, and set PASSWORD accordingly.
- If your database is located on another machine, set HOST and PORT appropriately.
- If you are running Oracle, see https://docs.djangoproject.com/en/1.4/ref/databases/#id11 for how to configure the database settings.
If you are running MySQL, add the line SOUTH_TESTS_MIGRATE = False to your settings file
set the SEND_MAIL parameter to True or False depending on whether you want emails actually being sent.
set the EMAIL_* parameters appropriately for sending out emails.
Under utils/ copy indivo_data.xml.default to indivo_data.xml and edit to configure initial accounts, records, and sample data profiles.
Note
Make sure to complete these steps before running the reset script below
Install Required Packages¶
We provide a pip requirements file for Indivo, so you just need to make sure you have pip installed on your system. The easiest way to get pip in Ubuntu 12.04 is to run:
apt-get install python-pip
If you want to read more about other installation methods, please visit http://www.pip-installer.org/en/latest/installing.html
Depending on your database of choice, you will want to edit requirements.txt and make sure you comment out or uncomment the libraries you need. Once you have done this and have pip installed, run:
pip install -r requirements.txt
Resetting the Database¶
MySQL¶
Currently some migrations on our development branch create rows that are too wide for MySQL when using UTF-8, so until we merge these appropriately you will have to perform a few extra steps
turn off South migrations in your settings.py by commenting it out from your INSTALLED_APPS
run:
python manage.py syncdb
turn South migrations back on
run:
python manage.py syncdb python manage.py migrate --fake indivo python utils/reset.py --no-syncdb
Other¶
On other database backends, we don’t yet have reset scripts. You can reset Indivo by:
Flushing the database:
python manage.py flush
Telling South that the database is actually in the correct state after migration:
python manage.py migrate --fake
Importing the initial Indivo data:
python utils/importer.py -v
Loading Coding Systems (optional):
python load_codingsystems.py
You must run reset.py or utils/importer.py before the accounts and records you set up in indivo_data.xml exist.
Database Cleanup¶
Each request made against Indivo Server generates some oauth-related data that is stored in the database for security reasons (for example, session tokens are stored whenever a user logs in, and oAuth Nonces are stored for every request). This data is only relevant for a certain duration (i.e., the length of a web session), after which point it becomes needless clutter in the database. In order to remove all such clutter, from APP_HOME run:
python manage.py cleanup_old_tokens
This command should be set up to run as a cron job, and should be run regularly to make sure the size of the database doesn’t get out of control (we recommend at least once a week, and more frequently for high traffic installations).
Testing Indivo Backend Server¶
Indivo uses the django-tests framework to provide some basic unit and API testing. If you want to make sure everything is setup properly before opening the server up to the network, running these tests is a good start. Django tests set up a clean test database for each run of the tests, so don’t worry about your installation being corrupted. To run the Indivo tests, in APP_HOME run:
python manage.py test indivo
Indivo UI Server¶
Get the Code¶
From a Packaged Release¶
Download the latest release of Indivo X UI Server from our tags page and untar into indivo_ui_server/.
Note
Do not change this directory name–it will break the django settings file.
From Github¶
From the commandline, run:
cd /desired/install/directory git clone --recursive git://github.com/chb/indivo_ui_server.git
If you want to run the bleeding edge version of Indivo, you’re done. If you want to use an official release, you can list releases with:
git tag -n1
and checkout your desired release and update its submodules with:
git checkout {TAGNAME} git submodule update
where tagname might be (i.e., for Version 2.0) v2.0.0
Configuration¶
- Copy settings.py.default to settings.py, and update a few key parameters:
- set SERVER_ROOT_DIR to the complete filesystem path to the location where you’ve installed indivo_ui_server, e.g. /web/indivo_ui_server, with no trailing slash.
- set INDIVO_UI_SERVER_BASE to the URL at which your UI server will be accessible, e.g. http://localhost, with no trailing slash.
- set INDIVO_SERVER_LOCATION, CONSUMER_KEY, CONSUMER_SECRET appropriately to match the Indivo Server’s location and chrome credentials (check indivo_server/utils/indivo_data.xml BEFORE you reset the database on the indivo_server end).
- set SECRET_KEY to a unique value, and don’t share it with anybody
Running Indivo¶
Django Development Servers¶
The Django development servers are easy to run at the prompt. The backend server can run on localhost in the configuration given above:
cd /web/indivo_server/
python manage.py runserver 8000
The UI server, if you want it accessible from another machine, needs to specify a hostname or IP address. If you want port 80, you need to be root of course:
cd /web/indivo_ui_server/
python manage.py runserver HOSTNAME:80
IMPORTANTLY, if you’ve installed Apache, you’ll need to turn it off to run your UI server from the prompt:
/etc/init.d/apache2 stop
Apache¶
Assuming you installed Indivo Server and UI in /web, the steps to getting Apache2 serving Indivo and its UI are:
in /etc/apache2/sites-available/default, add:
<VirtualHost *:8000> ServerAdmin YOU@localhost ServerName localhost DocumentRoot /web/indivo_server Alias /static/ /web/indivo_server/static/ EnableMMAP On EnableSendfile On LogLevel warn <Directory /web/indivo_server> Order deny,allow Allow from all </Directory> WSGIApplicationGroup %{GLOBAL} WSGIScriptAlias / /web/indivo_server/django.wsgi WSGIPassAuthorization On </VirtualHost> <VirtualHost *:80> ServerAdmin YOU@localhost ServerName localhost DocumentRoot /web/indivo_ui_server Alias /static/ /web/indivo_ui_server/ui/static/ EnableMMAP On EnableSendfile On LogLevel warn <Directory /web/indivo_ui_server> Order deny,allow Allow from all </Directory> WSGIDaemonProcess indivo_ui user=www-data group=www-data processes=1 maximum-requests=500 threads=10 WSGIScriptAlias / /web/indivo_ui_server/django.wsgi WSGIPassAuthorization On </VirtualHost>
In our experience, using WSGIProcessGroup directive with a specific group (not global), even when it matches the WSGIDaemonProcess group name (i.e. indivo_ui), can cause a permission issue with reading the Unix socket. We will continue to investigate this issue. However, due to incompatibilities between the lxml package and mod_wsgi, it is necessary to set Indivo Server to the global WSGIProcessGroup instead of running daemons.
Make sure ports.conf has:
NameVirtualHost *:80 Listen 80 Listen 8000
Make sure that www-data (or whoever is in /etc/apache2/envvars) has access to indivo_server and indivo_ui_server AND can write to indivo_server/indivo.log and indivo_ui_server/sessions/*, including the sessions/ directory itself.
Since you probably did a python manage.py syncdb, you almost certainly want to just remove the current indivo.log before you move ahead.
Really, have you checked this www-data permission issue? This will be the cause of all your problems if you don’t check this carefully.
Check your /etc/apache2/sites-enabled/000-default file again and make sure that your Alias /static/ lines match the above example exactly
Restart Apache:
service apache2 restart
What Next?¶
You should be able to log in and add the default apps. These apps are purposely limited in functionality. May the best apps win.