Zebrafish Stock Manager - Initial Deployment
This guide is for the initial deployment of a system. If you are updating a running deployment, please go here
Firewall
Set up your firewall to allow 'SSH' and 'WWW Full' traffic.
Users and Groups
Create an account called dbbackups. It will have minimal privileges.
adduser dbbackups
# then follow the prompts
Create a user and a group called zsm which will be used to run a separate zf-server for each customer. Also, zsm staff will use this account to log in and work on this deployment. It will have sudo privileges, so protect it well - ssh login only.
adduser zsm
# then follow the prompts
usermod -G www-data -g sudo,dbbackups zsm
If you want to create accounts for each developer who needs to work on the system, go ahead and do that. They should be set up just like the zsm user.
For all these users, set up their ~/.ssh/authorized_keys with the public keys of anyone who needs to log in as that user with ssh.
Domain name
You will need a domain name for the deployment as a whole. For the purpose of illustration, this guide will assume you have purchased zsm.com and that you set up DNS records to point at your host's IP address.
It is a good idea to set up test.zsm.com, and demo.zsm.com as an initial subdomains. Later when you are configuring a system for a particular facility, you will be adding one subdomain per facility. If you happen to know that you are going set up managers for facilities eue and acdc, you could create those subdomains too. But you can always do that later. The DNS entries for each subdomain should point to the ip address of your host.
You can get a domain name and set up your DNS at any number of providers like namesilo.com.
Mail service
The system sends mail to users. It uses nodemailer under the covers to do that. You are still responsible for ensuring that you have a mail host you can use to send mail.
You could run your own mail server. Amazon SES is another option that I have used successfully. I found that using a gmail account is quick to set up, but it is not effective in the long run because Google tries to block automated sending.
You want to make sure that your mail host supports STARTTLS when receiving messages from the system. You also want to make sure that the mail host sends messages securely and properly self-identifies (using DKIM, for example), so that receiving mail services will not discard messages from the system. The details are left to you.
You will need to have the following configuration information for your mail host:
- mail host name
- mail host userid
- mail host password (make your password extremely secure, please)
- "from" email address
Setting up the server computer environment
Host setup
Experienced administrators can zip through this guide quickly, but it is a mini site-admin guide for the benefit of less experienced people. In truth, it's just a memory aide for me.
For this guide, we will deploy on a Debian 11(+) computer.
The system can be deployed in other environments in which case, you will need to understand how to perform equivalent operations in that environment.
On Debian, make sure everything is up-to-date:
sudo apt-get update
sudo apt-get upgrade
Node and NPM
We need to use node and the node package manager.
sudo apt install nodejs npm
I noticed that the default node and npm on a fresh Debian 10 machine were a bit behind the times, so I had to upgrade them.
MariaDB
Please refer to MariaDB resources to determine how to install MariaDB in your server environment.
Here is a friendly article for how to install MariaDB securely on Debian . Later, when you go to create individual databases for each zebrafish facility, the guide will assume you have followed the good, secure procedure described there. You will have created an administrative user called admin, which you will use to create databases for each facility. You must remember the password for that administrative user.
Database Backup
You need to back up your databases regularly and copy those backups to a host that is remote from the host that is hosting your database. There are a zillion ways to do this. This is what I did.
Linux user
You already created Linux user "dbbackups" to perform your backups.
Create a ssh key for this user (to allow rsync of backups to another server
using ssh).
Remote Host
On some "backup" host that is geographically separate from your database host, create a user and include the public key for "dbbackups" in that user's .ssh/authorized_keys file. This enables rsync from the database host to the backup host.
Database user
Create a database user with sufficient capabilities to dump all your databases but not to write to them.
create user 'dbbackups'@'localhost' identified by 'good-password-here';
GRANT SELECT, RELOAD, SHOW DATABASES, LOCK TABLES, EVENT ON *.* TO `dbbackups`@`localhost`;
Automysqlbackup
I chose to use automysqlbackup. By default, it gives me exactly what I need. Daily backups rotated every week, weekly backups rotated every 5 weeks and monthly backups that are never rotated.
sudo apt-get install automysqlbackup
Edit the automysqlbackup config file in /etc/default/automysqlbackup to
- tell it about the database user credentials
- change the backup directory (default = /var/lib/automysql) to /home/dbbackups/backup
- edit the configurable command that runs after the backup is complete. The command will simply rsync the backup files to the remote host. The command will look something like this with the obvious substitutions:
'rsync -a /your/chosen/local/backup/dir/ remotebackupuser@remote.backup.host:/your/chosen/remote/backup/dir'
ToDo the --delete option to rsync does not seem to work from within the cron job. It can be run manually once in a while which will clean up the chaff on the backup server.
Schedule Backup
I believe this should be done with a timer and a service in systemd, but I failed, so I used a good old cron job.
- edit /etc/cron.allow to include the dbbackups user
- (Debian specific) create a cron file /etc/cron.d/automysqlbackup which will
run as user dbbackups. Mine looks like this:
SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin 17 22 * * * dbbackups /usr/sbin/automysqlbackup
An alternate solution to all of this is to have automysqlbackup run remotely, but that has other attendant problems.
Web Server
The system uses a web server to:
- serve the zf-client to the end user
- serve zebrafish facility-specific configuration to the zf-client
- handle HTTPS traffic between each zf-client and the correct facility-specific zf-server
Apache2 is usually pre-installed on Linux servers, but just in case, here is friendly article on how to install Apache on Debian
Set up a virtual host for your domain (in this case zsm.com). You can put anything you want in the root directory, it is a base website you can use to welcome folks. You might want to put the user documentation there or a demo system or whatever you like.
You will need to set up a separate virtual host for each zebrafish facility the system manages. It is a good idea to set up an example one now. The process of setting up a virtual host for a facility is covered here.
Make sure that the rewrite, proxy and proxy_http modules are enabled for Apache.
zf-server and zf-client deployment
There a many ways to do the following this is just how I do it. It is done with an eye to making updates easier later on.
# For the initial deployment, create a working directory like /var/www/zsm
# where you plan to clone and build the code.
# You probably wont have write privilege so sudo
cd /var/www
sudo mkdir zsm
#change the permissions to you work here
sudo chown -your-username- zsm
sudo chgrp -your-usergroup- zsm
cd zsm
Clone the GitHub repository
This is where you download all zf-client and zf-server code. Oopsy, it's a private repo now and you need ssh access to it. You can get it through me, ted.
# For example:
cd /var/www/zsm/
git clone git@github.com:tmoens/zebrafish-facility-manager.git
zf-server build
# The server uses the NestJS framework. The NestJS cli is required to build the server.
npm install -g @nestjs/cli
# navigate to the zf-server sub-directory
cd /var/www/zsm/zebrafish-facility-manager/zf-server
# Download npm packages
npm install
# Build the zf-server
npm run build
This generates a "dist" directory containing the main server executable main.js.
zf-client build
# The client uses the Angular Framework, install the Angular cli
npm install -g @angular/cli
# navigate to the the zf-client sub-directory
cd /var/www/zsm/zebrafish-facility-manager/zf-client
# Download npm packages
npm install
# Build the zf-client
ng build --configuration=production
The result of this operation will be a directory called /dist/zf-client. We suggest you make a copy of this directory for deployment. That way you are able to tweak and rebuild the zf-client without affecting the users.
# go to the "distribution" client directory
cd /var/www/zsm/zebrafish-facility-manager/zf-client/dist
# copy it to a well known place
sudo cp -R zf-client /var/www/
Facility Configuration
You are finally ready to configure the system to manage a zebrafish facility. Here is the Facility Configuration Guide.