Installation/Appliance/Howto

Installation

Work with Openbravo

Configuration of appliance

Howto / Optional configuration

Migration to new appliances

This document refer to appliances 12.04, 14.04 and 18.04.

Login by ssh

For EC2 look here.

From Linux

ssh openbravo@<IP_ADDRESS>

Default password: openbravo

From Windows

1. Open putty
2. In Session: hostname=<IP_ADDRESS>
3. Click Open connection
4. Enter the username, by default = openbravo
5. And the password, by default = openbravo

Login by ssh in EC2

From Linux

ssh openbravo@<IP_ADDRESS> -i <ec2-key>.pem

From Windows

1. Open putty
2. In Session: hostname=<IP_ADDRESS>
3. Connection/Data: Auto-login username=openbravo
4. SSH/Auth: Private key file for authentication = Select the file with your private key
5. Click Open connection

Create ssh key

In Linux

ssh-keygen

Generating public/private rsa key pair.
Enter file in which to save the key (/home/<your_user>/.ssh/id_rsa):
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/<your_user>/.ssh/id_rsa.
Your public key has been saved in /home/<your_user>/.ssh/id_rsa.pub.
The key fingerprint is:
xxxxxxxxxxxxxxxxxxxxxxx

The results of this command:

• The private key: /home/<your_user>/.ssh/id_rsa
• The public key: /home/<your_user>/.ssh/id_rsa.pub

In windows

1. Open puttygen.
2. Select at bottom "SSH-2 RSA" and "Number of bits": 2048
3. Click on generate a new one.
4. Click on save private key (id_rsa.ppk).
5. Click on save public key (id_rsa.txt).
6. Select the text in the box "Public key for pasting into OpenSSH authorized_keys file" copy it and store in a file .pub (id_rsa.pub)

Importance of these keys

These keys can be the only way to access to your server, so ensure that you save it in a good place and better to have backups of them.

Add ssh keys to the appliance

The new Appliance 18.04 will no longer accept the deprecated ssh-keys of type DSA (ssh-dss in authorized_keys). If you still have a ssh-key of that type you need to make a new one as written below.

If you don't have a ssh key follow this guide.

From linux

ssh-copy-id openbravo@<IP_ADDRESS>

Alternatively it is possible to manually append the contents of your public key (usually .ssh/id_rsa.pub) to the file /home/openbravo/.ssh/authorized_keys of the appliance.

From windows

1. Load in puttygen your public key
2. Copy the contents of "Public key for pasting into authorized_keys file"
3. Open with putty a connection to the Openbravo appliance
4. Open the file to add the key
   • nano /home/openbravo/.ssh/authorized_keys
5. Paste the key in the file, in putty you can use to paste the right button of the mouse.
6. Press 'Ctrl + o' , to save in nano, press 'Enter' to use the same filename.
7. Press 'Ctrl + x' to exit from nano
8. Important: before you close the ssh connection try connect with the public key that you have already added, with this in case you did something wrong you still have the old sh connection to fix it.
9. To test the connection with public key, connect with putty and specify:
   • Session: hostname=<IP_ADDRESS>
   • Connection/Data: Auto-login username=openbravo
   • SSH/Auth: Private key file for authentication = Select the file with your private key
   • When opening the connection it will ask the password of your public key
10. Once that it works you can close the connections

Setup ssh with only public key

The appliances of EC2 already has ssh only with public key, so no needed to do this steps on it.

IMPORTANT: Add your public to the instance, before enable access by only public key.

Test ssh with public key in linux

ssh -v openbravo@<IP_ADDRESS>

With '-v' it will show that it is using the public key and will ask for the password and will allow to connect.

Check that when doing the previous ssh with debug (-v) show a line with this:

debug1: Authentication succeeded (publickey).

Test ssh with public key in windows

To test the connection with public key, connect with putty and specify:

1. Session: hostname=<IP_ADDRESS>
2. Connection/Data: Auto-login username=openbravo
3. SSH/Auth: Private key file for authentication = Select the file with your private key
4. When opening the connection it will ask the password of your public key

Change to only allow ssh with public key

VERY IMPORTANT: Check that the access with public key it is working, before remove the ssh with password !!

After login by ssh, copy paste this command and run it:

sudo sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config

Restart ssh to apply the changes

sudo /etc/init.d/ssh restart

Check to make a new connection before close the current connection.

Check that only it is enabled ssh with public key in linux

Doing ssh with '-v' it should show a line with:

debug1: Authentications that can continue: publickey

And should not appear password in the options.

Memory

The very minimum needed memory to run the appliance is 1.5GB, in order to adjust to this very low memory tomcat is configured with 1GB and postgres with 128MB. This values works for few concurrent users and small/medium databases.

If you plan to go production you probably need to increase this values, but there is not a defined values that works for all, so you need to adjust the memory settings depending of your needs.

Before increase the memory, check that you have have enough memory on the system.

Remember to not assign all the memory of the system to tomcat and postgres, and leave some ram for the operating system and some more free that will be use for caching that will help to improve the performance.

Tomcat memory

It is recommended to increase the memory assigned to Tomcat when:

• You have many concurrent users
• When you see in the logs of tomcat (/var/log/openbravo/catalina.out) errors like "java heap out of memory" or "gc overhead limit exceed".

A tip to setup the correct value:

• To see if the current value it is enough or not, you can use tools like jmap to know the usage inside the java heap, for more details take a look to the jps and jmap topics in the monitoring section.

To increase the tomcat memory edit the file /etc/profile.d/tomcat.sh

sudo nano /etc/profile.d/tomcat.sh

Increase the Xmx parameter, for example for set 2048 MB to tomcat, Change:

-Xmx1024M

With

-Xmx2048M

If you give to tomcat more memory than the one that the system has free it will show a "can not allocate memory" or "The size of the object heap + VM data exceeds the maximum representable size" error in the log (/var/log/openbravo/catalina.out).

Postgres memory

It is recommended to increase the memory of postgres when:

• You have a big database or there is tables with high numbers of rows.

A tips to which value you need for postgres memory could be:

• If you have enough ram, you can start with a initial value that was a bit bigger that the size of the database. To see the size of the database you can connect to the db and list all the databases with their sizes, look to the openbravo database:

psql -h localhost -U tad openbravo
\l+
\q

• To fine tuning for a specific slow query, identify the sql that takes more time and resources to run, then increase the postgres memory to a value like 256MB, if the sql runs in less time continue increasing to 512MB, 1GB, etc.
• When you see that increasing the memory don't help to reduce the sql execution time, then this could be a possible value.

To increase postgres memory edit the postgresql.conf file

sudo nano /etc/postgresql/9.1/main/postgresql.conf              # Appliance 12.04
sudo nano /etc/postgresql/9.3/main/postgresql.conf              # Appliance 14.04
sudo nano /etc/postgresql/10/main/conf.d/91-local-custom.conf   # Appliance 18.04

Edit the shared_buffers, for example to set 1GB:

shared_buffers = 1GB

Note: if you want to set one or more gigabytes you can specify in GB instead of MB. GB does not accept decimals so if you are not going to use integer numbers you need to use MB

The following about shmmax only applies to old Appliance 12.04 with postgres < 9.3.

Increasing the shmmax parameter of the kernel.

Take into account that there is not only postgres which uses the shm memory, so set this value to a higher value that the one that you have set for postgres.

For example if you set 1GB for postgres you can add 256MB for the rest of the system. The value should be set in bits, you can calculate it taking 1GB plus the 256MB and multiplying the result for 1024 two times.

You can use this command to calculate, replace the first 1024 with the MB set to postgres and the 256 with the MB set for the system

echo $(( (1024 + 256) * 1024 * 1024 ))

Edit the /etc/sysctl.d/40-openbravo.conf

sudo nano /etc/sysctl.d/40-openbravo.conf

Set the shmmax parameter with the previously calculated value, for example with 1GB for postgres and 256MB for system will be:

kernel.shmmax = 1342177280

This value will be loaded in the next system start, for load this value without need to reboot:

sudo sysctl -p /etc/sysctl.d/40-openbravo.conf

To get the new values is needed to restart postgres, but remember that before stop postgres it is needed to stop tomcat

sudo /etc/init.d/tomcat stop
sudo /etc/init.d/postgresql restart
sudo /etc/init.d/tomcat start

If you give more to postgres more memory than the one that the system has free it will show a "requested shared memory size overflows size_t" error.

Swap

The swap it is a file used by the system when the RAM it is not enough so move some data to disk and when needed retrieve it to RAM.

When the system needs contiguously more memory that RAM and uses intensively the swap, the system performance falls down. But if the system goes out of memory (RAM + swap) the kernel will start killing process, and in general it is no good kill a process in a hard way, but if this process are the Postgres ones this can have a very bad end. This can be seen in dmesg with a 'Out of memory: kill process [...]'.

The recommendation is configure the memory of tomcat and postgres so they don't need to use the swap that will drop down the performance. But have enough swap that in a special situation the system uses the swap and don't end with the kernel killing processes.

Execute this to create a swap of 2048 MB stored in /var/swap:

dd if=/dev/zero of=/var/swap bs=1M count=2048
mkswap /var/swap
chown root:root /var/swap
chmod 0600 /var/swap
swapon /var/swap

Note: this will be lost in the next reboot, but you can add to /etc/rc.local before the 'exit 0' so it is executed in each boot.

In Amazon EC2 the root disk / it is small but usually there is an ephemeral disk mounted in /mnt with plenty of space. Also this ephemeral is local to the instance and is quicker than /, but it is deleted on reboot. This ephemeral it is a good option to place the swap:

dd if=/dev/zero of=/mnt/swap bs=1M count=2048
mkswap /mnt/swap
chown root:root /mnt/swap
chmod 0600 /mnt/swap
swapon /mnt/swap

Note: this will be lost in the next reboot, but you can add to /etc/rc.local before the 'exit 0' so it is executed in each boot.

Network

DEVICE_NAME: use command 'ip a' to know the name of device to be configured.

Configure network with dhcp

Open the network config file

sudo nano /etc/netplan/01-network-manager-all.yaml

To use DHCP with Netplan, the configuration file would look something like this:

network:
   version: 2
   renderer: networkd
   ethernets:
      DEVICE_NAME:
         Addresses: []
         dhcp4: true
         optional: true

Test the Configuration

This command will validate the configuration before applying it. Netplan should succeed and apply the DHCP configuration. You could then issue the ip a command, get the dynamically assigned address

sudo netplan try

Apply the New Configuration

sudo netplan apply

Configure network with static ip

Open the network config file

sudo nano /etc/netplan/01-network-manager-all.yaml

network:
   Version: 2
   Renderer: networkd
   ethernets:
      DEVICE_NAME:
         Dhcp4: yes/no
         Addresses: [IP/NETMASK]
         Gateway: GATEWAY
         Nameservers:
            Addresses: [DNS-1, DNS-2]

Test the Configuration

This command will validate the configuration before applying it. If it succeeds, you will see Configuration accepted.

sudo netplan try

Apply the New Configuration

sudo netplan apply

EC2 Static IPs - Elastic IPs

Elastic IP addresses are static IP addresses that you can dynamically assign to the instance you choose. This is very useful for fast instance recovery. Example scenario:

• If your instance has a problem, you can launch a new instance from a backup.
• But your new instance will have a different IP address, and with an Elastic IP you can assign your old instance's IP address to this new one, in a matter of seconds.

To allocate a new static IP click on the Elastic IP item in the left navigation menu. And then select click on Allocate new Address.

File:Aws-console-elastic-ip-new.png

You can then associate this IP address to the instance you choose, by right clicking in that IP address:

File:Aws-console-elastic-ip-associate.png

WARNING: using Elastic IPs is free or charge, as long as the IPs are associated to an instance. Otherwise you'll be penalized with an hourly fee.

Change hostname

Note: the hostname should be added to the /etc/hosts to avoid undesired side effects.

Edit the file /etc/hostname

sudo nano /etc/hostname

Change the current value with the new one.

Load the configuration the file

sudo hostname -F /etc/hostname

Add the hostname to /etc/hosts

sudo nano /etc/hosts

Edit the line

127.0.0.1	localhost

Add at the end a space and the new hostname

127.0.0.1	localhost newhostname

Apache

The folder with the Apache configuration is /etc/apache2

Restart Apache

Usually not needed to restart it

sudo /etc/init.d/apache2 restart

If you only want apache to take new configuration, sometimes it is enough doing a reload

sudo /etc/init.d/apache2 reload

Monitoring

Tools of monitoring

This guide is not intended to give a full tutorial of how to monitor a linux server, but to provide a simple list of the typical tools that are used, in order to give a start point.

Real time

Top, displays the processes that use more cpu on the system:

top

Note: press 'q' to exit, for more info about top run 'man top'.

With top can be seen the machine load (at top right) or if the disk it is highly used (if the waiting percentage it is continuously high) and the cpu and ram used by each process.

Ps, list process, a typical call:

ps faux

Also it is posible to redirect the output to less so it is more easy to read:

ps faux | less

Note: to exit from less press 'q', you can use the arrows to navigate.

Free, list memory usage

free -m

Note: Total memory it is 'Mem' row and 'total' column . Free memory it is 'buffers/cache' row and 'free' column.

A more advance topic it is to see the memory usage inside tomcat:

You can use jps to list the java processes:

jps

The tomcat processes are named with 'Bootstrap', select the pid and you can use jmap to see the heap configuration:

sudo jmap <bootstrap_pid>

You can see here the current heap size (the sum of the capacities of eden, from, to and ps old) and the max size of the heap (MaxHeapSize).

When the current heap size it is the same as the max heap size and all the space at usage are near 99% specially the ps old, then you should consider increasing the maximum heap size, take a look to memory guide.

With jmap also can be monitored the current permgen size (PS Perm Generation, capacity), the max permgen size (MaxPermSize) and the usage of the current perm (PS Perm Generation, % used). Normally with 256MB it is not needed to increase it in almost any situation, you can see that usually the usage of the perm it is 99% but the size has not expand to the max allowed.

Monitoring services

There is programs like munin that collect statistics along time of the cpu usage, system load, database, memory , etc.

This kind of tools helps to analize when the system it is more used, and help to decide if it is needed a server with more performance or in the other side if the current server it is able to handle all the load.

To be documented how to install and configure it.

There is other programs like nagios that can be configured to send email notifications when the server is down, or the disk it is almost full, the system load it is very high, etc.

To be documented how to install and configure it.

Emails notifications

The system can be configured to send email notifications when a scheduled backup has failed or in other situations.

To be documented how to configure it.

Screen

Screen is a program that creates a virtual console, it is very useful when executing commands through ssh that takes long time to execute.

Lets put the example that you want run a restore of a big database dump of many gigabytes, this process can take many hours. If you run this through ssh and the ssh gets close because a micro network cut, then you need to restart the process from the beginning.

With screen you can execute a process, close your ssh, and connect again by ssh and re-attach to the screen virtual console.

Lets see the basic screen commands. To create a new screen:

screen

Press 'space' or 'enter' to close the welcome message.

Now you can execute any process inside the screen.

You can deattach from the screen with:

ctrl + a  d

Note: press 'ctrl' and 'a' together, then release and press 'd'.

You can close even you ssh connection, and connect again.

For attach again to the screen:

screen -r

To list the screens:

screen -ls

To terminate a screen, the same that a normal console: being inside the screen execute 'exit'.

Updates

Normal minor updates

Remember that if there are updates for PostgreSQL or Tomcat those services will be restarted, so do those updates when nobody is working on the server.

sudo apt-get update
sudo /etc/init.d/tomcat stop
sudo apt-get dist-upgrade
sudo reboot   # recommended, to be sure all affected services are restarted

Notes about updates:

• If there is kernel updates you must reboot the server in order to use the new kernel.
• Kernel updates: In case of updates for the linux kernel a dialog will show related to grub configuration. You should use the the new / maintainer config files in this case to make sure that the newly installed kernel version will be used after the reboot.
• Reboot after updates: if you are unsure of what services you need to restart after update or don't know what are the kernel packages, a recommendation it is to reboot the server after an update.

Major release upgrade

The command that perform a major release upgrade it is 'do-release-upgrade'

Do 'NOT' execute this command !!

Direct upgrades from Appliances 12.04 to 14.04 or 14.04 to 18.04 are NOT supported. Instead do a migration.

Date and Time

Set date and time

It is not a good idea to change the date or time of the server, this usually leads to undesired side effects.

If you want you server to have the time updated use NTP, this program will do small update of the time in order to try to not affect the rest of processes.

If you are in a different time zone, change the timezone of the server, this updates all the dates of the system (db, files, etc) so it will cause less problems that change date or time of the server.

Changing the date of the server can produce really undesired effects, including Openbravo stop working and data corruption. So it is highly discourage to change the date or time of the system.

If you still want to change the date or time on your own risk, you can use this command:

sudo date nnddhhmmyyyy

• nn is a two digit month, between 01 to 12
• dd is a two digit day, between 01 and 31, with the regular rules for days according to month and year applying
• hh is two digit hour, using the 24-hour period so it is between 00 and 23
• mm is two digit minute, between 00 and 59
• yyyy is the year; it can be two digit or four digit: your choice. I prefer to use four digit years whenever I can for better clarity and less confusion
• (optionaly) ss is two digit seconds. It is added at the end with a period ‘.’ before the ss. (nnddhhmmyyyy.ss)

Verify if change worked correctly: both of the following commands should show the same time matching the time change

date   # system date and time
psql -h localhost -U tad openbravo -c 'select now()'   # postgres date and time

Configure NTP

NTP is a TCP/IP protocol for synchronising time over a network. Basically a client requests the current time from a server, and uses it to set its own clock.

To install it:

sudo apt-get install ntp

NTP by default will do only small time adjustments, so big time differences can take some time to synchronize.

Halt and reboot

Running a halt in EC2 depending of the instance configuration can cause the termination without undo option of the instance.

To halt the system and power down the system run

sudo halt -p

To reboot the system

sudo reboot