devstack ocata on ubuntu 16.04 setup

The article below is more like a memo to me about how to fast setup DevStack based private cloud on Ubuntu 16.04. It might take several hours or days solving all kind of issue related to different part of OpenStack and system configuration if start process without good stack knowledge. Or detailed step-by-step notes from previous successful installation. I hope it will save some hours for me next time.

Install Ubuntu 16.04

Create bootable USB drive from ubuntu iso

wget http://ubuntu.volia.net/ubuntu-releases/xenial/ubuntu-16.04.3-server-amd64.iso
dd if=ubuntu-16.04.3-server-amd64.iso of=/dev/sdb

Install server minimum (+ ssh server)

Remove virtualization tools if accidentally chosen

sudo apt-get remove --purge qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils

Update all components

sudo su -
apt update
apt upgrade

 Disable IPv6

cat <<EOF >>/etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.all.rp_filter=0
net.ipv4.ip_forward=1
net.ipv4.conf.all.proxy_arp = 1
net.ipv4.conf.all.proxy_arp_pvlan = 1
EOF
sysctl -p
 Switch to static IP
vim /etc/network/interfaces
# The primary network interface
auto enp3s0
iface enp3s0 inet static
    address 10.9.8.254
    netmask 255.255.255.0
    gateway 10.9.8.241
    dns-search local
    dns-nameservers 10.9.8.241
 visudo
# User privilege specification
root    ALL=(ALL:ALL) NOPASSWD:ALL
 
# Members of the admin group may gain root privileges
%admin ALL=(ALL) NOPASSWD:ALL
 
# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) NOPASSWD:ALL

 Add ssh key to user(s)

mkdir .ssh
chmod 700 .ssh
echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCv1hNcFg6w3pOLPDdoc+WdBegk1a9VspjV9MaHTeJm6W1InpyBsSEzg/WUeQT9qfIin8dmjX8tRY9clGsp8xzrpiT2I/3mN8dgLvCfpWlNYNUkIPL5LlerbUmm9N0bnTm8OUpykBY8QPU2xJH9SB4fUDCOQ9kQOuoS/0CKNlainXQCrKpt50BHC4QdSsVwemN5Gqj5O2LqzAQGZBfykuY6WJU1ab/0YmgOcwb6QNP/KuI+biceZr3B49rroCN1/YWVCDVd6j7Wu898a5vh3qsQ0q/acBKrI7mdix+Lbc+TJfKfWVl89wAFOCmswMJku+nhj+kt5STDFklvRAFnuPg/ oliutyi@server' >~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

 reboot

reboot

Devstack setup

 Clone DevStack

cd /opt
git clone https://git.openstack.org/openstack-dev/devstack
cd devstack/
git checkout stable/ocata

 Create stack user

/opt/devstack/tools/create-stack-user.sh
chown stack:stack /opt/devstack/ -R

Create config

vi /opt/devstack/local.conf

 Minimal configuration

[[local|localrc]]
FLOATING_RANGE=10.9.8.224/28
FIXED_RANGE=10.11.12.0/24
FIXED_NETWORK_SIZE=256
FLAT_INTERFACE=enp3s0
ADMIN_PASSWORD=password_1
DATABASE_PASSWORD=password_2
RABBIT_PASSWORD=password_3
SERVICE_PASSWORD=password_4

 More settings

[[local|localrc]]
IP_VERSION=4
HOST_IP=10.9.8.254
FLAT_INTERFACE=enp3s0
FLOATING_RANGE=10.9.8.224/28
IPV4_ADDRS_SAFE_TO_USE=10.9.9.0/24
FIXED_RANGE=10.11.12.0/24
FIXED_NETWORK_SIZE=256
ADMIN_PASSWORD=password_1
DATABASE_PASSWORD=password_2
RABBIT_PASSWORD=password_3
SERVICE_PASSWORD=password_4
VOLUME_GROUP_NAME="stack-volumes"
VOLUME_NAME_PREFIX="volume-"
VOLUME_BACKING_FILE_SIZE=100G

 start installation

su - stack
cd /opt/devstack/
./stack.sh

Finish

=========================
DevStack Component Timing
=========================
Total runtime 749

run_process 55
test_with_retry 3
apt-get-update 2
pip_install 72
restart_apache_server 14
wait_for_service 13
apt-get 13
=========================

This is your host IP address: 10.9.8.254
Horizon is now available at http://10.9.8.254/dashboard
Keystone is serving at http://10.9.8.254/identity/
The default users are: admin and demo
The password: password_1
2017-10-04 23:56:18.294 | WARNING:
2017-10-04 23:56:18.294 | Using lib/neutron-legacy is deprecated, and it will be removed in the future
DevStack Version: ocata
Change: 8cb43d098ca88ce57ec216423c009cccd07f1299 Use "pip list" in check_libs_from_git 2017-10-02 11:09:56 -0400
OS Version: Ubuntu 16.04 xenial

2017-10-04 23:56:18.295 | stack.sh completed in 749 seconds.

 

Connect storage

apt install nfs-common
mkdir /storage
mount.nfs storage.local:/volume1/backup /storage

Import Images

http://10.9.8.254/dashboard/project/api_access/

export OS_AUTH_URL=http://10.9.8.254/identity/v3
export OS_PROJECT_ID=7da2ddbd9df5404da160a42eb7594ff2
export OS_PROJECT_NAME="demo"
export OS_USER_DOMAIN_NAME="Default"
unset OS_TENANT_ID
unset OS_TENANT_NAME
export OS_USERNAME="admin"
export OS_PASSWORD=Password1
export OS_REGION_NAME="RegionOne"
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3
glance image-create --name DB --min-disk 5 --min-ram 1 --disk-format qcow2 --container-format bare --progress --file /storage/openstack/db-2017-10-01.qcow2;
glance image-create --name HOME --min-disk 5 --min-ram 1 --disk-format qcow2 --container-format bare --progress --file /storage/openstack/home-2017-10-01.qcow2;
glance image-create --name NOC --min-disk 5 --min-ram 1 --disk-format qcow2 --container-format bare --progress --file /storage/openstack/noc-2017-10-01.qcow2;
glance image-create --name JIRA --min-disk 5 --min-ram 1 --disk-format qcow2 --container-format bare --progress --file /storage/openstack/jira-2017-10-01.qcow2;
glance image-create --name WIKI --min-disk 5 --min-ram 1 --disk-format qcow2 --container-format bare --progress --file /storage/openstack/wiki-2017-10-01.qcow2;

Compress qcow2 images

prepare

before image creation important to fill empty disk spaces with zeroes

# 70GB disk space zeroed by
dd if=/dev/zero of=/zerofile bs=1M count=70k
rm /zerofile
# zero to swapfile (or partition)
swopoff -a
dd if=/dev/zero of=/swapfile1 bs=1K count=1M
mkswap /swapfile1
swapon -a

compress

qemu-img convert -O qcow2 -c /storage/openstack/wiki-2017-10-08.qcow /storage/openstack/wiki-2017-10-08-zip.qcow

result

-rw-r----- user users 8.0G Oct 8 13:22 wiki-2017-10-08.qcow
-rw-r--r-- user users 5.5G Oct 8 13:36 wiki-2017-10-08-zip.qcow

Create Floating IP

openstack subnet list
PUBLICSUBNET=a9de3987-7954-4c2a-811d-87d02af4ed91
IPS="10.9.8.225 10.9.8.226 10.9.8.227 10.9.8.228 10.9.8.229 10.9.8.230 10.9.8.231"
for IP in ${IPS}; do
openstack floating ip create --floating-ip-address ${IP} ${PUBLICSUBNET}
done
openstack floating ip list

Open firewall on OpenStack

nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default tcp 80 80 0.0.0.0/0
nova secgroup-add-rule default tcp 443 443 0.0.0.0/0
nova secgroup-add-rule default tcp 8080 8080 0.0.0.0/0
nova secgroup-add-rule default tcp 8090 8090 0.0.0.0/0
nova secgroup-add-rule default tcp 10051 10051 0.0.0.0/0
nova secgroup-add-rule default tcp 10050 10050 0.0.0.0/0

Launching instances

nova image-list
nova flavor-list
nova boot --flavor 2 --image DB db
nova boot --flavor 2 --image HOME home
nova boot --flavor 2 --image NOC noc
nova boot --flavor 3 --image JIRA jira
nova boot --flavor 4 --image WIKI wiki
nova list

Assign floating IPs

nova floating-ip-associate db 10.9.8.226
nova floating-ip-associate home 10.9.8.228
nova floating-ip-associate noc 10.9.8.227
nova floating-ip-associate jira 10.9.8.231
nova floating-ip-associate wiki 10.9.8.230

Change Application settings

Change DB settings for Confluence

vi /var/atlassian/application-data/confluence/confluence.cfg.xml
/etc/init.d/confluence stop
/etc/init.d/confluence start

Change DB settings for JIRA

vi /var/atlassian/application-data/jira/dbconfig.xml
/etc/init.d/jira stop
/etc/init.d/jira start

Postgres network check

 vi /etc/postgresql/9.3/main/pg_hba.conf

Zabbix DB settings

 vi /etc/zabbix/zabbix_server.conf
 vi /etc/zabbix/web/zabbix.conf.php

Nginx

vi /etc/nginx/sites-enabled/zabbix
vi /etc/nginx/sites-enabled/wiki
vi /etc/nginx/sites-enabled/jira
service nginx restart

Restore DevStack instances after unplanned server restart

Backup failed stack data

cp -R /opt/stack/data /storage/openstack/failed-stack

Setup stack (with data lost)

re-install OS might not be needed, sometimes it is enough to run stack again:

su - stack
cd /opt/devstack/
./stack.sh
  • All steps from planed restore. Use available (old, or clear OS) images to re-create virtual machines with the same flavor and disk size.
  • Stop virtual machines.
  • Exchange disks in /opt/stack/data/nova/instances with disks from backup
rm -f /opt/stack/data/nova/instances/_base/*
cp /storage/openstack/failed-stack/data/nova/instances/_base/* /opt/stack/data/nova/instances/_base/
cp -f cp /storage/openstack/failed-stack/data/nova/instances/${OLD_VM_ID}/*  /opt/stack/data/nova/instances/${NEW_VM_ID}/

Automation1

for name in db noc wiki jira home;do ln -s /opt/stack/data/nova/instances/`sudo grep root@$name /opt/stack/data/nova/instances/*/console.log |cut -d/ -f7| sort| uniq` /opt/stack/data/nova/instances/$name; done 
for name in db noc wiki jira home;do ln -s /storage/openstack/failed-state/nova/instances/`sudo grep root@$name /storage/openstack/failed-state/nova/instances/*/console.log |cut -d/ -f7| sort| uniq` /storage/openstack/failed-state/nova/instances/$name; done 
for name in db noc wiki jira home;do echo $name; sudo cp -f /storage/openstack/2018-01-31-failed-state/nova/instances/$name/disk /opt/stack/data/nova/instances/$name/disk; sudo chmod 644 /opt/stack/data/nova/instances/$name/disk; sudo chown root:root /opt/stack/data/nova/instances/$name/disk; done
  • Start VMs
  • See Application re-setup (for planned restore instruction)

Add SSH Key to be used for new instances created from public cloud images

ssh-keygen
openstack keypair create --public-key /opt/stack/.ssh/id_rsa.pub keyname
nova boot --key-name keyname --flavor 2 --image Ubuntu db2

Leave a Reply

Your email address will not be published. Required fields are marked *