Cocktail Private Hub Server Installation

1.Upload package to harbor server Upload files to harbor server using sftp or utility such as FileZilla. Directory structure should be like as follow:

/root/
├── cocktail
│   ├── cert
│   │   ├── docker                  // Shell script file for generating docker certification files.
│   │   ├── harbor                  // Shell script file for generating harbor certification files.
│   │   └── openssl.cnf             // OpenSSL configuration file.
│   └── harbor                      // harbor root directory.
│       ├── LICENSE
│       ├── NOTICE
│       ├── common
│       ├── docker-compose.clair.yml
│       ├── docker-compose.notary.yml
│       ├── docker-compose.yml
│       ├── harbor.cfg
│       ├── harbor.v1.3.0.tar.gz
│       ├── harbor_1_1_0_template
│       ├── install.sh
│       ├── prepare
│       └── upgrade
├── cube
│   ├── cert
│   │   ├── id_rsa                     // SSH private key. Just for convenience.
│   │   └── id_rsa.pub                 // SSH public key. Just for convenience.
│   ├── cube.yaml
│   ├── cube                          // cube executable file.
│   └── cubescripts                    // cube scripts root directory
│       ├── README-Bearmetal.md
│       ├── README.md
│       ├── addon.yml
│       ├── ansible.cfg
│       ├── cert.yml
│       ├── cluster.retry
│       ├── cluster.yml
│       ├── cocktail.yml
│       ├── cube.log
│       ├── group_vars
│       ├── inventories
│       ├── node.yml
│       ├── pre-destroy.yml
│       ├── provision
│       ├── reset.retry
│       ├── reset.yml
│       ├── roles
│       └── scripts
└── software
    ├── docker-ce-17.03              // docker ce rpm file
    │   ├── docker
    │   ├── docker-compose
    │   ├── docker-containerd
    │   ├── docker-containerd-ctr
    │   ├── docker-containerd-shim
    │   ├── docker-init
    │   ├── docker-proxy
    │   ├── docker-runc
    │   ├── dockerd
    │   └── service
    └── pods                        // docker images root directory.
        ├── cocktail_2.1.1           // cocktail images
        ├── kube                   // kubernetes images
        ├── monitoring              // Prometheus, AlertManager, nodeExport, Grafana
        ├── prepare_image.sh      // Shell script for saving images to local
        ├── test                     // Test images
        └── upload_image.sh       // upload script to harbor

2.Stop firewall, change selinux option and reboot

// Changes as root user
# sudo su - root

// Stop firewall
# systemctl stop firewalld
# systemctl disable firewalld

// Changes selinux to disabled
# vi /etc/sysconfig/selinux
SELINUX=disabled

# reboot

3.Create harbor and docker private certification files

  • Add harbor ip address to /root/cocktail/cert/openssl.conf
# vi /root/cocktail/cert/openssl.cnf
[ req ]
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_ca ]
basicConstraints = critical, CA:TRUE
keyUsage = critical, digitalSignature, keyEncipherment, keyCertSign

[ v3_req_server ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names_harbor

[ v3_builder_server ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names_builder

[ v3_builder_client ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth

[ alt_names_harbor ]
DNS.1 = localhost
**IP.1 = 172.30.1.56 // registry server IP**
IP.2 = 127.0.0.1

[ alt_names_builder ]
DNS.1 = localhost
**IP.1 = 172.30.1.56 // registry server IP**
IP.2 = 127.0.0.1
  • Create harbor and docker certificate files
// Creates harbor certificate
# chmod 755 /root/cocktail/cert/harbor/make_harbor_cert.sh
# /root/cocktail/cert/harbor/make_harbor_cert.sh

// Creates docker certificate
# chmod 755 /root/cocktail/cert/docker/make_docker_cert.sh
# /root/cert/cocktail/docker/make_docker_cert.sh {registry_ip}

4.Copy docker and cube binary to /usr/bin and verify

# cp /root/software/docker-ce-17.03/docker* /usr/bin
# cp /root/cube/cube_linux_x.x.x /usr/bin/cube

# docker&
# fg
# ctrl+C

# docker-compose version
# cube version

# cp /root/software/docker-ce-17.03/service/docker.service /etc/systemd/system/

# systemctl enable docker; systemctl start docker

# logout and login
# docker info
# docker ps

5.Harbor Setting and startup

  • Modify Harbor.cfg and install harbor
# cd /root/cocktail/harbor
# vi harbor.cfg
...
hostname = {harbor_ip}
ui_url_protocol = https
ssl_cert = /root/cocktail/cert/harbor/harbor.crt     // harbor crt file path
ssl_cert_key = /root/cocktail/cert/harbor/harbor.key // harbor certificate key file path
harbor_admin_password = C0ckt@1lAdmin
...

# ./install.sh
... output harbor installing ...

// Verifies whether Harbor started up normally. (Normal if 7 containers in total are running)
# docker ps
  • Visit harbor site and create project

Run bowser and connect to https://{harbor_ip}. You should be login successfully as id/password in harbor.cfg and make projects (google_containers, monitoring, cocktail, test) which will be used to upload docker images later

  • Verifying whether Harbor login via Docker client is successfully.

Copy the ca.crt file created on the Harbor server to /etc/docker/certs.d/{harbor_ip}/ where the Docker client verifies the certificate.

Important! Folder creation must follow the format shown below.

# mkdir -p /etc/docker/certs.d/{harbor_ip}
# cp /root/cocktail/cert/harbor/ca.crt /etc/docker/certs.d/{harbor_ip}
# docker login https://{harbor_ip}

// upload docker images to harbor
# cd /root/software/pods
# ./upload_image.sh

Note - When Running, Stopping, or Reconfiguring Harbor

Harbor runs automatically at boot time. To manually start, stop, or reconfigure Harbor, execute the following command:

# cd ~/cocktail/harbor
# docker-compose start
# docker-compose stop

If settings have been changed, run the install.sh file again.

6.Master node Upload files to master server using sftp or utility such as FileZilla. Directory structure should be like as follow:

/root/
software/
├── README.md
├── bash_alias
├── docker-ce-17.03
│   ├── docker
│   ├── docker-compose
│   ├── docker-containerd
│   ├── docker-containerd-ctr
│   ├── docker-containerd-shim
│   ├── docker-init
│   ├── docker-proxy
│   ├── docker-runc
│   ├── dockerd
│   └── service
│       └── docker.service
├── docker-ce.repo
├── etcd-3.2.15-2.el7.x86_64.rpm
├── jq-1.5-1.el7.x86_64.rpm
├── kubeadm
├── kubeadm_v1.8.14
├── kubectl-1.8.14-0.x86_64.rpm
├── kubelet-1.8.14-0.x86_64.rpm
├── kubernetes-cni-0.5.1-1.x86_64.rpm
├── kubernetes.repo
├── oniguruma-5.9.5-3.el7.x86_64.rpm
└── socat-1.7.3.2-2.el7.x86_64.rpm

# vi /etc/exports        // configure CIDR
/nfs xxx.xxx.xxx.xxx/24(rw,sync,no_root_squash,no_all_squash,no_subtree_check)

// Register and enable rpcbind, nfs-server service.
# systemctl enable rpcbind
# systemctl enable nfs-server
# systemctl start rpcbind
# systemctl start nfs-server
  • Copy docker to /usr/bin
# cp /root/software/docker-ce-17.03/docker* /usr/bin
# cp /root/cube/cube_linux_x.x.x /usr/bin/cube

# docker&
# fg
# ctrl+C

# cp /root/software/docker-ce-17.03/service/docker.service /etc/systemd/system/

# systemctl enable docker; systemctl start docker

# logout and login
# docker info // verify docker configuration
# docker ps
  • Verifying whether Harbor login via Docker client is successfully.

Copy the ca.crt file created on the Harbor server to /etc/docker/certs.d/{harbor_ip}/ where the Docker client verifies the certificate.

Important! Folder creation must follow the format shown below.

# mkdir -p /etc/docker/certs.d/{harbor_ip}
# scp root@{harbor_ip}:/root/cocktail/cert/harbor/ca.crt /etc/docker/certs.d/{harbor_ip}
# docker login https://{harbor_ip}

7.Worker node Upload files to master server using sftp or utility such as FileZilla. Directory structure should be like as follow:

/root/
software/
├── docker-ce-17.03
│ ├── docker
│ ├── docker-compose
│ ├── docker-containerd
│ ├── docker-containerd-ctr
│ ├── docker-containerd-shim
│ ├── docker-init
│ ├── docker-proxy
│ ├── docker-runc
│ ├── dockerd
│ └── service
│ └── docker.service
├── kubeadm
├── kubeadm_v1.8.14
├── kubectl-1.8.14-0.x86_64.rpm
├── kubelet-1.8.14-0.x86_64.rpm
├── kubernetes-cni-0.5.1-1.x86_64.rpm
└── socat-1.7.3.2-2.el7.x86_64.rpm
  • Check connection to nfs-server is normal
# showmount -e {master_ip} // configure node ip address.
  • Copy docker to /usr/bin
# cp /root/software/docker-ce-17.03/docker* /usr/bin
# cp /root/cube/cube_linux_x.x.x /usr/bin/cube

# docker&
# fg
# ctrl+C

# cp /root/software/docker-ce-17.03/service/docker.service /etc/systemd/system/

# systemctl enable docker; systemctl start docker

# logout and login
# docker info // verify docker configuration
# docker ps
  • Verifying whether Harbor login via Docker client is successfully.

Copy the ca.crt file created on the Harbor server to /etc/docker/certs.d/{harbor_ip}/ where the Docker client verifies the certificate.

Important! Folder creation must follow the format shown below.

# mkdir -p /etc/docker/certs.d/{harbor_ip}
# scp root@{harbor_ip}:/root/cocktail/cert/harbor/ca.crt /etc/docker/certs.d/{harbor_ip}
# docker login https://{harbor_ip}

8.Install kubernetes and cocktail using cube

  • Prepare ssh certification file and copy it all node
# ssh-keygen -f /root/cube/cert/id_rsa -t rsa -N ''
# ssh-copy-id -i /root/cube/cert/id_rsa.pub root@{master_ip}
# ssh-copy-id -i /root/cube/cert/id_rsa.pub root@{node_ip}
  • Modify default values such as harbor ip, docker certification value
# vi /root/cube/cubescripts/provision/templates/all.yml.tpl
api_image: {harbor_ip}/google_containers/kube-apiserver-amd64:v${k8s_version}
controller_image: {harbor_ip}/google_containers/kube-controller-manager-amd64:v${k8s_version}
scheduler_image: {harbor_ip}/google_containers/kube-scheduler-amd64:v${k8s_version}
kube_proxy_image: {harbor_ip}/google_containers/kube-proxy-amd64:v${k8s_version}

# vi /root/cube/cubescripts/roles/addon/defaults/main.yaml
Change image's domain to harbor_ip

# vi /root/cube/cubescripts/roles/cocktail/defaults/main.yaml
...
cph_capem : Result of “cat /root/.docker/ca.crt | base64 -w0” on harbor server
cph_certpem: Result of “cat /root/.docker/cert.pem | base64 -w0” on harbor server
cph_keypem: Result of “cat /root/.docker/key.pem | base64 -w0” on harbor server
...
  • Execute cube to create cluster
# cd /root/cube

// modify cube.yaml (refer to baremetal on mac, linux section)

# cube create --cube-image={harbor_ip}/cocktail/cubetool:1.0.2.B000004 (cubetool image version may be different)

9.Verify installation

  • Verify all pods running normally
# cube status
  • Connect to cocktail dashboard Run bowser and connect to http://{master_ip}:30000. You should be login successfully default account.

results matching ""

    No results matching ""