Cocktail Private Hub Server Installation

1.サーバーにパッケージをアップロードする sftpやFileZillaなどのユーティリティを使用して 'harbor'サーバーにファイルをアップロードします。 ディレクトリ構造は次のようにする必要があります:

/root/
├── cocktail
│   ├── cert
│   │   ├── docker                  // Shell script file for generating docker certification files.
│   │   ├── harbor                  // Shell script file for generating harbor certification files.
│   │   └── openssl.cnf             // OpenSSL configuration file.
│   └── harbor                      // harbor root directory.
│       ├── LICENSE
│       ├── NOTICE
│       ├── common
│       ├── docker-compose.clair.yml
│       ├── docker-compose.notary.yml
│       ├── docker-compose.yml
│       ├── harbor.cfg
│       ├── harbor.v1.3.0.tar.gz
│       ├── harbor_1_1_0_template
│       ├── install.sh
│       ├── prepare
│       └── upgrade
├── cube
│   ├── cert
│   │   ├── id_rsa                     // SSH private key. Just for convenience.
│   │   └── id_rsa.pub                 // SSH public key. Just for convenience.
│   ├── cube.yaml
│   ├── cube                          // cube executable file.
│   └── cubescripts                    // cube scripts root directory
│       ├── README-Bearmetal.md
│       ├── README.md
│       ├── addon.yml
│       ├── ansible.cfg
│       ├── cert.yml
│       ├── cluster.retry
│       ├── cluster.yml
│       ├── cocktail.yml
│       ├── cube.log
│       ├── group_vars
│       ├── inventories
│       ├── node.yml
│       ├── pre-destroy.yml
│       ├── provision
│       ├── reset.retry
│       ├── reset.yml
│       ├── roles
│       └── scripts
└── software
    ├── docker-ce-17.03              // docker ce rpm file
    │   ├── docker
    │   ├── docker-compose
    │   ├── docker-containerd
    │   ├── docker-containerd-ctr
    │   ├── docker-containerd-shim
    │   ├── docker-init
    │   ├── docker-proxy
    │   ├── docker-runc
    │   ├── dockerd
    │   └── service
    └── pods                        // docker images root directory.
        ├── cocktail_2.1.1           // cocktail images
        ├── kube                   // kubernetes images
        ├── monitoring              // Prometheus, AlertManager, nodeExport, Grafana
        ├── prepare_image.sh      // Shell script for saving images to local
        ├── test                     // Test images
        └── upload_image.sh       // upload script to harbor

2.ファイアウォールを停止し、selinuxオプションを変更してリブートする

// Changes as root user
# sudo su - root

// Stop firewall
# systemctl stop firewalld
# systemctl disable firewalld

// Changes selinux to disabled
# vi /etc/sysconfig/selinux
SELINUX=disabled

# reboot

3.harborとdocker個人認証ファイル生成

  • Add harbor ip address to /root/cocktail/cert/openssl.conf
# vi /root/cocktail/cert/openssl.cnf
[ req ]
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_ca ]
basicConstraints = critical, CA:TRUE
keyUsage = critical, digitalSignature, keyEncipherment, keyCertSign

[ v3_req_server ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names_harbor

[ v3_builder_server ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names_builder

[ v3_builder_client ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth

[ alt_names_harbor ]
DNS.1 = localhost
**IP.1 = 172.30.1.56 // registry server IP**
IP.2 = 127.0.0.1

[ alt_names_builder ]
DNS.1 = localhost
**IP.1 = 172.30.1.56 // registry server IP**
IP.2 = 127.0.0.1
  • Create harbor and docker certificate files
// Creates harbor certificate
# chmod 755 /root/cocktail/cert/harbor/make_harbor_cert.sh
# /root/cocktail/cert/harbor/make_harbor_cert.sh

// Creates docker certificate
# chmod 755 /root/cocktail/cert/docker/make_docker_cert.sh
# /root/cert/cocktail/docker/make_docker_cert.sh {registry_ip}

4.dockerとcube実行ファイルを/ usr / binにコピーし、検証

# cp /root/software/docker-ce-17.03/docker* /usr/bin
# cp /root/cube/cube_linux_x.x.x /usr/bin/cube

# docker&
# fg
# ctrl+C

# docker-compose version
# cube version

# cp /root/software/docker-ce-17.03/service/docker.service /etc/systemd/system/

# systemctl enable docker; systemctl start docker

# logout and login
# docker info
# docker ps

5.Harbor設定および起動

  • Harbor.cfg修正とインストール ```

    cd /root/cocktail/harbor

    vi harbor.cfg

    ... hostname = {harbor_ip} ui_url_protocol = https ssl_cert = /root/cocktail/cert/harbor/harbor.crt // harbor crt file path ssl_cert_key = /root/cocktail/cert/harbor/harbor.key // harbor certificate key file path harbor_admin_password = C0ckt@1lAdmin ...

./install.sh

... output harbor installing ...

// Verifies whether Harbor started up normally. (Normal if 7 containers in total are running)

docker ps


* Visit harbor site and create project

Run bowser and connect to https://{harbor_ip}.
You should be login successfully as id/password in harbor.cfg and make projects (google_containers, monitoring, cocktail, test) which will be used to upload docker images later

* Verifying whether Harbor login via Docker client is successfully.

Copy the ca.crt file created on the Harbor server to /etc/docker/certs.d/{harbor_ip}/ where the Docker client verifies the certificate.

`Important!` Folder creation must follow the format shown below.

mkdir -p /etc/docker/certs.d/{harbor_ip}

cp /root/cocktail/cert/harbor/ca.crt /etc/docker/certs.d/{harbor_ip}

docker login https://{harbor_ip}

// upload docker images to harbor

cd /root/software/pods

./upload_image.sh


`Note` - When Running, Stopping, or Reconfiguring Harbor

Harbor runs automatically at boot time. To manually start, stop, or reconfigure Harbor, execute the following command:

cd ~/cocktail/harbor

docker-compose start

docker-compose stop

If settings have been changed, run the install.sh file again.


**6.Master node**
sftpやFileZillaなどのユーティリティを使用して 'harbor'サーバーにファイルをアップロードします。
ディレクトリ構造は次のようにする必要があります:

/root/ software/ ├── README.md ├── bash_alias ├── docker-ce-17.03 │ ├── docker │ ├── docker-compose │ ├── docker-containerd │ ├── docker-containerd-ctr │ ├── docker-containerd-shim │ ├── docker-init │ ├── docker-proxy │ ├── docker-runc │ ├── dockerd │ └── service │ └── docker.service ├── docker-ce.repo ├── etcd-3.2.15-2.el7.x86_64.rpm ├── jq-1.5-1.el7.x86_64.rpm ├── kubeadm ├── kubeadm_v1.8.14 ├── kubectl-1.8.14-0.x86_64.rpm ├── kubelet-1.8.14-0.x86_64.rpm ├── kubernetes-cni-0.5.1-1.x86_64.rpm ├── kubernetes.repo ├── oniguruma-5.9.5-3.el7.x86_64.rpm └── socat-1.7.3.2-2.el7.x86_64.rpm

vi /etc/exports // configure CIDR

/nfs xxx.xxx.xxx.xxx/24(rw,sync,no_root_squash,no_all_squash,no_subtree_check)

// Register and enable rpcbind, nfs-server service.

systemctl enable rpcbind

systemctl enable nfs-server

systemctl start rpcbind

systemctl start nfs-server


* Copy docker to /usr/bin

cp /root/software/docker-ce-17.03/docker* /usr/bin

cp /root/cube/cube_linux_x.x.x /usr/bin/cube

docker&

fg

ctrl+C

cp /root/software/docker-ce-17.03/service/docker.service /etc/systemd/system/

systemctl enable docker; systemctl start docker

logout and login

docker info // verify docker configuration

docker ps


* Verifying whether Harbor login via Docker client is successfully.

Copy the ca.crt file created on the Harbor server to /etc/docker/certs.d/{harbor_ip}/ where the Docker client verifies the certificate.

`Important!` Folder creation must follow the format shown below.

mkdir -p /etc/docker/certs.d/{harbor_ip}

scp root@{harbor_ip}:/root/cocktail/cert/harbor/ca.crt /etc/docker/certs.d/{harbor_ip}

docker login https://{harbor_ip}


**7.Worker node**
sftpやFileZillaなどのユーティリティを使用して 'harbor'サーバーにファイルをアップロードします。
ディレクトリ構造は次のようにする必要があります:

/root/ software/ ├── docker-ce-17.03 │ ├── docker │ ├── docker-compose │ ├── docker-containerd │ ├── docker-containerd-ctr │ ├── docker-containerd-shim │ ├── docker-init │ ├── docker-proxy │ ├── docker-runc │ ├── dockerd │ └── service │ └── docker.service ├── kubeadm ├── kubeadm_v1.8.14 ├── kubectl-1.8.14-0.x86_64.rpm ├── kubelet-1.8.14-0.x86_64.rpm ├── kubernetes-cni-0.5.1-1.x86_64.rpm └── socat-1.7.3.2-2.el7.x86_64.rpm


* Check connection to nfs-server is normal

showmount -e {master_ip} // configure node ip address.


* Copy docker to /usr/bin

cp /root/software/docker-ce-17.03/docker* /usr/bin

cp /root/cube/cube_linux_x.x.x /usr/bin/cube

docker&

fg

ctrl+C

cp /root/software/docker-ce-17.03/service/docker.service /etc/systemd/system/

systemctl enable docker; systemctl start docker

logout and login

docker info // verify docker configuration

docker ps


* Verifying whether Harbor login via Docker client is successfully.

Copy the ca.crt file created on the Harbor server to /etc/docker/certs.d/{harbor_ip}/ where the Docker client verifies the certificate.

`Important!` Folder creation must follow the format shown below.

mkdir -p /etc/docker/certs.d/{harbor_ip}

scp root@{harbor_ip}:/root/cocktail/cert/harbor/ca.crt /etc/docker/certs.d/{harbor_ip}

docker login https://{harbor_ip}


**8.Install kubernetes and cocktail using cube**
* Prepare ssh certification file and copy it all node

ssh-keygen -f /root/cube/cert/id_rsa -t rsa -N ''

ssh-copy-id -i /root/cube/cert/id_rsa.pub root@{master_ip}

ssh-copy-id -i /root/cube/cert/id_rsa.pub root@{node_ip}


* Modify default values such as harbor ip, docker certification value

vi /root/cube/cubescripts/provision/templates/all.yml.tpl

api_image: {harbor_ip}/google_containers/kube-apiserver-amd64:v${k8s_version} controller_image: {harbor_ip}/google_containers/kube-controller-manager-amd64:v${k8s_version} scheduler_image: {harbor_ip}/google_containers/kube-scheduler-amd64:v${k8s_version} kube_proxy_image: {harbor_ip}/google_containers/kube-proxy-amd64:v${k8s_version}

vi /root/cube/cubescripts/roles/addon/defaults/main.yaml

Change image's domain to harbor_ip

vi /root/cube/cubescripts/roles/cocktail/defaults/main.yaml

... cph_capem : Result of “cat /root/.docker/ca.crt | base64 -w0” on harbor server cph_certpem: Result of “cat /root/.docker/cert.pem | base64 -w0” on harbor server cph_keypem: Result of “cat /root/.docker/key.pem | base64 -w0” on harbor server ...


* Execute cube to create cluster

cd /root/cube

// modify cube.yaml (refer to baremetal on mac, linux section)

cube create --cube-image={harbor_ip}/cocktail/cubetool:1.0.2.B000004 (cubetool image version may be different)


**9.インストールの検証**
* Verify all pods running normally

cube status

```

  • Connect to cocktail dashboard Run bowser and connect to http://{master_ip}:30000. You should be login successfully default account.

results matching ""

    No results matching ""