MultiCloud Kubernetes Plugin

This plugin is a part of the MultiCloud component in ONAP. It will allow ONAP to deploy/manage VM and container based VNFs over cloud regions that support Kubernetes as the orchestrator.

Kubernetes Reference Deployment

The Kubernetes Reference Deployment(KUD) provides an automated mechanism to install and configure the Kubernetes services required for the MultiCloud Kubernetes plugin.

Table of contents

KUD Architecture

This document explains the different components of the Kubernetes Reference Deployment project and how they can be configured to modify its default behaviour.

Vagrantfile

This file describes how the Virtual Machines are going to be configured and the scripts and arguments used during their provisioning process. This file uses elastic/ubuntu-16.04-x86_64 vagrant box for VirtualBox and Libvirt providers.

config/

This folder contains the POD Descriptor File (PDF) which is used by Vagrant during the provisioning process. The samples folder contains examples for some setups (All-in-One, Mini, NoHA, HA, etc.) that can be used.

This list contains the valid entries used by Vagrant to define the virtual resources used by Vagrant during the creation of the Virtual Machines:

  • ip - The static IP address assigned to the VM. (String value)

  • memory - The amount of memory RAM. (KB - Integer value)

  • cpus - Number of CPUs. (Integer value)

  • volumes - List of volumes to be formatted and mounted to the VM.

config/default.yml

If there is no pdf.yml file present in config folder, Vagrant will use the information specified in the config/default.yml. The following diagram displays how the services are installed in the nodes using the default configuration.

_images/default_pdf.png

docs/

This folder contains documentation files using reStructuredText (RST) syntax. It’s possible to generate documentation in html format using python tox module . Once this is installed, it’s possible to build html files using this following command:

tox -e docs

After its execution, the docs/build subfolder will contain subfolders and html files that can be opened from any web browser.

galaxy-requirements.yml

This file contains third party Ansible roles. Only those tasks which are not related with the main installation process have been placed in this file.

installer.sh

Main bash script that installs dependencies and executes ansible playbooks for provisioning KUD components on external nodes.

inventory/

This folder contains the Ansible host inventory file. The inventory/host.ini file, which is used during the execution of Ansible playbooks, is created by Vagrant using the values specified in the config/pdf.yml file (or config/default.yml).

inventory/group_vars/k8s-cluster.yml

A preferred practice in Ansible is to not store variables in the main inventory file. The configuration variables required for Kubespray are stored in this file.

node.sh

This bash script is executed in every node after this has been provisioned. The script provides the possibility to partition and mount external volumes.

playbooks/

This folder contains a set of Ansible playbooks which perform the tasks required for configuring services like Multus, Virtlet and/or OVN.

playbooks/configure-kud.yml

This ansible playbook collects the common actions among all the Kubernetes AddOns offered by the KUD.

playbooks/kud-vars.yml

This file centralizes the version numbers and source URLs used for different components offered by the KUD. Bumping a version requires extensive testing to ensure compatibility.

setup.sh

This bash script is used for the installation and configuration of dependencies required for the usage of the KUD via Virtual Machines. Some of this dependencies are:

The -p argument determines the Virtualization provider to be used and installed in the host machine.

sudo ./setup.sh -p libvirt

Vagrant uses VirtualBox as default Virtualization provider. It’s possible to modify this behavior using the global enviroment variable named VAGRANT_DEFAULT_PROVIDER.

Note

The execution of this script is recommended only during the initial setup.

tests/

This folder contains the health check scripts that guarantee the proper installation/configuration of Kubernetes AddOns. Its execution is disabled by default. In order to enable it, it’s necessary to export KUD_ENABLE_TESTS=true environment variable before calling the installer.sh bash script, usually through changing the arguments in the Vagrantfile.

Bare-Metal Provisioning

The Kubernetes Deployment, aka KUD, has been designed to be consumed by Virtual Machines as well as Bare-Metal servers. The baremetal/aio.sh script contains the bash instructions for provisioning an All-in-One Kubernetes deployment in a Bare-Metal server. This document lists the Hardware & Software requirements and walkthrough the instructions that baremetal/aio.sh contains.

Hardware Requirements

Concept

Amount

CPUs

8

Memory

32GB

Hard Disk

150GB

Software Requirements

  • Ubuntu Server 16.04 LTS

baremetal/aio.sh

This bash script provides an automated process for deploying an All-in-One Kubernetes cluster.

The following two instructions start the provisioning process.

$ sudo su
# git clone https://git.onap.org/multicloud/k8s/
# cd k8s/kud/hosting_providers/baremetal/
# ./aio.sh

In overall, this script can be summarized in three general phases:

  1. Generating Inventory.

  2. Enabiling Nested-Virtualization.

  3. Deploying KUD services.

Inventory

Ansible works agains multiple systems, the way for selecting them is through the usage of the inventory. The inventory file is a static source for determining the target servers used for the execution of ansible tasks. The aio.sh script creates an inventory file for addressing those tasks to localhost.

cat <<EOL > inventory/hosts.ini
[all]
localhost

[kube-master]
localhost

[kube-node]
localhost

[etcd]
localhost

[ovn-central]
localhost

[ovn-controller]
localhost

[virtlet]
localhost

[k8s-cluster:children]
kube-node
kube-master
EOL

KUD consumes kubespray for provisioning a Kubernetes base deployment.

Ansible uses SSH protocol for executing remote instructions. The following instructions create and register ssh keys which avoid the usage of passwords.

# echo -e "\n\n\n" | ssh-keygen -t rsa -N ""
# cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# chmod og-wx ~/.ssh/authorized_keys

Enabling Nested-Virtualization

KUD installs Virtlet Kubernetes CRI for running Virtual Machine workloads. Nested-virtualization gives the ability of running a Virtual Machine within another. The node.sh bash script contains the instructions for enabling Nested-Virtualization.

# ./node.sh

Deploying KUD services

Finally, the KUD provisioning process can be started through the use of installer.sh bash script. The output of this script is collected in the kud_installer.log file for future reference.

# ./installer.sh | tee kud_installer.log
_images/installer_workflow.png

Sample cURL commands

POST

URL: localhost:8081/v1/vnf_instances/

Request Body
{
    "cloud_region_id": "region1",
    "namespace": "test-namespace",
    "csar_id": "uuid",
    "namespace": "test",
    "oof_parameters": [{
        "key1": "value1",
        "key2": "value2",
        "key3": {}
    }],
    "network_parameters": {
        "oam_ip_address": {
            "connection_point": "string",
            "ip_address": "string",
            "workload_name": "string"
        }
    }
}
Expected Response
{
    "vnf_id": "52fdfc07",
    "cloud_region_id": "cloudregion1",
    "namespace": "test-namespace",
    "vnf_components": {
        "deployment": [
            "cloudregion1-test-namespace-52fdfc07-kubedeployment"
        ],
        "service": [
            "cloudregion1-test-namespace-52fdfc07-kubeservice"
        ]
    }
}

GET

URL: localhost:8081/v1/vnf_instances

Expected Response
{
    "vnf_id_list": [
        "52fdfc07"
    ]
}

GET

URL: localhost:8081/v1/vnf_instances/cloudregion1/test-namespace/52fdfc07

Expected Response
{
    "vnf_id": "52fdfc07",
    "cloud_region_id": "cloudregion1",
    "namespace": "test-namespace",
    "vnf_components": {
        "deployment": [
            "cloudregion1-test-namespace-52fdfc07-kubedeployment"
        ],
        "service": [
            "cloudregion1-test-namespace-52fdfc07-kubeservice"
        ]
    }
}

* DELETE *

URL: localhost:8081/v1/vnf_instances/cloudregion1/test-namespace/52fdfc07

Expected Response

202 Accepted