Etsicatalog Project

Architecture

Introduction

The Etsicatalog project provides a runtime catalog service which can be consumed by other projects or components, such as UUI, VF-C, etc. The catalog can be used to store packages distributed by the SDC, and also includes a TOSCA parser service.

Etsicatalog is a web application based on python3 and Django framework. It is a standalone micro-service which provides:

• Package Management Service

• Parser Service

Developer Guide

Etsicatalog is a web application based on python3 and Django framework.

Pre-requisites

• Python3 & pip

• MariaDB

Etsicatalog can run standalone. However, if you want to try the whole functionality, you should have other components like SDC, DMaap(Non-mandatory), MSB(Non-mandatory) running.

You should set the component information in the environment variables as followed:

SDC_ADDR=https://{SDC_IP}:30204
MSB_ENABLED=true
MSB_ADDR=https://{MSB_IP}:30283
DMAAP_ENABLED=true
DMAAP_ADDR=https://{DMAAP_IP}:30226

Note:

• The default value of MSB_ENABLED is false. Since Guilin Release, MSB is a Non-mandatory component. If you have no MSB installed or intention to use it, you can just omit MSB_ADDR and MSB_ENABLED.

• The default value of DMAAP_ENABLED is false. If you want to use SDC subscription and notification function, you should set it true and set DMAAP_ADDR properly.

Build & Run

Clone repository:

$ git clone https://gerrit.onap.org/r/modeling/etsicatalog
$ cd etsicatalog

Create database:

$ cd /resources/dbscripts/mysql

Run modeling-etsicatalog-createdb.sql to create database.

Run commands followed to init database:

$ python manage.py makemigrations
$ python manage.py makemigrations database
$ python manage.py migrate
$ python manage.py migrate database

Review and edit catalogpubconfigconfig.py

MySQL default configuration is as follows:

DB_IP = "127.0.0.1"
DB_PORT = 3306
DB_NAME = "etsicatalog"
DB_USER = "etsicatalog"
DB_PASSWD = "etsicatalog"

Start server:

$ python manage.py runserver 8806

Test

Run Healthcheck:

GET /api/catalog/v1/health_check

You should get:

{
    "status": "active"
}

View API document:

http://127.0.0.1:8806/api/catalog/v1/swagger

Run from Docker image

You can run Modeling/etsicatalog directly from the docker image by following commands:

$ docker run -d -p 3306:3306 --name etsicatalog-db -v /var/lib/mysql -e MYSQL_USER="etsicatalog" -e MYSQL_PASSWORD="etsicatalog" -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE="etsicatalog" nexus3.onap.org:10001/library/mariadb

$ docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' etsicatalog-db
Get the IP of etsicatalog-db

$ docker run -d --name modeling-etsicatalog -v /var/lib/mysql -e DB_IP=<ip address> -e SDC_ADDR=<ip address> nexus3.onap.org:10001/onap/modeling/etsicatalog

Note:

You can also build the docker image instead of using the existed image from nexus3.onap.org:10001.

$ cd docker
$ docker build -t ${IMAGE_NAME} .

Offered APIs

Etsicatalog provides the followed APIs:

• NS package management interface

Provides runtime NS package management interface

URL	Method	Description
/api/nsd/v1/ns_descriptors	POST	Create a new NS descriptor resource.
/api/nsd/v1/ns_descriptors	GET	Query information about multiple NS descriptor resources.
/api/nsd/v1/ns_descriptors/{{nsdInfoId}}	GET	Read information about an individual NS descriptor resource.
/api/nsd/v1/ns_descriptors/{{nsdInfoId}}/nsd_content	PUT	Upload the content of a NSD.
/api/nsd/v1/ns_descriptors/{{nsdInfoId}}/nsd_content	GET	Fetch the content of a NSD.
/api/nsd/v1/ns_descriptors/{{nsdInfoId}}	DELETE	Delete an individual NS descriptor resource.
/api/nsd/v1/pnf_descriptors	POST	Create a new PNF descriptor resource.
/api/nsd/v1/pnf_descriptors	GET	Query information about multiple PNF descriptor resources.
/api/nsd/v1/pnf_descriptors/{{pnfdInfoId}}	GET	Read an individual PNFD resource.
/api/nsd/v1/pnf_descriptors/{{pnfdInfoId}}/pnfd_content	PUT	Upload the content of a PNFD.
/api/nsd/v1/pnf_descriptors/{{pnfdInfoId}}/pnfd_content	GET	Fetch the content of a PNFD.
/api/nsd/v1/pnf_descriptors/{{pnfdInfoId}}	DELETE	Delete an individual PNF descriptor resource.

• VNF package management interface

Provides runtime VNF package management interface

URL	Method	Description
/api/vnfpkgm/v1/vnf_packages	POST	Create a new individual VNF package resource
/api/vnfpkgm/v1/vnf_packages	GET	Query VNF packages information
/api/vnfpkgm/v1/vnf_packages/{{vnfPkgId}}	GET	Read information about an individual VNF package
/api/vnfpkgm/v1/vnf_packages/{{vnfPkgId}}/package_content	PUT	Upload a VNF package by providing the content of the VNF package
/api/vnfpkgm/v1/vnf_packages/{{vnfPkgId}}/package_content/upload_from_uri	PUT	Upload a VNF package by providing the address information of the VNF package
/api/vnfpkgm/v1/vnf_packages/{{vnfPkgId}}/package_content	GET	Fetch an on-boarded VNF package
/api/vnfpkgm/v1/vnf_packages/{{vnfPkgId}}/vnfd	GET	Read VNFD of an on-boarded VNF package
/api/vnfpkgm/v1/vnf_packages/{{vnfPkgId}}/artifacts/{{artifactPath}}	GET	Fetch individual VNF package artifact
/api/vnfpkgm/v1/vnf_packages/{{vnfPkgId}}	DELETE	Delete an individual VNF package
/api/vnfpkgm/v1/subscriptions	POST	Subscribe to notifications related to on-boarding and/or changes of VNF packages
/api/vnfpkgm/v1/subscriptions	GET	Query multiple subscriptions
/api/vnfpkgm/v1/subscriptions/{{subscriptionId}}	GET	Read an individual subscription resource
/api/vnfpkgm/v1/subscriptions/{{subscriptionId}}	DELETE	Terminate a subscription

• Catalog interface

Provides APIs to query/fetch package from SDC catalog

URL	Method	Description
/api/catalog/v1/nspackages	POST	Fetch NS package from SDC catalog
/api/catalog/v1/vnfpackages	POST	Fetch NVF package from SDC catalog
/api/catalog/v1/service_packages	POST	Fetch Service package from SDC catalog

• Parser interface

Provide APIs to parser VNF/PNF/NS/Service package

URL	Method	Description
/api/parser/v1/parserpnfd	POST	Parse PNF package
/api/parser/v1/parservnfd	POST	Parse VNF package
/api/parser/v1/parsernsd	POST	Parse NS package
/api/parser/v1/parser	POST	Parse package

You can download the following API yaml file and paste the content into the swagger tool: https://editor.swagger.io to view more detail of APIs.

etsicatalog_API_v1.yaml

Consumed APIs

SDC

Etsicatalog invokes SDC APIs to query/fetch package from SDC catalog.

URL	Method	Description
/api/sdc/v1/catalog/services/	GET	Get service list
/api/sdc/v1/catalog/services/{{csarId}}/metadata	GET	Get a service metadata
/api/sdc/v1/catalog/resources	GET	Get resource list
/api/sdc/v1/catalog/resources/{{csarId}}/metadata	GET	Get a resource metadata
/api/sdc/v1/catalog/services/{{csarId}}/toscaModel	GET	Download a service package

Micro Service Bus

Etsicatalog invokes Micro Service Bus APIs to register service to MSB.

URL	Method	Description
/api/microservices/v1/services	POST	Register service to the Microservice Bus

Installation

Contents

• Installation

  • OOM Charts

  • Installing or Upgrading

  • Etsicatalog Pods

  • Exposing ports

This document describes Modeling/etsicatalog installation by OOM.

OOM Charts

The Modeling/etsicatalog K8S charts are located in the OOM repository: https://gerrit.onap.org/r/admin/repos/oom

For OOM deployment you can refer to the below links:

• https://docs.onap.org/projects/onap-oom/en/latest/oom_user_guide.html#oom-user-guide

• https://docs.onap.org/projects/onap-oom/en/latest/oom_quickstart_guide.html#oom-quickstart-guide

Installing or Upgrading

The assumption is you have cloned the charts from the OOM repository into a local directory.

Step 1 Go into local copy of OOM charts

From your local copy, edit the values.yaml file to make desired changes.

Step 2 Build the chart

$ cd oom/kubernetes
$ make modeling
$ helm search local|grep modeling

Step 3 Un-install if installed before

$ helm delete  dev-modeling --purge
$ kubectl -n onap get pod |grep modeling-mariadb

Step 4 Delete persistent volume claim and NFS persisted data for etsicatalog

$ kubectl -n onap get pvc |grep dev-modeling|awk '{print $1}'|xargs kubectl -n onap delete pvc
$ rm -rf /dockerdata-nfs/dev-modeling/

Step 5 Reinstall

$ helm install local/modeling --namespace onap --name dev-modeling
$ kubectl -n onap get pod |grep modeling

Etsicatalog Pods

To get the etsicatalog Pod, run the following command:

$ kubectl -n onap get pods | grep modeling

dev-modeling-etsicatalog-754f4d6f94-lmjzz       2/2     Running                 2          92d

To access the etsicatalog docker container, run the command:

$ kubectl -n onap exec -it dev-modeling-etsicatalog-754f4d6f94-lmjzz -c modeling-etsicatalog -- /bin/bash

To restart the pod, run the command:

$ kubectl delete pod dev-modeling-etsicatalog-754f4d6f94-lmjzz -n onap

From Guilin Release, etsicatalog uses the public database:

$ kubectl -n onap get pods | grep mariadb-galera

dev-mariadb-galera-0                               2/2     Running            0          14d
dev-mariadb-galera-1                               2/2     Running            0          14d
dev-mariadb-galera-2                               2/2     Running            0          14d

Exposing ports

For security reasons, the port for the etsicatalog container is configured as ClusterIP and thus not exposed. If you need the port in a development environment, then the following command will expose it.

$ kubectl -n onap expose service modeling-etsicatalog --target-port=8806 --type=NodePort

Administration

Administration Links

Here are some links to our administration pages found in the ONAP Service websites

• Modeling Jira

• Modeling Jenkins

• Modeling Confluence Portal

Human Interfaces

There is No Human Interface (pure Rest API project)

Any “Rest Client” application may be used (Postman, …) to interact with etsicatalog application.

Release Notes

etsicatalog provides package management service and parser service as Micro Service.

Version: 1.0.14

Release Date

2022-04-12

Bug Fixes

• Fix tox test loop in mvn-phase-script.sh

Released components:

• etsicatalog 1.0.14

Version: 1.0.13

Release Date

2022-03-30

New Features

• update Django version to 3.1.4

• Update the vulnerable direct dependencies

• Update Docker image to fix CSIT Failure

Released components:

• etsicatalog 1.0.13

Version: 1.0.11

Release Date

2021-08-31

New Features

• update PyYAML & httplib2 version

Released components:

• etsicatalog 1.0.11

Version: 1.0.10

Release Date

2021-03-03

New Features

• Support Subscription and Notification for NSD packages

• Refactor logging to remove dependency on onaplogging component

• Upgrade to Python3.8

• Update document of installation and developer guide

Bug Fixes

• Fix bug: Get VNF Package Artifact endpoint doesn’t accept file extensions

Released components:

• etsicatalog 1.0.10

Version: 1.0.9

Release Date

2020-11-03

New Features

• Support SDC-ETSI Catalog Manager direct interface

• Query SDC for ETSI packages from the ETSI_PACKAGE directory and store the packages

• Optimize the docker image

• Remove the mandatory dependency on MSB

Released components:

• etsicatalog 1.0.9

Version: 1.0.6

Release Date

2020-05-11

New Features

• optimize the docker image

Released components:

• etsicatalog 1.0.6

Version: 1.0.5

Release Date

2020-03-03

New Features

• VNF subscription and notification support

• Support ONBOARDING_PACKAGE directory to get original vendor package

• Unify API endpoint

• Add API to read VNFD

Released components:

• etsicatalog 1.0.5

Bug Fixes

None

Known Issues

By now etsicatalog has not supported HTTPS directly. But all of APIs have registered to MSB and client can call etsicatalog APIs through MSB HTTPS request, such as: curl -X GET ‘https://msb_ip:msb_port/api/vnfpkgm/v1/subscriptions’.

Security Issues

None

Upgrade Notes

Update API endpoint:

• Chang “api/parser/v1/service_packages” -> “api/catalog/v1/service_packages”

• Unify parser API as “api/parser/V1/…”

Deprecation Notes

None

Other

Version: 1.0.4

Release Date

2019-09-17

New Features

• Update to python3

• Optimize the process of service package distribution

• Merge with vfc/catalog

Released components:

• etsicatalog 1.0.4

Bug Fixes

None

Known Issues

None

Security Issues

• El Alto Vulnerability Report

Upgrade Notes

None

Deprecation Notes

None

Other

---

Version: 1.0.2

Release Date

2019-06-06

New Features

• Package management service.

• Parser service.

Released components:

• etsicatalog 1.0.2

Bug Fixes

This is the initial release

Known Issues

None

Security Issues

None

Upgrade Notes

This is the initial release

Deprecation Notes

This is the initial release

Other

---

End of Release Notes