7.4. Monitoring & Management

In ONAP, DCAE is responsible of collecting, receiving, and analyzing NF monitoring data. This data serves the basis for tracking the health, performance, and operational status of the NF. DCAE provides a number of predefined interfaces based upon accepted, open standards to support monitoring data ingestion. Some of these interfaces collect data by polling or pulling data from the NF using standard protocols. Other DCAE interfaces receive monitoring data (such as VES events) that are pushed from the NFs.

A NF that produces monitoring data and uses protocols that are compatible with ONAP’s predefined monitoring ingestion capabilities can easily be integrated with ONAP through simple configuration rather than custom development.

This chapter will define the expected requirements for a NF to easily integrate with an instance of ONAP.

7.4.1. Monitoring and Fault Protocol Selection

This section provides the proper guidance on how a NF should determine the protocol and data format for providing a specific types of telemetry data to ONAP.

Requirement: R-82909 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF MUST report faults and alarms using either Virtual Function Event Streaming (VES) or SNMP. (NOTE: See relevant sections for more detailed requirements)
Requirement: R-554966 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF MUST report performance metrics using Virtual Function Event Streaming (VES) or Bulk Performance Measurement.
Requirement: R-69111 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF MUST report application logs using either Virtual Function Event Streaming (VES) or Syslog in compliance with RFC 5424 .
Requirement: R-209104 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: guilin
The VNF or PNF producing VES syslog events SHOULD restrict these events to those that convey significant errors or warnings needed to support the operation or troubleshooting of the VNF or PNF. It is expected the volume of such events would be lower (e.g. less than 2000 per day) than more detailed events produced in the course of normal operations.
Requirement: R-332680 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: casablanca
updated: guilin
impacts: dcae
validation_mode: in_service
The VNF or PNF producing VES events SHOULD deliver syslog messages that meet the criteria in R-209104 to the VES Event Listener using the syslog VES domain.
Requirement: R-935717 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF MUST report heartbeats using Virtual Function Event Streaming (VES).
Requirement: R-697654 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MAY
introduced: casablanca
updated: guilin
impacts: DCAE
validation_mode: in_service
The VNF or PNF MAY leverage ONAP’s High Volume VNF Event Streaming (HV-VES) when there is a need to deliver large volumes of real-time performance management metrics. See HV-VES collector service details for more information.
Requirement: R-857511 ../_images/arrow-right-circle.svg
target: VNF or PNF PROVIDER
keyword: MUST
introduced: guilin
impacts: DCAE
validation_mode: none
VNF or PNF Provider MUST have agreement with the Service Provider before utilizing the HV-VES option for monitoring as this option does not fully integrate with the ONAP’s DCAE event processing capabilities.
Requirement: R-908291 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MAY
introduced: casablanca
updated: guilin
impacts: dcae, dmaap
validation_mode: in_service

The VNF or PNF MAY leverage a bulk VNF or PNF telemetry transmission mechanism in instances where other transmission methods are not practical or advisable.

NOTE: For additional information and use cases for the Bulk Telemetry Transmission Mechanism, please refer to the Bulk Performance Measurement requirements and the 5G - Bulk PM ONAP Development Wiki page.

Requirement: R-63105 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MAY
introduced: guilin
impacts: dcae
The VNF or PNF MAY produce telemetry data using the RESTConf Collector, but this requires additional coordination with the operator to appropriately map the data internally to a VES-like structure used within ONAP. If this option is needed, then the VNF or PNF Provider must coordinate with with the Operator for the data to be successfully collected and processed by DCAE.

7.4.2. SNMP Monitoring Requirements

Requirement: R-261501 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
If the VNF or PNF is using SNMP, then the VNF or PNF Provider MUST provide a Management Information Base (MIB) file that uniquely identifies and describes all SNMP events exposed by the network function.
Requirement: R-233922 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: guilin
If the VNF or PNF is using SNMP, then the VNF or PNF Provider SHOULD provide examples of all SNMP alarms.

7.4.3. Virtual Function Event Streaming (VES) Client Requirements

The VES protocol enables NFs to transmit telemetry data in a non-proprietary, extensible format to ONAP using the HTTP protocol. This chapter will define the requirements for a NF to deliver events to ONAP’s VES event listeners in a manner that conforms with the appropriate VES Event Listener specifications, and ensures the NF can be configured to maximize the reliability of telemetry data delivery.

7.4.3.1. Event Definition and Registration

Requirement: R-520802 ../_images/arrow-right-circle.svg
target: VNF or PNF PROVIDER
keyword: MUST
introduced: casablanca
updated: guilin
impacts: dcae
validation_mode: static

If the VNF or PNF is using VES, then the VNF or PNF Provider MUST provide a YAML file formatted in adherence with the VES Event Registration specification that defines the following information for each event produced by the VNF:

  • eventName
  • Required fields
  • Optional fields
  • Any special handling to be performed for that event
Requirement: R-120182 ../_images/arrow-right-circle.svg
target: VNF or PNF PROVIDER
keyword: MUST
introduced: casablanca
updated: guilin
impacts: dcae
validation_mode: static

A VNF or PNF Provider utilizing VES MUST indicate specific conditions that may arise, and recommend actions that may be taken at specific thresholds, or if specific conditions repeat within a specified time interval, using the semantics and syntax described by the VES Event Registration specification.

NOTE: The Service Provider may override VNF or PNF provider Event Registrations using the ONAP SDC Design Studio to finalizes Service Provider engineering rules for the processing of the VNF or PNF events. These changes may modify any of the following:

  • Threshold levels
  • Specified actions related to conditions
Requirement: R-123044 ../_images/arrow-right-circle.svg
target: VNF or PNF PROVIDER
keyword: MAY
introduced: casablanca
updated: dublin
impacts: dcae
validation_mode: in_service
The VNF or PNF Provider MAY require that specific events, identified by their eventName, require that certain fields, which are optional in the common event format, must be present when they are published.

7.4.3.2. Event Formatting and Usage

Requirement: R-570134 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: casablanca
updated: guilin
impacts: dcae
validation_mode: in_service

The VES events produced by the VNF or PNF MUST be compliant with the common event formats defined in one of the following specifications:

The latest version (7.2) should be preferred. Earlier versions are provided for backwards compatibility.

Requirement: R-528866 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: casablanca
updated: guilin
impacts: dcae
validation_mode: in_service
The VES events produced by the VNF or PNF MUST conform to the schema and other formatting requirements specified in the relevant VES Event Listener specification.
Requirement: R-283988 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST NOT
introduced: casablanca
updated: guilin
impacts: dcae
validation_mode: in_service
A VNF or PNF producing VES events MUST NOT send information through extensible structures if the event specification has explicitly defined fields for that information.
Requirement: R-470963 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: casablanca
updated: guilin
impacts: dcae
validation_mode: in_service
A VNF or PNF producing VES events SHOULD leverage camel case to separate words and acronyms used as keys that will be sent through extensible fields. When an acronym is used as the key, then only the first letter shall be capitalized.
Requirement: R-408813 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: casablanca
updated: guilin
impacts: dcae
validation_mode: none
A VNF or PNF producing VES events MUST pass all information it is able to collect even if the information field is identified as optional. However, if the data cannot be collected, then optional fields can be omitted.
Requirement: R-408814 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
impacts: dcae
validation_mode: none

The VNF or a PNF producing VES stndDefined domain events to report standards-organization defined events to ONAP, MUST set the event.stndDefinedNamespace property. By default, ONAP ships with support for the following:

  • 3GPP-Provisioning
  • 3GPP-Heartbeat
  • 3GPP-FaultSupervision
  • 3GPP-PerformanceAssurance

Another namespace, outside of the list provided, needs to registered in ONAP in coordination with the operator before it can be used.

Requirement: R-408815 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
impacts: dcae
validation_mode: none
If the VNF or PNF producing VES stndDefined domain events provides the event.stndDefinedFields.schemaReference then it MUST set its value to the publicUrl value in DCAE’s VES Collector etc/externalRepo/schema-map.json that describes the data being sent in event.stndDefinedFields.data.
Requirement: R-408816 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
impacts: dcae
validation_mode: none
If the VNF or PNF producing VES stndDefined domain events provides the event.stndDefinedFields.schemaReference then it MUST only pass events that conform to schema references previously registered with DCAE otherwise the event will be rejected. By default, ONAP ships with support for schemas found in DCAE’s VES Collector etc/externalRepo/schema-map.json.
Requirement: R-408817 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
impacts: dcae
validation_mode: none
The VNF or PNF Provider producing stndDefined events MUST coordinate with the operator, willing to validate stndDefined events, to configure DCAE to accept any new event schema prior to sending those events or the events will be rejected.
Requirement: R-408818 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
impacts: dcae
validation_mode: none
If the VNF or PNF producing VES stndDefined domain events provides the event.stndDefinedFields.schemaReference then it MUST set the event.stndDefined.schemaReference property to an exact structure, from supported schemaReference, describing the notification within an openAPI specification, using JSON Pointer as URI fragment e.g. “https://forge.3gpp.org/…/faultMnS.yaml#/components/schemas/notifyNewAlarm

7.4.3.3. Configuration Requirements

This section defines the types the configuration options and defaults a NF producing VES events should provide to ensure the NF can be configured properly for the Service Provider’s ONAP environment and ensure reliable delivery of VES events.

There are several methods available to provide configuration settings to a network function. This document does not specify the exact manner in which the configuration elements described below must be required. The configuration can be provided during instantiation (e.g. preload), provided by an ONAP controller action, or provided manually.

Requirement: R-460012 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF producing VES events MUST allow the configuration of the attributes defined in Table 1 and utilize the provided default value (where applicable) when the configuration value is not provided by the Service Provider.
Requirement: R-940591 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: guilin
A VNF or PNF producing VES events SHOULD use the recommended parameter name for the configurable value from Table 1.
Table 1: VES Configurable Values
Parameter Description Default Parameter Name (VES 7.2+)
VES Listener Endpoint FQDN or IP of the Event Listener n/a ves_listener_endpoint
Heartbeat Interval Frequency in seconds the NF must send a heartbeat to the event listener 60 ves_heartbeat_interval_seconds
Timeout Value Duration in seconds the NF should wait for ACK from the event listener before timeout 5 ves_timeout_seconds
Measurement Interval Window size in seconds to use for aggregated measurements 300 ves_measurement_interval_seconds
HTTP Username Required if NF supports HTTP Basic Authentication with the VES Event Listener n/a ves_http_username
HTTP Password Required if NF supports HTTP Basic Authentication with the VES Event Listener n/a ves_http_password

7.4.3.3.1. VES Listener Endpoint and DNS Resolution

In a high availability deployment of a VES Event Listener, a round-robin DNS or dynamic DNS may be used to either load balance or provide fault tolerance of the Event Listener. Adherence to the following requirements ensure the VNF or PNF interacts properly with this deployment configuration.

Requirement: R-70492 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF MUST support DNS resolution of the VES Listener Endpoint if a Fully Qualified Domain Name (FQDN) is provided.
Requirement: R-130645 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF MUST respect the Time To Live provided by the DNS for the VES Event Listener FQDN.

7.4.3.4. Event Delivery Requirements

Requirement: R-06924 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
updated: guilin
The VNF or PNF producing VES events MUST deliver VES events as it becomes available or according to the configured measurement interval.
Requirement: R-655209 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF producing VES events MUST respect the configured VES Timeout Value when delivering VES events, and abort any call where the VES Event Listener does not successfully acknowledge the delivery of event(s) within the Timeout Value. These failed transactions should be buffered and retried in accordance with the Buffering and Redelivery Requirements.
Requirement: R-176945 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD NOT
introduced: guilin
The VNF or PNF producing VES events SHOULD NOT send syslog events to the VES Event Listener during debug mode, but rather store syslog events locally for access or possible file transfer.

7.4.3.5. Buffering and Redelivery

To maximize the reliable delivery of VES events when the VES Listener becomes unavailable or unhealthy, the NF must adhere to these requirements.

Requirement: R-658596 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin

A VNF or PNF producing VES events MUST buffer events that meet the following criteria if the VES Event Listener is unreachable or the request encounters a timeout.

  • Faults with eventSeverity of MINOR, MAJOR, NORMAL, or CRITICAL
  • Syslog with syslogSev of Emergency, Alert, Critical, Error, or Warning
  • All measurement events
Requirement: R-636251 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
A VNF or PNF producing VES events MUST size the event buffer referenced in R-658596 such that it can buffer a minimum of 1 hours of events under nominal load.
Requirement: R-498679 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MAY
introduced: guilin
A VNF or PNF producing VES events MAY discard buffered events older than a maximum retention period, not less than 1 hour, even if the event was never successfully delivered to the event listener. While discarding based on this retention period is supported for backwards compatibility, it is recommended to retain events until the maximum buffer size is reached per R-346137 as that will maximize the number of events delivered.
Requirement: R-346137 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
A VNF or PNF producing VES events that is buffering events per R-658596 MUST store in-scope events even when the maximum capacity of the buffer (defined in R-636251) has been reached. To make room for new events in this situation, hte oldest event in the buffer shall be removed as necessary. (i.e. First In First Out)
Requirement: R-379523 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin

A VNF or PNF producing VES events that is buffering events due to an unavailable VES Event Listener MUST redeliver all buffered events according to the following rules when the VNF or PNF detects the VES Event Listener has become available:

  • Deliver all previously buffered events before sending new events
  • Deliver buffered events in the order they were received
Requirement: R-818859 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: guilin
The VNF or PNF producing VES events MUST not allow an unavailable or timing out VES Event Listener to impact the performance, stability, or correct execution of network function.
Requirement: R-103464 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MAY
introduced: guilin
A VNF or PNF producing VES events that is buffering events due to an unavailable VES Event Listener MAY leverage to publishEventBatch operation to redeliver buffered events. Please note this can only be used when all buffered events belong to the same domain due to the restrictions in place for the operation.

7.4.3.6. Security

Requirement: R-68165 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
updated: dublin
The VNF or PNF MUST encrypt any content containing Sensitive Personal Information (SPI) or certain proprietary data, in addition to applying the regular procedures for securing access and delivery.
Requirement: R-33878 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: MUST
introduced: el alto
updated: guilin
The VNF or PNF MUST utilize one of the authentication methods prescribed by the relevant VES Event Listener specification.

7.4.4. Bulk Performance Measurement

Requirement: R-841740 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: casablanca
updated: dublin
impacts: dcae, dmaap
The VNF or PNF SHOULD support FileReady VES event for event-driven bulk transfer of monitoring data.
Requirement: R-440220 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: casablanca
updated: dublin
impacts: dcae, dmaap
The VNF or PNF SHOULD support File transferring protocol, such as FTPES or SFTP, when supporting the event-driven bulk transfer of monitoring data.
Requirement: R-75943 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: casablanca
updated: guilin
impacts: dcae, dmaap
The VNF or PNF SHOULD support the data schema defined in 3GPP TS 32.435 or 3GPP TS 28.532, when supporting the event-driven bulk transfer of monitoring data.
Requirement: R-807129 ../_images/arrow-right-circle.svg
target: VNF or PNF
keyword: SHOULD
introduced: dublin
impacts: dcae, dmaap

The VNF or PNF SHOULD report the files in FileReady for as long as they are available at VNF or PNF.

Note: Recommended period is at least 24 hours.