Multi-Vim logging
The purpose of logging is to generate machine-readable, indexable output logs and support to trace requests through sub-component, it need to ship logs to logging enhancement project a centralized logging analysis system capturing diagnostic information.
Problem Description
So far the logging of multi-vim is not able to support customer configuration, handler context specific logging like MDC MDC_Document , also it dose’t propagate transaction-ID in REST headers which is critical to tracing request. There are 4 python containers in oom project need to configure filebeat container for shipping logs.
In addition the current logging is very difficult to understand behavior and performance.
Proposed Change
The proposed change will include three parts.
Filebeat container
Logging architecture Log_Architecture use Filebeat collects logs from multi-vim containers and ships them to the centralized logging stack. To enable this feature it need to add Filebeat container in multi-vim pod that was deployed by OOM, as well Yaml file will be used to configure Filebeat.
Tracing ID
ONAP logging uses a global unique “RequestID”[RequestID_Document] in logging to track the processing of each request across all the components, multi-vim will receive this id from http header by vary “X-TransactionID”, then record it in logs. Meanwhile single component should generate a InvocationID that records the relationship between RequestID and InvocationID for proper tracing. So Mulit-vim will set unique InvocationID for each single request,also output it in logs.
python AOP logging library
Currently logging enhancement project just has java AOP logging library, For multi-vim which based on python need a python version. The basic feature of AOP logging library could provide customer configuration include retention policy、output location、text output format、message level and so on, support MDC context specific logging, able to change configuration at runtime, and make logging quite fast.
Supporting Python3 version
Right now, this library only has be used in Python2 version. Python2 will not been maintained after 2020, besides part of ONAP project have used python3 version. It’s be better to support Python2 and Python3 version
Markers
Markers can be used to characterize log entries. They allow message that has a specific meaning to be cheaply and easily identified in logger output, without inherently unreliable schemes like scanning for magic strings in the text of each log message. Onap logging requires the emission of markers reporting entry, exit and invocation as the execution if requests pass between ONAP components. This information is used to generate a call graph. Useful and commonplace, See https://stackoverflow.com/questions/4165558/best-practices-for-using-markers-in-slf4j-logback
colored terminal output
As we known, in log4j coloring is supported. It would be better to render logging messages in colors. Bash colors refer: https://en.wikipedia.org/wiki/ANSI_escape_code#Colors
The standard colors(nosupport windows) list as below: +———————+————————-+————————+ | Text color | Hightlight color | Text Attribute | +=====================+==================================================+ | Black | Black | Normal | +———————+————————————————–+ | Red | Red | Bold | +———————+————————————————–+ | Green | Green | Underline | +———————+————————————————–+ | Yellow | Yellow | Blink | +————————————————————————+ | Blue | Blue | Invert | +————————————————————————+ | Purple | Purple | Hide | +————————————————————————+ | Cyan | Cyan | | +————————————————————————+ | White | White | | +————————————————————————+
Test
Unit tests with tox
CSIT tests, verify marker label in logging message