APPC Logging Guide

The APPC modules manage logging functionality according to the logging configuration file org.ops4j.pax.logging.cfg. APPC configuration defines the location of the logging configuration file. Each APPC module configured to generate logs reads the configuration file periodically and generates logs according to the current logging level.

Logging Architecture

The following diagram illustrates the APPC logging architecture.

image0

Log Types

APPC generates several types of logs to capture information needed to operate, troubleshoot, and report on performance:

  • Audit Log: The Audit Log provides a summary view of the processing of a request – a transaction – within an application. It captures activity requests received by APPC and includes such information as the time the activity initiated, when it finishes, and the API invoked at the component.
  • Metric Log: The Metrics Log provides a more detailed view into the processing of a transaction within APP-C. It captures the beginning and ending of activities necessary to complete the transaction within APPC modules, for example, the Dispatcher module, and other ONAP components such as A&AI.
  • Error Log: The Error Log captures INFO, WARN, ERROR, and FATAL conditions sensed (“exception handled”) by the APPC software components.
  • Debug Log: The optional Debug Log captures data that may be required to debug and correct abnormal conditions of the application.

The APPC modules manage logging functionality according to the logging configuration file org.ops4j.pax.logging.cfg. APPC configuration defines the location of the logging configuration file. Each APPC module configured to generate logs reads the configuration file periodically and generates logs according to the current logging level.

Logging Levels

Level Log type
    Errors
  Audit Metrics FATAL ERROR WARN
OFF No No No No No
FATAL Yes Yes Yes No No
ERROR Yes Yes Yes Yes No
WARN Yes Yes Yes Yes Yes

Response Codes

Response code ranges Error type
100-199 Permission errors
200-299 Availability errors/Timeouts
300-399 Data errors
400-499 Schema errors
500-599 Business process errors
900-999 Unknown errors

Logging Output

APPC generates logging output according to Event and Error Logging Framework(EELF)requirements in the following format:

Error Log

Field Description
Timestamp Date-time when error occurs in UTC.
RequestID Universally unique transaction requestID (UUID).
ThreadId

Blank

This traces processing of a request over a number of threads of a single ONAP component.

ServiceName Externally advertised API invoked by clients of this component, for example, “Audit”, “HealthCheck”.
PartnerName Client or user invoking the API.
TargetEntity ONAP component/subcomponent or non-ONAP entity invoked for this suboperation’s activities, for example, A&AI, VF, VM, MySql.
TargetServiceName Known name of External API/operation activities invoked on TargetEntity (ONAP component/subcomponent or non-ONAP entity), for example, VM UUID, VF UUID.
ErrorCategory WARN or ERROR
ErrorCode Code representing the error condition according to the error categories.
ErrorDescription Human readable description of the error - Error name, for example: “CONFIGURATION_STARTED”.
detailMessage

This field may contain any additional information useful in describing the error condition, for example:

Application Controller (APP-C) initialization started at {0}|\ No resolution is required, this is an informational message|\ The APP-C component configuration has been started because the component is being initialized or loaded for the first time, or a new instance of the component is being created. This message indicates that the component is starting.

Debug Log

Field Description
Timestamp Date-time of the log record.
RequestID Universally unique transaction requestID (UUID).
DebugInfo Debug Information
End of Record Designates the logical end of a multi-line debug record.

Audit Log

Field Description
BeginTimestamp Date-time of the start of a request activity.
EndTimestamp Date-time of the end of a request activity.
RequestID Universally unique transaction request ID (UUID).
serviceInstanceID Uniquely identifies a service instance, for example, “service graph”. The primary key, for example, in A&AI, to reference or manage the service instance as a unit.
threadId Empty
physical/virtual server name Empty (the value added by the log files collecting agent).
serviceName Externally advertised API invoked by clients of this component, for example, “Audit”, “HealthCheck”.
PartnerName Client or user invoking the API.
StatusCode High-level success or failure of the request (COMPLETE or ERROR).
ResponseCode Application specific response code - LCM API error codes categorized according to the logging categories.
ResponseDescription Human readable description of the application specific response code, for example, “INVALID INPUT PARAMETER - ${detailedErrorMsg}”.
instanceUUID Universally unique identifier to differentiate between multiple instances of the same (named), log writing component - the specific APPC instance UUID.
Category log level Enum: “INFO” | “WARN” |”DEBUG” | “ERROR” | “FATAL”. Current log level for the entire APP-C.
Severity Blank
Server IP address Blank
ElapsedTime Elapsed time to complete processing of an API or request at the granularity available to the component system. This value should be the difference between BeginTimestamp and EndTimestamp fields.
Server VM FQDN if virtualized, otherwise the host name of the logging component.
ClientIPaddress Requesting remote client application’s IP address if known, otherwise empty.

Metrics Log

Field Description
BeginTimestamp Date-time when a suboperation activity is begun in UTC
EndTimestamp Date-time when a supoperation activity is completed in UTC
RequestID Universally unique transaction request ID (UUID)
serviceInstanceID VMUUID, VFUUID
threadId Optional
physical/virtual server name Empty if its value can be added by the log files collecting agent.
serviceName For example: “Audit”, “HealthCheck” etc
PartnerName Client or user invoking the API  
TargetEntity APPC internal subcomponent, for example, MD-SAL, or external component, for example, A&AI, SSH, Netconf, invoked for this suboperation.
TargetServiceName Operation activities invoked on TargetEntity e.g. A&AI GET generic- vnf
StatusCode High level success or failure of the suboperation activities (COMPLETE or ERROR)
ResponseCode Specific response code returned by the suboperation activities.
ResponseDescription Human readable description of the response code.
instanceUUID APPC instance ID.
Category log level Enumerated values: “INFO” | “WARN” |”DEBUG” | “ERROR” | “FATAL”.
Severity Empty
Server IP address The logging component host server’s IP address.
ElapsedTime Elapsed time to complete processing of the sub operation activities at the granularity available to the component system.  This value should be the difference between EndTimestamp and BeginTimestamp fields.
Server VM FQDN if virtualized, otherwise the host name of the logging component.
ClientIP Requesting remote client application’s IP address.
class name Optional. The name of the class that has caused the log record creation. For OO programing languages that support this concept.
Unused  
ProcessKey Optional
TargetVirtualEntity Empty
CustomField1 Empty (specific attributes exposed by developers)
CustomField2 Empty
CustomField3 Empty
CustomField4 Empty
detailMessage Empty

Log File Locations

The logging configuration file, org.ops4j.pax.logging.cfg are located in appc Git repository:

/appc/appc-common/src/main/resources/org/onap/appc/org.ops4j.pax.logging.cfg

The logs are stored at the location defined by the appropriate appender:

log4j.appender.error.File=${karaf.data}/log/APPC/appc-error.log

log4j.appender.debug.file=${karaf.data}/log/APPC/appc-debug.log

log4j.appender.metric.File=${karaf.data}/log/APPC/appc-metric.log

log4j.appender.audit.File=${karaf.data}/log/APPC/appc-audit.log

Enabling APPC Logging

APPC uses Event and Error Logging Framework (EELF) for application logs. To enable EELF logging:

  1. Replace the default configuration file located at /opt/opendaylight/current/etc/org.ops4j.pax.logging.cfg

    with the configuration file that is checked into git:

    /appc/appc-common/src/main/resources/org/onap/appc/org.ops4j.pax.logging.cfg

  2. Stop and restart ODL controller for the configuration changes to take effect.

  3. Verify logging changes at the following log paths:

    • /opt/opendaylight/current/data/log/eelf/karaf.log This log contains the regular karaf.log output reformatted to use the EELF MDC properties and the pattern that is configured in the org.ops4j.pax.logging.cfg file.
    • /opt/opendaylight/current/data/log/APPC/<package-name> This directory contains the audit, metric, error, and debug logs that are configured in the org.ops4j.pax.logging.cfg file.
Note:
/opt/opendaylight/current/data/log/APPC/controller contains the logs generated from the package org.openecomp.\* (all APPC logs)
  • Error.log: alarms –ERROR level logs and above
  • Info.log: INFO level logs only
  • Debug.log: debugging – DEBUG level and above
  • Audit – AUDIT level and above

Log Rotation

Log rotation is performed after every 100 MB size limit is reached. The log rotation interval is defined as part of the EELF framework.