DMaaP Data Router
Architecture
Capabilities
The DMaaP Data Router (DR) provisioning API is an HTTPS-based, REST-like API for creating and managing DR feeds and subscriptions, which provides a pub/sub architectural model for the transfer of data.
The DR API also supports AAF CADI authorization.
To use this feature, the API client must provide a valid AAF AppID / MechID with each request. To enable this feature, see the Configuration section.
Note
In future releases, AAF CADI auth will be enabled by default.
Usage Scenarios
Typically, DR pub clients request the provisioning of a new DR feed. Once created, DR sub clients can then subscribe to the feed to receive all data published to that feed.
The DR provisioning API is not meant to be used directly by DR end users (publishers and subscribers) for feed / subscription CRUD (create, read, update, delete) operations.
Instead, prospective publishers and subscribers should use the DMaaP Bus Controller API, which will call the DR provisioning API to manage feeds and subscriptions.
See DMaaP Bus Controller API docs for more information:
High level Architecture
The following diagram shows the high-level relationship between the system components:
- DMaaP DR architecture uses the Eclipse Jetty server as an application server to service it’s front-end.
dmaap-dr-prov services all provisioning requests.
dmaap-dr-node services the publishing of data to feed subscribers.
- DMaaP DR uses MariaDB as it’s storage component for the following:
DR Provisioning data. (feeds, subscribers, etc.)
Historical logging data related to feed activity. (Publish, Delivery, etc.)
Offered APIs
DMaaP Data Router Swagger API
To view API documentation in the interactive swagger UI download the JSON file and paste into the swagger tool here: https://editor.swagger.io
DMaaP DR API |
Swagger JSON |
---|---|
DR API |
DMaaP Data Router API
The API Provisioning Model
The DMaaP Data Router (DR) provisioning API defines two resource types - the feed and the subscription, each with JSON representations. The API models the provisioning data as a collection of feeds that are known to the DR (the feeds collection), with each feed containing a collection of the subscriptions to the feed. The standard HTTP operations (POST, GET, PUT, and DELETE), used in conjunction with these resource representations, allow an API user to create, get information about, modify, and delete feeds and subscriptions.
HTTP Service APIs
DMaaP Data Router utilizes an HTTPS REST API to service all transactions. HTTPS and REST standards are followed so clients as varied as CURL, Java applications and even Web Browsers will work to interact with the Data Router.
General HTTP Requirements
A DMaaP Data Router transactions consists of 4 distinct segments, HTTP URL, HTTP Header, HTTP Body (POST/PUT) and HTTP Response. The general considerations for each segment are as follows and are required for each of the specific transactions described in this section.
HTTP URL
http[s]://{serverBaseURL}/{resourcePath}
The serverBaseURL points to DMaaP Data Router host:port that will service the request.
The resourcePath specifies the service that the client is attempting to reach.
HTTP Header
Specifies HTTP Headers, such as Content-Type, that define the parameters of the HTTP Transaction
HTTP Body
The HTTP Body contains the feed content when creating a feed.
HTTP Authorization
The user-id:password pair:
If AAF enabled: A valid AAF AppId to be authenticated and authorized by the AAF CADI framework.
If Non AAF : When publishing or retracting a file, a valid EID Object with publish permissions.
Create a Feed
Description: Creates a unique feed URL to service the publisher/subscriber model.
POST /
Request Parameters:
Field |
Description |
Param Type |
Data Type |
MaxLen |
Set By |
Updatable? |
Required |
Valid/Example Values |
---|---|---|---|---|---|---|---|---|
name |
Feed name |
Body |
String |
<=20 |
Client |
N |
Y |
|
version |
Feed version |
Body |
String |
<=20 |
Client |
N |
Y |
v1.0.0 |
description |
Feed description |
Body |
String |
<=256 |
Client |
Y |
N |
|
business description |
Business description |
Body |
String |
<=256 |
Client |
Y |
N |
|
authorization |
Information for authorizing publishing requests |
Body |
Client |
Y |
Y |
|||
suspend |
Set to true if the feed is in the suspended state |
Body |
Boolean |
Client |
Y |
N |
|
|
group-id |
Auth group for feed management |
Body |
Integer |
Client |
Y |
N |
0 (default) |
|
aaf_instance |
The instance passed to aaf during permission checks |
Body |
String |
<=256 |
Client |
N |
N |
legacy (default) |
Content-Type |
To specify type of message |
Header |
String |
Client |
N |
Y |
application/vnd.dmaap-dr.feed |
|
publisher |
Publisher identity as passed in X-DMAAP-DR-ON-BEHALF-OF at creation time |
Header |
String |
<=8 |
DR |
N |
Y |
username |
Authorization |
The user / AppId to be authorized by the AAF CADI framework |
Header |
String |
Client |
N |
Y if AAF enabled |
dcae@dcae.onap.org:{password} |
|
X-EXCLUDE-AAF |
To determine if the feed to create is legacy or AAF |
Header |
Boolean |
Client |
N |
Y if AAF enabled |
|
|
Links |
URLs related to this feed |
Body |
DR |
N |
N |
Response Codes
- Success:
200
- Error:
See Response Codes
- Consumes
application/json
- Produces
application/json
Sample Request
curl -k -X POST -H "Content-Type:application/vnd.dmaap-dr.feed" -H "X-DMAAP-DR-ON-BEHALF-OF:{user}" --data-ascii @createFeed.json https://{host}:{port}
Sample Body
{
"name": "ONAP Data Feed",
"version": "v1.0",
"authorization": {
"classification": "unclassified",
"endpoint_addrs": [
],
"endpoint_ids": [
{
"id": "dradmin",
"password": "dradmin"
}
]
}
}
Sample Response
{
"suspend": false,
"groupid": 0,
"description": "",
"version": "v1.0",
"authorization": {
"endpoint_addrs": [
],
"classification": "unclassified",
"endpoint_ids": [
{
"password": "dradmin",
"id": "dradmin"
},
{
"password": "demo123456!",
"id": "onap"
}
]
},
"name": "ONAP Data Feed",
"business_description": "",
"aaf_instance": "legacy",
"publisher": "dradmin",
"links": {
"subscribe": "https://dmaap-dr-prov/subscribe/1",
"log": "https://dmaap-dr-prov/feedlog/1",
"publish": "https://dmaap-dr-prov/publish/1",
"self": "https://dmaap-dr-prov/feed/1"
}
}
Update a Feed
Description: Update a feed with new parameters.
PUT /feed/{feedId}
Request Parameters:
Field |
Description |
Param Type |
Data Type |
MaxLen |
Required |
---|---|---|---|---|---|
name |
Feed name |
Body |
String |
<=20 |
Y |
description |
Feed description |
Body |
String |
<=256 |
N |
business description |
Business description |
Body |
String |
<=256 |
N |
authorization |
Information for authorizing publishing requests |
Body |
Y |
||
suspend |
Set to true if the feed is in the suspended state |
Body |
Boolean |
N |
|
group-id |
Auth group for feed management |
Body |
Integer |
N |
|
Content-type |
To specify type of message (feed,subscriber,publisher) |
Header |
String |
Y |
|
publisher |
Publisher identity as passed in X-DMAAP-DR-ON-BEHALF-OF at creation time |
Header |
String |
<=8 |
Y |
Authorization |
The user / AppId to be authorized by the AAF CADI framework |
Header |
String |
Y if AAF enabled |
Response Codes
- Success:
200
- Error:
See Response Codes
- Consumes
application/json
- Produces
application/json
Sample Request
curl -k -X PUT -H "Content-Type: application/vnd.dmaap-dr.feed" -H "X-DMAAP-DR-ON-BEHALF-OF: {user}" --data-ascii @updateFeed.json --location-trusted https://{host}:{port}/feed/{feedId}
Sample Body
{
"name": "ONAP Data Feed",
"business_description": "Updated ONAP Feed",
"groupid": 33,
"description": "Updated ONAP Feed",
"authorization": {
"endpoint_addrs": [
"10.1.2.3"
],
"classification": "unclassified",
"endpoint_ids": [
{
"password": "dradmin",
"id": "dradmin"
},
{
"password": "demo123456!",
"id": "onap"
}
]
}
}
Sample Response
{
"suspend": false,
"groupid": 33,
"description": "Updated ONAP Feed",
"authorization": {
"endpoint_addrs": [
"10.1.2.3"
],
"classification": "unclassified",
"endpoint_ids": [
{
"password": "dradmin",
"id": "dradmin"
},
{
"password": "demo123456!",
"id": "onap"
}
]
},
"name": "ONAP Data Feed1",
"business_description": "Updated ONAP Feed",
"aaf_instance": "legacy",
"publisher": "dradmin",
"links": {
"subscribe": "https://dmaap-dr-prov/subscribe/1",
"log": "https://dmaap-dr-prov/feedlog/1",
"publish": "https://dmaap-dr-prov/publish/1",
"self": "https://dmaap-dr-prov/feed/1"
}
}
Get a Feed
Description: Retrieves a representation of the specified feed.
GET /feed/{feedId}
Request Parameters:
Field |
Description |
Param Type |
Data Type |
MaxLen |
Required |
---|---|---|---|---|---|
publisher |
Publisher identity as passed in X-DMAAP-DR-ON-BEHALF-OF at creation time |
Header |
String |
<=8 |
Y |
Authorization |
The user / AppId to be authorized by the AAF CADI framework |
Header |
String |
Y if AAF enabled |
Response Codes
- Success:
200
- Error:
See Response Codes
- Produces
application/json
Sample Request
curl -k -H "X-DMAAP-DR-ON-BEHALF-OF: {user}" https://{host}:{port}/feed/{feedId}
Sample Response
{
"suspend": false,
"groupid": 33,
"description": "Updated ONAP Feed",
"version": "v1.0",
"authorization": {
"endpoint_addrs": [
"10.1.2.3",
"173.2.33.4"
],
"classification": "unclassified",
"endpoint_ids": [
{
"password": "dradmin",
"id": "dradmin"
},
{
"password": "demo123456!",
"id": "onap"
}
]
},
"name": "ONAP Data Feed",
"business_description": "Updated ONAP Feed",
"aaf_instance": "legacy",
"publisher": "dradmin",
"links": {
"subscribe": "https://dmaap-dr-prov/subscribe/1",
"log": "https://dmaap-dr-prov/feedlog/1",
"publish": "https://dmaap-dr-prov/publish/1",
"self": "https://dmaap-dr-prov/feed/1"
}
}
Delete a Feed
Description: Deletes a specified feed
DELETE /feed/{feedId}
Request Parameters:
Field |
Description |
Param Type |
Data Type |
MaxLen |
Required |
---|---|---|---|---|---|
publisher |
Publisher identity as passed in X-DMAAP-DR-ON-BEHALF-OF at creation time |
Header |
String |
<=8 |
Y |
Authorization |
The user / AppId to be authorized by the AAF CADI framework |
Header |
String |
Y if AAF enabled |
Response Codes
- Success:
204
- Error:
See Response Codes
Sample Request
curl -k -X DELETE -H "X-DMAAP-DR-ON-BEHALF-OF: {user}" https://{host}:{port}/feed/{feedId}
Subscribe to Feed
Description: Subscribes to a created feed to receive files published to that feed.
POST /subscribe/{feedId}
Request Parameters:
Field |
Description |
Param Type |
Data Type |
MaxLen |
Set By |
Updatable? |
Required |
Valid/Example Values |
---|---|---|---|---|---|---|---|---|
feedId |
ID for the feed you are subscribing to |
Path |
String |
Client |
N |
Y |
1 |
|
delivery |
Address and credentials for delivery |
Body |
Client |
Y |
Y |
|||
follow_redirect |
Set to true if feed redirection is expected |
Body |
Boolean |
Client |
Y |
N |
|
|
metadata_only |
Set to true if subscription is to receive per-file metadata |
Body |
Boolean |
Client |
Y |
Y |
|
|
suspend |
Set to true if the subscription is in the suspended state |
Body |
Boolean |
Client |
Y |
N |
|
|
decompress |
Set to true if the data is to be decompressed for subscriber |
Body |
Boolean |
Client |
Y |
N |
|
|
group-id |
Auth group for sub management |
Body |
Integer |
Client |
Y |
Y |
22 |
|
aaf_instance |
The instance passed to aaf during permission checks |
Body |
String |
<=256 |
Client |
N |
N |
|
Content-type |
To specify type of message (feed,subscriber,publisher) |
Header |
String |
Client |
N |
Y |
application/vnd.dmaap-dr.subscription |
|
subscriber |
Subscriber identity as passed in X-DMAAP-DR-ON-BEHALF-OF at creation time |
Header |
String |
<=8 |
DR |
N |
Y |
username |
Authorization |
The user / AppId to be authorized by the AAF CADI framework |
Header |
String |
Client |
N |
Y if AAF enabled |
dcae@dcae.onap.org:{password} |
|
X-EXCLUDE-AAF |
To determine if the feed to create is legacy or AAF |
Header |
Boolean |
Client |
N |
Y if AAF enabled |
|
|
Links |
URLs related to this subscription |
Body |
DR |
N |
N |
Response Codes
- Success:
201
- Error:
See Response Codes
- Consumes
application/json
- Produces
application/json
Sample Request
curl -k -X POST -H "Content-Type:application/vnd.dmaap-dr.subscription" -H "X-DMAAP-DR-ON-BEHALF-OF:{user}" --data-ascii @addSubscriber.json https://{host}:{port}/subscribe/{feedId}
Sample Body
{
"delivery": {
"url": "http://dmaap-dr-subscriber:7070/",
"user": "LOGIN",
"password": "PASSWORD",
"use100": true
},
"metadataOnly": false,
"groupid": 22,
"follow_redirect": true,
"suspend": false,
"decompress": true
}
Sample Response
{
"suspend": false,
"delivery": {
"use100": true,
"password": "PASSWORD",
"user": "LOGIN",
"url": "http://dmaap-dr-subscriber:7070/"
},
"subscriber": "onap",
"groupid": 1,
"metadataOnly": false,
"follow_redirect": true,
"decompress": true,
"aaf_instance": "legacy",
"links": {
"feed": "https://dmaap-dr-prov/feed/1",
"log": "https://dmaap-dr-prov/sublog/1",
"self": "https://dmaap-dr-prov/subs/1"
},
"created_date": 1553707279509
}
Update subscription
Description: Update a subscription to a feed.
PUT /subs/{subId}
Request Parameters:
Field |
Description |
Param Type |
Data Type |
MaxLen |
Required |
---|---|---|---|---|---|
subId |
ID for the subscription you are updating |
Path |
String |
Y |
|
delivery |
Address and credentials for delivery |
Body |
Y |
||
follow_redirect |
Set to true if feed redirection is expected |
Body |
Boolean |
N |
|
metadata_only |
Set to true if subscription is to receive per-file metadata |
Body |
Boolean |
Y |
|
suspend |
Set to true if the subscription is in the suspended state |
Body |
Boolean |
N |
|
decompress |
Set to true if the data is to be decompressed for subscriber |
Body |
Boolean |
N |
|
group-id |
Auth group for sub management |
Body |
Integer |
Y |
|
Content-type |
To specify type of message (feed,subscriber,publisher) |
Header |
String |
Y |
|
subscriber |
Subscriber identity as passed in X-DMAAP-DR-ON-BEHALF-OF at creation time |
Header |
String |
<=8 |
Y |
Authorization |
The user / AppId to be authorized by the AAF CADI framework |
Header |
String |
Y if AAF enabled |
|
X-EXCLUDE-AAF |
To determine if the feed to create is legacy or AAF |
Header |
Boolean |
Y if AAF enabled |
Response Codes
- Success:
200
- Error:
See Response Codes
- Consumes
application/json
- Produces
application/json
Sample Request
curl -k -X PUT -H "Content-Type:application/vnd.dmaap-dr.subscription" -H "X-DMAAP-DR-ON-BEHALF-OF:{user}" --data-ascii @updateSubscriber.json https://{host}:{port}/subs/{subId}
Sample Body
{
"delivery": {
"url": "http://dmaap-dr-subscriber:7070/",
"user": "NEW_LOGIN",
"password": "NEW_PASSWORD",
"use100": false
},
"metadataOnly": true,
"groupid": 67,
"follow_redirect": false,
"decompress": false
}
Sample Response
{
"suspend": false,
"delivery": {
"use100": false,
"password": "NEW_PASSWORD",
"user": "NEW_LOGIN",
"url": "http://dmaap-dr-subscriber:7070/"
},
"subscriber": "onap",
"groupid": 67,
"metadataOnly": true,
"follow_redirect": false,
"decompress": false,
"aaf_instance": "legacy",
"links": {
"feed": "https://dmaap-dr-prov/feed/1",
"log": "https://dmaap-dr-prov/sublog/1",
"self": "https://dmaap-dr-prov/subs/1"
},
"created_date": 1553714446614
}
Get a Subscription
Description: Retrieves a representation of the specified subscription.
GET /subs/{subId}
Request Parameters:
Field |
Description |
Param Type |
Data Type |
MaxLen |
Required |
---|---|---|---|---|---|
subscriber |
Subscriber identity as passed in X-DMAAP-DR-ON-BEHALF-OF at creation time |
Header |
String |
<=8 |
Y |
Authorization |
The user / AppId to be authorized by the AAF CADI framework |
Header |
String |
Y if AAF enabled |
Response Codes
- Success:
200
- Error:
See Response Codes
- Produces
application/json
Sample Request
curl -k -H "X-DMAAP-DR-ON-BEHALF-OF:{user}" https://{host}:{port}/subs/{subId}
Sample Response
{
"suspend": false,
"delivery": {
"use100": false,
"password": "NEW_PASSWORD",
"user": "NEW_LOGIN",
"url": "http://dmaap-dr-subscriber:7070/"
},
"subscriber": "onap",
"groupid": 67,
"metadataOnly": true,
"privilegedSubscriber": false,
"follow_redirect": false,
"decompress": false,
"aaf_instance": "legacy",
"links": {
"feed": "https://dmaap-dr-prov/feed/2",
"log": "https://dmaap-dr-prov/sublog/6",
"self": "https://dmaap-dr-prov/subs/6"
}
}
Delete a subscription
Description: Deletes a specified subscription
DELETE /subs/{subId}
Request Parameters:
Field |
Description |
Param Type |
Data Type |
MaxLen |
Required |
---|---|---|---|---|---|
subscriber |
Subscriber identity as passed in X-DMAAP-DR-ON-BEHALF-OF at creation time |
Header |
String |
<=8 |
Y |
Authorization |
The user / AppId to be authorized by the AAF CADI framework |
Header |
String |
Y if AAF enabled |
Response Codes
- Success:
204
- Error:
See Response Codes
Sample Request
curl -k -X DELETE -H "X-DMAAP-DR-ON-BEHALF-OF:{user}" https://{host}:{port}/subs/{subId}
Publish to Feed
Description: Publish data to a given feed
PUT /publish/{feedId}/{fileId}
Request parameters
Name |
Description |
Param Type |
Data Type |
MaxLen |
Required |
Valid/Example Values |
---|---|---|---|---|---|---|
feedId |
ID of the feed you are publishing to |
Path |
String |
Y |
||
fileId |
Name of the file when it is published to subscribers |
Path |
String |
Y |
||
Content-type |
To specify type of message format |
Header |
String |
Y |
application/octet-stream |
|
X-DMAAP-DR-META |
Metadata for the file. Accepts only non nested json objects of the following type:
|
Header |
String |
<=4096 |
N |
|
Authorization |
An EID Object with publish permissions. If AAF CADI is enabled, use a valid AAF user/AppId instead. |
Header |
String |
Y |
|
Response Codes
- Success:
204
- Error:
See Response Codes
Sample Request
curl -k -X PUT --user {user}:{password} -H "Content-Type:application/octet-stream" -H "X-DMAAP-DR-META:{\"filetype\":\"txt\"}" --data-binary @sampleFile.txt --post301 --location-trusted https://{host}:{port}/publish/{feedId}/{fileId}
Delete/Retract a Published file
Description: Deletes/retracts a specified published file
DELETE /publish/{feedId}/{fileId}
Request Parameters:
Field |
Description |
Param Type |
Data Type |
Required |
Valid/Example Values |
---|---|---|---|---|---|
Authorization |
An EID Object with publish permissions. If AAF CADI is enabled, use a valid AAF user/AppId instead. |
Header |
String |
Y |
|
feedId |
ID of the feed that was publishing to |
Path |
String |
Y |
|
fileId |
Name of the file when it was published to subscribers |
Path |
String |
Y |
Response Codes
- Success:
204
- Error:
See Response Codes
Sample Request
curl -k -X DELETE --user {user}:{password} --location-trusted https://{host}:{port}/publish/{feedId}/{fileId}
Feed logging
Description: View logging information for specified feeds, which can be narrowed down with further parameters
GET /feedlog/{feedId}?{queryParam}
Request parameters
Name |
Description |
Param Type |
Data Type |
Required |
Valid/Example Values |
---|---|---|---|---|---|
feedId |
Id of the feed you want logs for |
Path |
String |
Y |
1 |
type |
Select records of the specified type |
Path |
String |
N |
|
publishId |
Select records with specified publish id, carried in the X-DMAAP-DR-PUBLISH-ID header from original publish request |
Path |
String |
N |
|
start |
Select records created at or after specified date |
Path |
String |
N |
A date-time expressed in the format specified by RFC 3339 |
end |
Select records created at or before specified date |
Path |
String |
N |
A date-time expressed in the format specified by RFC 3339 |
statusCode |
Select records with the specified statusCode field |
Path |
String |
N |
An HTTP Integer status code or one of the following special values:
|
expiryReason |
Select records with the specified expiry reason |
Path |
String |
N |
|
filename |
Select published records with the specified filename |
Path |
String |
N |
Response Parameters
Name |
Description |
---|---|
type |
Record type:
|
date |
The UTC date and time at which the record was generated, with millisecond resolution in the format specified by RFC 3339 |
publishId |
The unique identifier assigned by the DR at the time of the initial publication request (carried in the X-DMAAP-DR-PUBLISH-ID header in the response to the original publish request) |
requestURI |
The Request-URI associated with the request |
method |
The HTTP method (PUT or DELETE) for the request |
contentType |
The media type of the payload of the request |
contentLength |
The size (in bytes) of the payload of the request |
sourceIp |
The IP address from which the request originated |
endpointId |
The identity used to submit a publish request to the DR |
deliveryId |
The identity used to submit a delivery request to a subscriber endpoint |
statusCode |
The HTTP status code in the response to the request. A value of -1 indicates that the DR was not able to obtain an HTTP status code |
expiryReason |
The reason that delivery attempts were discontinued:
|
attempts |
Total number of attempts made before delivery attempts were discontinued |
filename |
File name associated with a publish record |
Response Codes
- Success:
200
- Error:
See Response Codes
- Produces
application/json
Sample Request
curl -k https://{host}:{port}/feedlog/{feedId}?statusCode=204``
Sample Response
[
{
"statusCode": 204,
"publishId": "1553715307322.dmaap-dr-node",
"requestURI": "https://dmaap-dr-node/publish/1/hello",
"sourceIP": "172.19.0.1",
"method": "PUT",
"contentType": "application/octet-stream",
"endpointId": "dradmin",
"type": "pub",
"date": "2019-03-27T19:35:07.324Z",
"contentLength": 14,
"fileName": "hello"
},
{
"statusCode": 204,
"publishId": "1553715312071.dmaap-dr-node",
"requestURI": "https://dmaap-dr-node/publish/2/hello",
"sourceIP": "172.19.0.1",
"method": "PUT",
"contentType": "application/octet-stream",
"endpointId": "onap",
"type": "pub",
"date": "2019-03-27T19:35:12.072Z",
"contentLength": 14,
"fileName": "hello2"
}
]
Subscription logging
Description: View logging information for specified subscriptions, which can be narrowed down with further parameters
GET /sublog/{subId}?{queryParam}
Request parameters
Name |
Description |
Param Type |
Data Type |
Required |
Valid/Example Values |
---|---|---|---|---|---|
subId |
Id of the subscription you want logs for |
Path |
String |
Y |
1 |
type |
Select records of the specified type |
Path |
String |
N |
|
publishId |
Select records with specified publish id, carried in the X-DMAAP-DR-PUBLISH-ID header from original publish request |
Path |
String |
N |
|
start |
Select records created at or after specified date |
Path |
String |
N |
A date-time expressed in the format specified by RFC 3339 |
end |
Select records created at or before specified date |
Path |
String |
N |
A date-time expressed in the format specified by RFC 3339 |
statusCode |
Select records with the specified statusCode field |
Path |
String |
N |
An Http Integer status code or one of the following special values:
|
expiryReason |
Select records with the specified expiry reason |
Path |
String |
N |
Response Parameters
Name |
Description |
---|---|
type |
Record type:
|
date |
The UTC date and time at which the record was generated, with millisecond resolution in the format specified by RFC 3339 |
publishId |
The unique identifier assigned by the DR at the time of the initial publication request(carried in the X-DMAAP-DR-PUBLISH-ID header in the response to the original publish request) to a feed log URL or subscription log URL known to the system |
requestURI |
The Request-URI associated with the request |
method |
The HTTP method (PUT or DELETE) for the request |
contentType |
The media type of the payload of the request |
contentLength |
The size (in bytes) of the payload of the request |
sourceIp |
The IP address from which the request originated |
endpointId |
The identity used to submit a publish request to the DR |
deliveryId |
The identity used to submit a delivery request to a subscriber endpoint |
statusCode |
The HTTP status code in the response to the request. A value of -1 indicates that the DR was not able to obtain an HTTP status code |
expiryReason |
The reason that delivery attempts were discontinued:
|
attempts |
Total number of attempts made before delivery attempts were discontinued |
Response Codes
- Success:
200
- Error:
See Response Codes
- Produces
application/json
Sample Request
curl -k https://{host}:{port}/sublog/{subId}?statusCode=204
Sample Response
[
{
"statusCode": 204,
"publishId": "1553715307322.dmaap-dr-node",
"requestURI": "https://dmaap-dr-node/publish/1/hello",
"sourceIP": "172.19.0.1",
"method": "PUT",
"contentType": "application/octet-stream",
"endpointId": "dradmin",
"type": "pub",
"date": "2019-03-27T19:35:07.324Z",
"contentLength": 14,
"fileName": "hello"
},
{
"statusCode": 204,
"publishId": "1553715312071.dmaap-dr-node",
"requestURI": "https://dmaap-dr-node/publish/2/hello",
"sourceIP": "172.19.0.1",
"method": "PUT",
"contentType": "application/octet-stream",
"endpointId": "onap",
"type": "pub",
"date": "2019-03-27T19:35:12.072Z",
"contentLength": 14,
"fileName": "hello2"
}
]
Feed Authorization Object
Field |
Type |
Description |
Restrictions |
---|---|---|---|
classification |
string |
An indicator of the feed’s data security classification |
Length <=32 |
endpoint_ids |
EID Object [] |
Array of objects defining the identities that are allowed to publish to this feed |
At least 1 id in the array |
endpoint_addrs |
string[] |
Array of IP addresses or IP subnetwork addresses that are allowed to publish to this feed; an empty array indicates that publish requests are permitted from any IP address |
Each string must be a valid textual representation of IPv4 or IPv6 host address or subnetwork address. |
Endpoint Identity Object
Field |
Type |
Description |
Restrictions |
---|---|---|---|
id |
string |
Publishing endpoint identifier |
Length <= 20 |
password |
string |
Password associated with id |
Length <= 32 |
Feed Links Object
Field |
Description |
Symbolic Name |
---|---|---|
self |
URL pointing to this feed, used for updating and deleting the feed. |
<feedURL> |
publish |
URL for publishing requests for this feed |
<publishURL> |
subscribe |
URL for subscribing to this feed |
<subscribeURL> |
log |
URL for accessing log information about this feed |
<feedLogURL> |
Delivery Object
Field |
Type |
Description |
Restrictions |
---|---|---|---|
url |
string |
URL to which deliveries for this subscription should be directed Valid HTTPS URL |
length <= 256 |
user |
string |
User ID to be passed in the Authorization header when deliveries are made |
Length <= 20 |
password |
string |
Password to be passed in the Authorization header when deliveries are made |
Length <= 32 |
use100 |
boolean |
Flag indicating whether the DR should use the HTTP 100-continue feature |
Must be: true to use 100-continue false to disable using 100-continue |
Sub Links Object
Field |
Description |
Symbolic Name |
---|---|---|
self |
URL pointing to this subscription, used for updating and deleting the subscription. |
<subscriptionURL> |
feed |
URL of the feed to which this subscription applies; the same URL as the <feedURL> in the representation of the feed |
<feedURL> |
log |
URL for accessing log information about this subscription |
<subLogURL> |
Response/Error Codes
Response statusCode |
Response Description |
---|---|
200 to 299 |
Success Response |
400 |
Bad request - The request is defective in some way. Possible causes:
|
401 |
Indicates that the request was missing the Authorization header or, if the header was presented, the credentials were not acceptable |
403 |
Forbidden - The request failed authorization. Possible causes:
|
404 |
Not Found - The Request-URI does not point to a resource that is known to the API. |
405 |
Method Not Allowed - The HTTP method in the request is not supported for the resource addressed by the Request-URI. |
406 |
Not Acceptable - The request has an Accept header indicating that the requester will not accept a response with application/vnd.dmaap-dr.log-list content. |
415 |
Unsupported Media Type - The media type in the requests Content-Type header is not appropriate for the request. |
500 |
Internal Server Error - The DR API server encountered an internal error and could not complete the request. |
503 |
Service Unavailable - The DR API service is currently unavailable |
-1 |
Failed Delivery |
Delivery
Below is a diagram of the DMaaP Data Router project docker containers and the connections between them.
Logging
Where to Access Information Data Router uses logback framework to generate logs.
Error / Warning Messages Currently Data Router does not have any unique error codes. However the following are the common HTTP error codes that could possibly occur in Data Router:
OK = 200 : The normal response from a successful update or get of a feed or subscription
CREATED = 201 : the normal response from successfully creating or subscribing to a feed.
NO_CONTENT = 204 : the normal response from a successful publish attempt and on successfully deleting a feed or subscription
MOVED_PERMANENTLY = 301 :the normal redirect response from prov to a publisher
BAD_REQUEST = 400: Usually indicates that either Json object in request body is incorrect in some way, or an Invalid parameter value was included in query string.
UNAUTHORIZED = 401 : Usually indicated either request was missing Authorization header, or indicates incorrect Username/password credentials
FORBIDDEN = 403 : Usually indicates the request originated from an unauthorized IP address, or that a client certificate was not a part of authorized list.
NOT_FOUND = 404 : Usually indicates an incorrect URI
METHOD_NOT_ALLOWED = 405 : Indicates an HTTP method is not accepted for given URI
Installation
Standalone Pre-requisites:
docker 18.09.3 or higher.
docker-compose 1.17.1 or higher.
Ensure port 8080 is not already in use.
Clone the DMaaP Data Router project from ONAP gerrit:
git clone https://gerrit.onap.org/r/dmaap/datarouter
Move/cd to the docker-compose directory and execute the following:
cd datarouter/datarouter-docker-compose/src/main/resources/
docker-compose up -d
The following docker containers should be deployed successfully:
docker ps --format '{{.Image}}'
nexus3.onap.org:10001/onap/dmaap/datarouter-node
nexus3.onap.org:10001/onap/dmaap/datarouter-prov
nexus3.onap.org:10001/onap/dmaap/datarouter-subscriber
mariadb:10.2.14
To verify that the provisioning API is active, get the IP of the datarouter-prov container:
docker inspect --format '{{ .NetworkSettings.Networks.resources_testing_net.IPAddress }}' datarouter-prov
and execute the following CURL command:
curl -k https://{DR_PROV_CONTAINER_IP}:8443/internal/prov
Configuration
Configuration properties for both Data Router Provisioning server and Data Router Node server should remain as default values.
The only exception to this, is when enabling the AAF CADI framework to authorize the DR provisioning requests.
Note
The AAF CADI filtering feature is disabled by default. When AAF CADI auth is enabled, all DR API calls must provide an AAF AppID to access the relevant API endpoint.
To enable DR AAF CADI authorization, the following steps are required:
- DR CADI Prerequisites:
AAF deployment
Update the following properties at deployment time.
DMaaP DR Prov AAF properties
# AAF config
org.onap.dmaap.datarouter.provserver.cadi.enabled = true
# AAF URL to connect to AAF server
org.onap.dmaap.datarouter.provserver.cadi.aaf.url = https://<RELEVANT_AAF_URL>:8095
DMaaP DR Node AAF properties
# AAF URL to connect to AAF server
AafUrl = https://<RELEVANT_AAF_URL>:8095
# AAF CADI enabled flag
CadiEnabled = true
Release Notes
Istanbul
Abstract
This document provides the release notes for the DMaaP DR Istanbul release.
Summary
No big changes for Istanbul Release.
Release Data
DMaaP Data Router Project |
|
Docker images |
|
Release designation |
9.0.0 Istanbul |
Release date |
2021-10-14 |
New features
Package upgrades - DMAAP-1608
Move csits to dr repo - DMAAP-1636
Sonar fixes - DMAAP-1623
Known Limitations, Issues and Workarounds
System Limitations
Known Vulnerabilities
Workarounds
Security Notes
References
For more information on the ONAP Istanbul release, please see:
Honolulu
Abstract
This document provides the release notes for the DMaaP DR Honolulu release.
Summary
No big changes for Honolulu Release.
Release Data
DMaaP Data Router Project |
|
Docker images |
|
Release designation |
8.0.0 Honolulu |
Release date |
2021-04-01 |
New features
Update commons-codec (listed in “Known Vulnerabilities” below) - DMAAP-1515
Change openjdk baseOS imgs to ONAP/integration- - INT-1864
Update Project Lead Information - DMAAP-1538
Known Limitations, Issues and Workarounds
System Limitations
Known Vulnerabilities
SONATYPE-2012-0050
Workarounds
Security Notes
References
For more information on the ONAP Honolulu release, please see:
Guilin
Abstract
This document provides the release notes for the DMaaP DR Guilin release.
Summary
Release Data
DMaaP Data Router Project |
|
Docker images |
|
Release designation |
7.0.0 guilin |
Release date |
2020-10-19 (TBD) |
New features
Known Limitations, Issues and Workarounds
System Limitations
Known Vulnerabilities
Workarounds
Security Notes
Frankfurt
Abstract
This document provides the release notes for the DMaaP DR Frankfurt release.
Summary
Release Data
DMaaP Data Router Project |
|
Docker images |
|
Release designation |
6.0.0 frankfurt |
Release date |
2020-05-14 (TBD) |
New features
Implement boolean flag in OOM to disable HTTP ports
Dynamic cert distribution from OOM AAF
Known Limitations, Issues and Workarounds
System Limitations
Known Vulnerabilities
CVE-2019-11358
Workarounds
Security Notes
References
For more information on the ONAP Frankfurt release, please see:
El Alto
Version: 2.1.2
- Release Date
2019-09-05
New Features:
JIRA ID |
Description |
---|---|
DMAAP-1227 |
Updating logging functionality to log events into correct log files as specified in logging spec |
DMAAP-1228 |
Updating Logging pattern to match logging spec |
DMAAP-1049 |
[DR] Update DR logging to match Platform maturity Logging Spec |
Bug Fixes:
JIRA ID |
Description |
---|---|
DMAAP-1421 |
[DR] ElAlto AAF certs expired |
Known Issues: N/A
Security Issues: N/A
Upgrade Notes: N/A
Deprecation Notes: N/A
Other: N/A
Dublin
Version: 2.1.0
- Release Date
2019-06-06
New Features:
JIRA ID |
Description |
---|---|
DMAAP-978 |
[DR] Query of publication history (new API) for use by Data File Collector) |
DMAAP-980 |
[DR] Optional consumer compression feed handling |
DMAAP-1016 |
DR provisioning AAF integration |
Bug Fixes:
JIRA ID |
Description |
---|---|
DMAAP-964 |
[DMAAP] DMAAP deployment failures starting 20190115 on casablanca branch |
DMAAP-1010 |
[DR] DMaaP Data Router fails healthcheck |
DMAAP-1047 |
[DR] Data Router docker version missing explicit version number |
DMAAP-1048 |
[DR] AAF certs expired on dmaap-dr-prov and dmaap-dr-node |
DMAAP-1161 |
[DR] filebeat container on DR-Node and DR-Prov are unable to publish to kibana |
Known Issues: N/A
Security Issues:
Fixed Security Issues
Known Security Issues
In default deployment DMAAP (dmaap-dr-prov) exposes HTTP port 30259 outside of cluster. [OJSI-158]
Known Vulnerabilities in Used Modules
DMAAP code has been formally scanned during build time using NexusIQ and all Critical vulnerabilities have been addressed, items that remain open have been assessed for risk and determined to be false positive. The DMAAP open Critical security vulnerabilities and their risk assessment have been documented as part of the project.
Upgrade Notes: N/A
Deprecation Notes: N/A
Other: N/A
Casablanca Maintenance
Version: 1.0.8
- Release Date
2019-02-28
New Features:
JIRA ID |
Description |
---|---|
Bug Fixes:
JIRA ID |
Description |
---|---|
DMAAP-1065 |
[DR] Casablanca - AAF certs expired on dmaap-dr-prov and dmaap-dr-node |
Known Issues: N/A
Security Issues: DMAAP code has been formally scanned during build time using NexusIQ and all Critical vulnerabilities have been addressed, items that remain open have been assessed for risk and determined to be false positive. The DMAAP open Critical security vulnerabilities and their risk assessment have been documented as part of the project.
Upgrade Notes: N/A
Deprecation Notes: N/A
Other: N/A
Casablanca
Version: 1.0.3
- Release Date
2018-11-30
New Features:
JIRA ID |
Description |
---|---|
DMAAP-20 |
REST api for publishing data to DR |
DMAAP-21 |
REST api for subscribing to data in DR |
Bug Fixes:
JIRA ID |
Description |
---|---|
DMAAP-877 |
DR Logging API not storing Feed/Sub data |
DMAAP-850 |
Second subscriber not receiving the published file |
DMAAP-596 |
DR - datarouter-prov container fails to come up successfully |
DMAAP-565 |
Incorrect nexusUrl parameter in datarouter pom files |
Known Issues: N/A
Security Issues: DMAAP code has been formally scanned during build time using NexusIQ and all Critical vulnerabilities have been addressed, items that remain open have been assessed for risk and determined to be false positive. The DMAAP open Critical security vulnerabilities and their risk assessment have been documented as part of the project.
Upgrade Notes: N/A
Deprecation Notes: N/A
Other: N/A