CPS Temporal Deployment
Deployment
Refer to CPS-Deployment page for deployment documentation related to CPS Temporal and all CPS components.
Once CPS Temporal is successfully deployed and running 2 pods are started, one for running the service and another one for running the database instance:
NAME READY STATUS RESTARTS AGE
cps-temporal-d4cf495b9-bbn7b 1/1 Running 0 8h
cps-temporal-db-0 1/1 Running 0 8h
Configuration
Application Properties
The following table lists some properties that can be specified as Helm chart values to configure the application to be deployed. This list is not exhaustive.
Property |
Description |
Default Value |
|---|---|---|
config.appUserName |
User name used by CPS Temporal service to configure the authentication for REST API it exposes. This is the user name to be used by CPS Temporal REST clients to authenticate themselves. |
|
config.appUserPassword |
Password used by CPS Temporal service to configure the authentication for REST API it exposes. This is the password to be used by CPS Temporal REST clients to authenticate themselves. If not defined, the password is generated when deploying the application. See also CPS Credentials Retrieval |
Not defined |
timescaledb.config.pgUserName |
Internal user name used bt CPS Temporal to connect to its own database. |
|
timescaledb.config.pgUserPassword |
Internal password used bt CPS Temporal to connect to its own database. If not defined, the password is generated when deploying the application. See also Credentials section. |
Not defined |
config.useStrimziKafka |
If targeting a custom kafka cluster, ie useStrimziKakfa: false, the config.eventConsumption.spring.kafka values must be set. |
true |
config.eventConsumption. spring.kafka.bootstrap-servers |
Kafka hostname and port |
|
config.eventConsumption. spring.kafka.consumer.group-id |
Kafka consumer group id |
|
config.eventConsumption. app.listener.data-updated.topic |
Kafka topic to listen to |
|
config.eventConsumption. spring.kafka.security.protocol |
Kafka security protocol. Some possible values are:
|
|
config.eventConsumption. spring.kafka.properties. sasl.mechanism |
Kafka security SASL mechanism. Required for SASL_PLAINTEXT and SASL_SSL protocols. Some possible values are:
|
Not defined |
config.eventConsumption. spring.kafka.properties. sasl.jaas.config |
Kafka security SASL JAAS configuration. Required for SASL_PLAINTEXT and SASL_SSL protocols. Some possible values are:
|
Not defined |
config.eventConsumption. spring.kafka.ssl.trust-store-type |
Kafka security SASL SSL store type. Required for SASL_SSL protocol. Some possible values are:
|
Not defined |
config.eventConsumption. spring.kafka.ssl.trust-store-location |
Kafka security SASL SSL store file location. Required for SASL_SSL protocol. |
Not defined |
config.eventConsumption. spring.kafka.ssl.trust-store-password |
Kafka security SASL SSL store password. Required for SASL_SSL protocol. |
Not defined |
config.eventConsumption. spring.kafka.properties. ssl.endpoint.identification.algorithm |
Kafka security SASL SSL broker hostname identification verification. Required for SASL_SSL protocol. Possible value is:
|
Not defined |
config.additional. app.query.response.max-page-size |
Maximum number of elements that can be retrieved by a single REST API query request using pagination feature. |
|
config.additional. spring.datasource.hikari. maximumPoolSize |
Maximum number of database connections in the connection pool. |
|
Credentials
- Once the deployment is completed, refer to CPS Credentials Retrieval
for more information related to credentials retrieval.
Running With Docker
For development purposes, CPS Temporal can be ran on any environment using Docker. Refer to README.md and docker-compose.yml files for more details.
Upgrade from Istanbul to Jakarta
Cps data-updated-event schema v2
Cps data-updated-event schema v2 used in Jakarta is backward compatible with data-updated-event schema v1 used in Istanbul.
It means that consumers using the schema v2 can process events generated by producers using the schema v2 or v1.
This implies that cps temporal (consumer) must be upgraded from Istanbul to Jakarta at the same time or before cps core (producer) is upgraded from Istanbul to Jakarta.
Database
In Jakarta, CPS Temporal database is upgraded from TimescaleDB 2.1.1 running PostgresSQL 13.2 to
TimescaleDB 2.5.1 running PosgresSQL 14.1. This is a major PostgresSQL upgrade subject to change data storage
format. Then, any existing CPS Temporal data from Istanbul needs to be migrated before it can be used in Jakarta.
The migration needs to be done in 2 main sequential steps for both TimescaleDB and PostgresSQL:
Upgrade TimescaleDB from 2.1.1 to 2.5.1
Upgrade PostgresSQL form 13.2 to 14.1
Bellow are the detailed steps to be completed for data migration (manual docker steps, to be reviewed for production upgrade).
Stop CPS Temporal Istanbul service.
Backup 2.1.1-pg13 data. This backup is to be used to restore data if needed.
# Start timescale 2.1.1-pg13 db instance
docker run --name postgres-cps-temporal-2.1.1-13 -d \
-e POSTGRES_DB=cpstemporaldb \
-e POSTGRES_USER=cpstemporal \
-e POSTGRES_PASSWORD=cpstemporal \
-v cps-temporal_data:/var/lib/postgresql/data \
-v cps-temporal_backup-2.1.1-13:/var/lib/postgresql/backup \
timescale/timescaledb:2.1.1-pg13
docker exec -it postgres-cps-temporal-2.1.1-13 psql -d cpstemporaldb -U cpstemporal -c "select version();"
docker exec -it postgres-cps-temporal-2.1.1-13 psql -d cpstemporaldb -U cpstemporal -c "\dx timescaledb"
# Backup 2.1.1-pg13 data
docker exec -it postgres-cps-temporal-2.1.1-13 pg_basebackup -U cpstemporal -D /var/lib/postgresql/backup/
docker exec -it postgres-cps-temporal-2.1.1-13 ls -l /var/lib/postgresql/backup/
# Stop db instance
docker container stop postgres-cps-temporal-2.1.1-13
docker container rm postgres-cps-temporal-2.1.1-13
Upgrade data from 2.1.1-pg13 to 2.5.1-pg-13
# Start timescale 2.5.1.-pg13
docker run --name postgres-cps-temporal-2.5.1-13 -d \
-e POSTGRES_DB=cpstemporaldb \
-e POSTGRES_USER=cpstemporal \
-e POSTGRES_PASSWORD=cpstemporal \
-v cps-temporal_data:/var/lib/postgresql/data \
-v cps-temporal_dump-2.5.1-13:/var/lib/postgresql/dump \
timescale/timescaledb:2.5.1-pg13
# Upgrade data to 2.5.1-pg13
docker exec -it postgres-cps-temporal-2.5.1-13 psql -d cpstemporaldb -U cpstemporal -c "\dx timescaledb"
docker exec -it postgres-cps-temporal-2.5.1-13 psql -X -d cpstemporaldb -U cpstemporal -c "ALTER EXTENSION timescaledb UPDATE;"
docker exec -it postgres-cps-temporal-2.5.1-13 psql -d cpstemporaldb -U cpstemporal -c "\dx timescaledb"
Create a dump of 2.5.1-pg-13 data
# Dump 2.5.1-pg13 data
docker exec -it postgres-cps-temporal-2.5.1-13 pg_dump -d cpstemporaldb -U cpstemporal -Fc -f /var/lib/postgresql/dump/dumpfile
docker exec -it postgres-cps-temporal-2.5.1-13 ls -l /var/lib/postgresql/dump/dumpfile
# Stop db instance
docker container stop postgres-cps-temporal-2.5.1-13
docker container rm postgres-cps-temporal-2.5.1-13
Upgrade data from 2.5.1-pg13 to 2.5.1-pg-14, by importing the dump in PostgresSQL 14 instance
# Start timescale 2.5.1-pg14
docker run --name postgres-cps-temporal-2.5.1-14 -p 5432:5432 -d \
-e POSTGRES_DB=cpstemporaldb \
-e POSTGRES_USER=cpstemporal \
-e POSTGRES_PASSWORD=cpstemporal \
-v cps-temporal_data-2.5.1-14:/var/lib/postgresql/data \
-v cps-temporal_dump-2.5.1-13:/var/lib/postgresql/dump \
timescale/timescaledb:2.5.1-pg14
docker exec -it postgres-cps-temporal-2.5.1-14 psql -d cpstemporaldb -U cpstemporal -c "select version();"
docker exec -it postgres-cps-temporal-2.5.1-14 psql -d cpstemporaldb -U cpstemporal -c "\dx timescaledb"
# Upgrade data to 2.5.1-pg14 by restoring the dump
docker exec -it postgres-cps-temporal-2.5.1-14 pg_restore -d cpstemporaldb -U cpstemporal -Fc /var/lib/postgresql/dump/dumpfile
docker exec -it postgres-cps-temporal-2.5.1-14 psql -d cpstemporaldb -U cpstemporal -c "select count(*) from network_data;"
# Stop db instances
docker container stop postgres-cps-temporal-2.5.1-14
docker container rm postgres-cps-temporal-2.5.1-14
Copy 2.5.1-pg14 data to data volume
# Start busybox
docker run -it --rm \
-v cps-temporal_data:/data \
-v cps-temporal_data-2.5.1-14:/data-2.5.1-14 \
busybox:1.34.1
# Run commands in busybox
rm -rf /data/*
cp -rp /data-2.5.1-14/* /data
diff -r /data /data-2.5.1-14
exit
Start Cps Temporal Jakarta service
Cleanup volumes that are not needed anymore
For more details about TimescaleDB and PostgresSQL upgrades, refer to: