Created question in FIWARE Q/A platform on 14-12-2021 at 21:12
Please, ANSWER this question AT https://stackoverflow.com/questions/70355924/cygnus-mongo-data-expiration-doesnt-work

Question:
CYGNUS_MONGO_DATA_EXPIRATION doesn't work?

Description:
I installed Orion, MongoDB, Cygnus and sth-comet with docker-compose. I set CYGNUS_MONGO_DATA_EXPIRATION=60 for Cygnus.
I created a subscription to an Orion entity, but the historical data doesn't expire. I want the old data gone.
dockyer-compose.yaml is as follows:
version: "3.5"

services:

1. Databases - Orion uses Mongo-DB, Cygnus is persisting to Postgres
2. Modified to use password authentication between mongoDB and Fiware Orion to improve security in docker.
   mongo-db:
   image: mongo:4.4
   hostname: mongo-db
   container_name: mongo-db
   ports:

• "27017:27017"
  #command: --nojournalS
  environment:
• "MONGO_INITDB_ROOT_USERNAME=$ {MONGOUSERNAME}" # Define the mongoDB user name
  - "MONGO_INITDB_ROOT_PASSWORD=${MONGOPASSWORD}" # Define passwords for security measures within docker
  - "MONGO_INITDB_DATABASE=admin"
  volumes:
  - ./db:/data/db
  - ./configdb:/data/configdb
  
  orion:
  image: fiware/orion:3.3.1
  hostname: orion
  container_name: fiware-orion
  depends_on:
  - mongo-db
  environment:
  - "ORION_MONGO_HOST=mongo-db:27017"
  - "ORION_MONGO_USER=${MONGOUSERNAME}

  " # Define the mongoDB user name

• "ORION_MONGO_PASSWORD=$ {MONGOPASSWORD}" # Define passwords for security measures within docker
  - "ORION_MONGO_AUTH_SOURCE=admin"
  - "ORION_PORT=1026"
  - "ORION_LOG_LEVEL=INFO"
  ports:
  - "1026:1026"
  command: -dbhost mongo-db
  networks:
  #- proxynet #Comment out the openwhisk network as it is not necessary when building the development server.
  - default
  
  # Cygnus is configured to write context data to PostgeSQL
  cygnus:
  image: fiware/cygnus-ngsi:2.14.0
  hostname: cygnus
  container_name: fiware-cygnus
  networks:
  - default
  depends_on:
  - mongo-db
  ports:
  - "5055:5055"
  - "5080:5080"
  environment:
  - "CYGNUS_MONGO_HOSTS=mongo-db:27017" # Hostname of the PostgreSQL server used to persist historical context data
  - "CYGNUS_MONGO_USER=${MONGOUSERNAME}"
  - "CYGNUS_MONGO_PASS=${MONGOPASSWORD}

  "

• "CYGNUS_MONGO_AUTH_SOURCE=admin"
• "CYGNUS_MONGO_SERVICE_PORT=5055"
• "CYGNUS_LOG_LEVEL=DEBUG"
• "CYGNUS_SERVICE_PORT=5055" # Notification Port that Cygnus listens to for Postgres subscriptions
• "CYGNUS_API_PORT=5080" # Port that Cygnus listens on for operational reasons
• "CYGNUS_ORION_SSL=true"
• "CYGNUS_ORION_HOST=orion"
• "CYGNUS_ORION_PORT=1026"
• "CYGNUS_MONGO_DATA_EXPIRATION=60"

sth-comet:
image: fiware/sth-comet
hostname: sth-comet
container_name: fiware-sth-comet
depends_on:

• mongo-db
  networks:
• default
  ports:
• "8666:8666"
  environment:
• "STH_HOST=0.0.0.0"
• "STH_PORT=8666"
• "DB_PREFIX=sth_"
• "DB_USERNAME=$ {MONGOUSERNAME}

  "

• "DB_PASSWORD=$ {MONGOPASSWORD}

  "

• "DB_AUTH_SOURCE=admin"
• "DB_URI=mongo-db:27017"
• "LOGOPS_LEVEL=DEBUG"

volumes:
mongo-db: ~

networks:
default:
ipam:
config:

• subnet: 172.18.1.0/24

Prior to docker-compose up -d, I made the following directories:

db
config

Next, I set a subscription like this:
curl X POST http://localhost:1026/v2/subscriptions -H 'Content-Type: application/json' -d @ <<EOF
{
"description": "Notify all context changes to Cygnus",
"subject": {
"entities": [

]
},
"notification": {
"http":

}
}
EOF

The oriong's log is:
time=2021-12-14T00:32:01.654Z | lvl=INFO | corr=4130a13c-5c75-11ec-b540-0242ac120103 | trans=1639441850-794-00000000001 | from=172.18.1.1 | srv=<none> | subsrv=<none> | comp=Orion | op=logTracing.cpp[148]:logInfoRequestWithPayload | msg=Request received: POST /v2/subscriptions, request payload (217 bytes): { "description": "Notify all context changes to Cygnus", "subject": { "entities": [

] }, "notification": { "http":

}}, response code: 201

Then, I inserted the following entity:
curl localhost:1026/v2/entities s -S -H 'Content-Type: application/json' -d @ <<EOF
{"id":"Room1","type":"Room","temperature":

,"pressure":{"value":720,"type":"Integer"}}
EOF

Orion said(I'm sorry that I lost the head of the log line):
subsrv=<none> | comp=Orion | op=logTracing.cpp[148]:logInfoRequestWithPayload | msg=Request received: POST /v2/entities, request payload (112 bytes): {"id":"Room1","type":"Room","temperature":

,"pressure":{"value":720,"type":"Integer"}}, response code: 201

Then, I updated the entity:
curl localhost:1026/v2/entities/Room1/attrs s -S -H 'Content-Type: application/json' -X PATCH -d @ <<EOF
{"temperature":

,"pressure":{"value":730,"type":"Float"}}
EOF

Orion said:
time=2021-12-14T00:37:12.558Z | lvl=INFO | corr=fa8479ba-5c75-11ec-8b92-0242ac120103 | trans=1639441850-794-00000000007 | from=172.18.1.1 | srv=<none> | subsrv=<none> | comp=Orion | op=logTracing.cpp[148]:logInfoRequestWithPayload | msg=Request received: PATCH /v2/entities/Room1/attrs, request payload (83 bytes): {"temperature":

,"pressure":{"value":730,"type":"Float"}}, response code: 204
time=2021-12-14T00:37:12.591Z | lvl=INFO | corr=fa8479ba-5c75-11ec-8b92-0242ac120103; cbnotif=1 | trans=1639441850-794-00000000008 | from=172.18.1.1 | srv=<none> | subsrv=/ | comp=Orion | op=logTracing.cpp[65]:logInfoHttpNotification | msg=Notif delivered (subId: 61b7e60138b3ce79144d71ab): POST cygnus:5055/notify, response code: 200

I waited some minutes and updated the entity again:
curl localhost:1026/v2/entities/Room1/attrs s -S -H 'Content-Type: application/json' -X PATCH -d @ <<EOF
{"temperature":

,"pressure":{"value":740,"type":"Float"}}
EOF

The log of orion is:
time=2021-12-14T00:40:15.524Z | lvl=INFO | corr=6793036e-5c76-11ec-8fa2-0242ac120103 | trans=1639441850-794-00000000009 | from=172.18.1.1 | srv=<none> | subsrv=<none> | comp=Orion | op=logTracing.cpp[148]:logInfoRequestWithPayload | msg=Request received: PATCH /v2/entities/Room1/attrs, request payload (83 bytes): {"temperature":

,"pressure":{"value":740,"type":"Float"}}, response code: 204
time=2021-12-14T00:40:15.558Z | lvl=INFO | corr=6793036e-5c76-11ec-8fa2-0242ac120103; cbnotif=1 | trans=1639441850-794-00000000010 | from=172.18.1.1 | srv=<none> | subsrv=/ | comp=Orion | op=logTracing.cpp[65]:logInfoHttpNotification | msg=Notif delivered (subId: 61b7e60138b3ce79144d71ab): POST cygnus:5055/notify, response code: 200

Lastly, I printed out the historical data:
It shows the historical table contains the old updating. How can I eliminate the old ones automatically?
The log of cygnus is(It's so big that I made an excerpt of the log):
https://gist.github.com/ishidakei/b3aea7b287899c472b74a438ce4451c8