Created question in FIWARE Q/A platform on 28-07-2020 at 16:07
Please, ANSWER this question AT https://stackoverflow.com/questions/63136089/fiware-iot-agent-data-loss-issue

Question:
Fiware IoT Agent - data loss issue

Description:
We are observing significant data loss in IoT Agent. The data loss happens when IoT agent receives data from our VerneMQ MQTT broker.
Our data is flowing at a rate of 1000/min. When the IoT Agent is started, it works fine for approximately 12 hours, but after 12 hours we see average data loss of 20%.
We have one other subscriber apart from IoT Agent which receives the same data from VerneMQ topic, and that subscriber receives all the data from VerneMQ. The data loss is only in IoT Agent.
We are running IoT Agent and Orion Context Broker on AWS environment using ECS/Fargate services. Each IoT Agent and Orion instance runs in a separate dedicated container.
When IoT Agent is running on one ECS container it works fine without data loss, but when ECS container increased to more than one, there seems to be a data loss. Can you please guide and help us to resolve this issue.
Details about our environment:

IoT Agent version:

Orion Context Broker version:

{ "orion":

}

Environment variables set in IoT Agent: