Details
-
Type:
Monitor
-
Status: Closed
-
Priority:
Major
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: 2021
-
Component/s: FIWARE-TECH-HELP
-
Labels:
-
HD-Enabler:Cygnus
Description
Created question in FIWARE Q/A platform on 17-02-2016 at 09:02
Please, ANSWER this question AT https://stackoverflow.com/questions/35451531/fiware-data-loss-prevention
Question:
Fiware: Data loss prevention
Description:
I’m working with the 0.27.0 version of context broker. I'm using the Cygnus generic enabler and I have established a MQTT agent that connects external devices to the context broker.
My major concern right now is how to prevent from data loss. I established the context broker and the Cygnus mongodb databases as replica sets, but that won't ensure that all data will be persisted into the databases. I have seen that Cygnus uses Apache flume. Looking at its configuration, the re-injection retries can be configured:
- Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries)
cygnusagent.sources.http-source.handler.events_ttl = -1
¿It is a good idea to establish the retries value to -1? I have read about events re-injected in the channel forever.
¿What can be done to ensure that all the data will be persisted?
¿Is there any functionality into fiware ecosystem oriented to that purpose?
Activity
| Transition | Time In Source Status | Execution Times | Last Executer | Last Execution Date | |||||
|---|---|---|---|---|---|---|---|---|---|
|
2h 57m | 1 | Backlog Manager | 22/May/17 6:05 PM | |||||
|
3h | 1 | Backlog Manager | 22/May/17 9:05 PM | |||||
|
6d 14h 32m | 1 | José Ignacio Carretero Guarde | 29/May/17 11:38 AM |
| Fix Version/s | 2021 [ 12600 ] |
| HD-Enabler | Cygnus [ 11304 ] | |
| Description |
Created question in FIWARE Q/A platform on 17-02-2016 at 09:02 {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/35451531/fiware-data-loss-prevention +Question:+ Fiware: Data loss prevention +Description:+ I’m working with the 0.27.0 version of context broker. I'm using the Cygnus generic enabler and I have established a MQTT agent that connects external devices to the context broker. My major concern right now is how to prevent from data loss. I established the context broker and the Cygnus mongodb databases as replica sets, but that won't ensure that all data will be persisted into the databases. I have seen that Cygnus uses Apache flume. Looking at its configuration, the re-injection retries can be configured: # Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries) cygnusagent.sources.http-source.handler.events_ttl = -1 ¿It is a good idea to establish the retries value to -1? I have read about events re-injected in the channel forever. ¿What can be done to ensure that all the data will be persisted? ¿Is there any functionality into fiware ecosystem oriented to that purpose? |
Created question in FIWARE Q/A platform on 17-02-2016 at 09:02
{color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/35451531/fiware-data-loss-prevention +Question:+ Fiware: Data loss prevention +Description:+ I’m working with the 0.27.0 version of context broker. I'm using the Cygnus generic enabler and I have established a MQTT agent that connects external devices to the context broker. My major concern right now is how to prevent from data loss. I established the context broker and the Cygnus mongodb databases as replica sets, but that won't ensure that all data will be persisted into the databases. I have seen that Cygnus uses Apache flume. Looking at its configuration, the re-injection retries can be configured: # Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries) cygnusagent.sources.http-source.handler.events_ttl = -1 ¿It is a good idea to establish the retries value to -1? I have read about events re-injected in the channel forever. ¿What can be done to ensure that all the data will be persisted? ¿Is there any functionality into fiware ecosystem oriented to that purpose? |
| Resolution | Done [ 10000 ] | |
| Status | Answered [ 10104 ] | Closed [ 6 ] |
| Assignee | Francisco Romero [ frb ] |
| Status | In Progress [ 3 ] | Answered [ 10104 ] |
2017-05-22 18:07|UPDATED status: transition Answer| # answers= 1, accepted answer= False
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Field | Original Value | New Value |
|---|---|---|
| Component/s | FIWARE-TECH-HELP [ 10278 ] |
2017-05-22 15:09|CREATED monitor | # answers= 1, accepted answer= False
2017-05-22 21:07|UPDATED status: transition Answered| # answers= 1, accepted answer= False