Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-2099

FIWARE.Request.Tech.IoT.IDAS.IDAS.CollectingDataIssue

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-TECH-HELP
    • Labels:
      None
    • HD-Chapter:
      IoT
    • HD-Enabler:
      IDAS

      Description

      Hi All,
      any update on this?

      This is still a blocking issue for FINESCE-WP4 Orion performances.
      @IdasTeam: can we have a joint phone call to solve this?

      Kind regards,
      P.

      Pasquale Andriani
      Direzione Ricerca e Innovazione - Research & Development Lab
      pasquale.andriani@eng.it

      Engineering Ingegneria Informatica spa
      Via Riccardo Morandi, 32 - 00148 Roma
      Tel. +39-06.87594138 (recently changed)
      Mob. +39 3476939263
      Fax. +39-06.83074408
      www.eng.it

      On Tue, Feb 3, 2015 at 11:23 AM, M. Simonov, PhD (ISMB) <simonov@ismb.it>
      wrote:

      > Dear IDAS team,
      >
      >
      >
      > as you probably know, in the context of FIENSCE project we installed
      > experimental smart metering solution at trial site in Italy. Our software
      > reads data from smart meters and POSTs data every 5 minutes to IDAS, which
      > routes data to the FINESCE ORION. For you, is the new Use Case example
      > showing the usefulness of the IDAS.
      >
      >
      >
      > *By the way, our colleagues from Engineering detected that EVERY INSTANCE
      > of metering data being sent by our software arrives to ORION several times.
      > Instances directed to IDAS are replicated and posted to ORION several times
      > at high frequency. It is an issue.*
      >
      >
      >
      > Currently we send observations* every 5 minutes* from some different
      > meters labeled “0001”, “0002”, “0003”, “0004”, “0005”, “6666”. Basically,
      > we are sending 5 instances every 5 minutes plus 5 additional instances
      > every 15 minutes. During the whole day, we POST data to IDAS 5*96 + 15*96
      > times. That means we POST 1920 instances to IDAS every day.
      >
      >
      >
      > This is an example of the data broadcast from our software to IDAS:
      >
      >
      >
      > 2015-02-02T11:10:07.674 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0002, time=1422874800, APN=0,
      > APP=72, RPQ1=0,RPQ2=0, RPQ3= 0, RPQ4=24
      >
      > 2015-02-02T11:10:12.256 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0003, time=1422874800, APN=0,
      > APP=1500, RPQ1=1116, RPQ2=0, RPQ3= 0, RPQ4=0
      >
      > 2015-02-02T11:10:16.531 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0004, time=1422874800, APN=0,
      > APP=72, RPQ1=0,RPQ2=0, RPQ3= 0, RPQ4=60
      >
      > 2015-02-02T11:10:20.587 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0005, time=1422874800, APN=0,
      > APP=3948, RPQ1=1464, RPQ2=0, RPQ3= 0, RPQ4=0
      >
      > 2015-02-02T11:10:24.895 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0001, time=1422874800, APN=0,
      > APP=15348, RPQ1=924, RPQ2=0, RPQ3= 0, RPQ4=0
      >
      >
      >
      >
      >
      > 2015-02-02T11:20:05.173 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0003, time=1422875700, APN=0,
      > APP=1428, RPQ1=1068, RPQ2=0, RPQ3= 0, RPQ4=0
      >
      > 2015-02-02T11:20:09.814 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0004, time=1422875700, APN=0,
      > APP=60, RPQ1=0,RPQ2=0, RPQ3= 0, RPQ4=48
      >
      > 2015-02-02T11:20:13.904 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0005, time=1422875700, APN=0,
      > APP=3624, RPQ1=1344, RPQ2=0, RPQ3= 0, RPQ4=0
      >
      > 2015-02-02T11:20:18.200 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0001, time=1422875700, APN=0,
      > APP=14136, RPQ1=1008, RPQ2=0, RPQ3= 0, RPQ4=0
      >
      > 2015-02-02T11:20:22.529 Now PublishPower is sending METERING OBJECT data
      > to IDAS 130.206.80.47:8002 as: MeterId=0002, time=1422875700, APN=0,
      > APP=84, RPQ1=0,RPQ2=0, RPQ3= 0, RPQ4=24
      >
      > …and so on.
      >
      >
      >
      > Our colleagues receive much more data through IDAS-2-ORION. It seems each
      > input instance is being proliferated. It seems that each instance we send,
      > it is arriving a variable number of times to ORION. In the following
      > extract from the ORION LOG file, You can see 9 , 3, and 10 instances of
      > data for meters “6666”, “0003” and “0004” respectively.
      >
      >
      >
      > contextBroker log
      >
      > -----------------
      >
      >
      >
      > ******************************************* for meter 6666 *****
      >
      > 1st:
      >
      > Monday 02 Feb 16:25:12
      > 2015(695):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 2nd:
      >
      > Monday 02 Feb 16:25:13
      > 2015(015):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 3rd:
      >
      > Monday 02 Feb 16:25:13
      > 2015(231):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 4th:
      >
      > Monday 02 Feb 16:25:13
      > 2015(433):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 5th:
      >
      > Monday 02 Feb 16:25:13
      > 2015(882):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 6th:
      >
      > Monday 02 Feb 16:25:14
      > 2015(088):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 7th:
      >
      > Monday 02 Feb 16:25:14
      > 2015(280):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 8th:
      >
      > Monday 02 Feb 16:25:14
      > 2015(482):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 9th:
      >
      > Monday 02 Feb 16:25:14
      > 2015(684):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:6666-f06b4490-8f97-4b3c-b065-d3eaa7e29257' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      >
      >
      > ******************************************** for meter 0003 *****
      >
      > 1st:
      >
      > Monday 02 Feb 16:25:14
      > 2015(768):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0003-d73187ba-5287-4cd1-8ba3-a134d92b8e60' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 2nd:
      >
      > Monday 02 Feb 16:25:15
      > 2015(048):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0003-d73187ba-5287-4cd1-8ba3-a134d92b8e60' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 3rd:
      >
      > Monday 02 Feb 16:25:15
      > 2015(290):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0003-d73187ba-5287-4cd1-8ba3-a134d92b8e60' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > ******************************************** for meter 0004 *****
      >
      > 1st:
      >
      > Monday 02 Feb 16:25:17
      > 2015(699):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 2nd:
      >
      > Monday 02 Feb 16:25:17
      > 2015(985):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 3rd:
      >
      > Monday 02 Feb 16:25:18
      > 2015(216):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 4th:
      >
      > Monday 02 Feb 16:25:18
      > 2015(424):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 5th:
      >
      > Monday 02 Feb 16:25:18
      > 2015(639):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 6th:
      >
      > Monday 02 Feb 16:25:18
      > 2015(929):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 7th:
      >
      > Monday 02 Feb 16:25:19
      > 2015(160):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 8th:
      >
      > Monday 02 Feb 16:25:19
      > 2015(345):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 9th:
      >
      > Monday 02 Feb 16:25:19
      > 2015(584):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > 10th:
      >
      > Monday 02 Feb 16:25:19
      > 2015(761):contextBroker-/MongoCommonUpdate.cpp[1433] processContextElement:
      > Updating entity
      > 'FINESCEModel.meter:0004-36346e7f-1d76-4ccf-ae4a-30d3ab8c2cdc' (no Service
      > Path), action 'APPEND'
      >
      >
      >
      > ********************************************
      >
      >
      >
      > Our questions:
      >
      >
      >
      > 1) Could you explain this strange behavior (proliferation of
      > instances being posted to IDAS)?
      >
      >
      >
      > 2) Why the number of replications is not constant (not always X, but
      > sometimes 3 time, or 9 times, or 10 times)?
      >
      >
      >
      > 3) Could you help to change the IDAS behavior in order to take one
      > instance from us and to deliver one instance only to ORION?
      >
      >
      >
      > 4) How it is possible to trace the arrival of our instances to IDAS
      > and the dispatch of the instances to ORION?
      >
      >
      >
      > 5) Where it is possible to read more details about “how“
      > IDAS-2-ORION works internally (e.g. “how” it treats instances)?
      >
      >
      >
      > Thank you. M. Simonov
      >
      >
      >
      >
      >
      >
      >

      _______________________________________________
      Fiware-lab-help mailing list
      Fiware-lab-help@lists.fi-ware.org
      https://lists.fi-ware.org/listinfo/fiware-lab-help

        Activity

        Hide
        marcocipriani Marco Cipriani added a comment -

        @IDAS support team:
        The user email address is: pasquale.andriani@eng.it

        Show
        marcocipriani Marco Cipriani added a comment - @IDAS support team: The user email address is: pasquale.andriani@eng.it
        Hide
        ralli Carlos Ralli Ucendo added a comment -

        We had this call already if I am not worng, I will close the ticket then.

        Thanks for using IDAS and please check out the new Architecture and Catalogue entry we have provided last week.

        Regards,

        Show
        ralli Carlos Ralli Ucendo added a comment - We had this call already if I am not worng, I will close the ticket then. Thanks for using IDAS and please check out the new Architecture and Catalogue entry we have provided last week. Regards,

          People

          • Assignee:
            ralli Carlos Ralli Ucendo
            Reporter:
            pandriani Pasquale Andriani
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: