Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-17361

[Fiware-general-help] Quantum Leap Time Series Tutorial Issues

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-GENERAL-HELP
    • Labels:
      None

      Description

      Hi

      I am having issues with the Fiware time series data tutorial not behaving as the tutorial states.

      I am following this tutorial:
      https://fiware-tutorials.readthedocs.io/en/latest/time-series-data/index.html#time-series-data-queries-quantumleap-api

      using these docker containers (very latest cloned):
      https://github.com/FIWARE/tutorials.Time-Series-Data/tree/a1b9e289a89695491635d05cb1faff2455366799

      First, I followed the tutorial sequentially. Tutorial section: "QuantumLeap API - List the first N Sampled Values - 4 Request" returned a result but tutorial section: "QuantumLeap API - List N Sampled Values at an Offset - 5 request" failed returning:

      { "description": "No records were found for such query.", "error": "Not Found" }

      So, I stopped and started the docker services to clear things and started troubleshooting at what point things failed. As far as I can tell, the point it fails is that it doesn't create a table in CrateDB for Motion when creating a subscription whereas creating a subscription for Lamp does create a table in CrateDB. I do not know why, and I have not been able to find any documentation on how to troubleshoot further and figure out why. My steps to reproduce:

      Before creating a subscription for Motion, I check the crateDB schemas (tutorial section: CrateDB API - Checking Data persistence-> 12 Request):
      $ curl -iX POST 'http://localhost:4200/_sql' -H 'Content-Type: application/json' -d '

      {"stmt":"SHOW SCHEMAS"}

      '

      {"cols":["schema_name"],"rows":[["blob"],["doc"],["information_schema"],["pg_catalog"],["sys"]],"rowcount":5,"duration":123.18835}

      and the tables (tutorial section: CrateDB API - Checking Data persistence-> 13 Request):
      $ curl -X POST 'http://localhost:4200/_sql' -H 'Content-Type: application/json' -d '

      {"stmt":"SHOW TABLES"}

      '

      {"cols":["table_name"],"rows":[],"rowcount":0,"duration":41.81386}

      As expected - no tables.

      I subscribe to Motion (tutorial section: Aggregate Motion Sensor Count Events -> 1 Request):
      $ curl -iX POST \
      'http://localhost:1026/v2/subscriptions/' \
      -H 'Content-Type: application/json' \
      -H 'fiware-service: openiot' \
      -H 'fiware-servicepath: /' \
      -d '{
      "description": "Notify QuantumLeap of count changes of any Motion Sensor",
      "subject": {
      "entities": [

      { "idPattern": "Motion.*" }

      ],
      "condition":

      { "attrs": [ "count" ] }

      },
      "notification": {
      "http":

      { "url": "http://quantumleap:8668/v2/notify" }

      ,
      "attrs": [
      "count"
      ],
      "metadata": ["dateCreated", "dateModified"]
      },
      "throttling": 1
      }'
      HTTP/1.1 201 Created
      Connection: Keep-Alive
      Content-Length: 0
      Location: /v2/subscriptions/6023b32d944ff95b637d9cbd
      Fiware-Correlator: 738ed596-6b89-11eb-94e7-0242c0a86007
      Date: Wed, 10 Feb 2021 10:19:25 GMT

      check the subscription (tutorial section: Checking Subscriptions for QuantumLeap - 3 Request):
      $ curl -X GET 'http://localhost:1026/v2/subscriptions/' -H 'fiware-service: openiot' -H 'fiware-servicepath: /'
      [{"id":"6023b32d944ff95b637d9cbd","description":"Notify QuantumLeap of count changes of any Motion Sensor","status":"active","subject":{"entities":[

      {"idPattern":"Motion.*"}

      ],"condition":{"attrs":["count"]}},"notification":{"timesSent":1,"lastNotification":"2021-02-10T10:19:25.000Z","attrs":["count"],"onlyChangedAttrs":false,"attrsFormat":"normalized","http":

      {"url":"http://quantumleap:8668/v2/notify"}

      ,"metadata":["dateCreated","dateModified"],"lastSuccess":"2021-02-10T10:19:25.000Z","lastSuccessCode":200},"throttling":1}]

      check for tables again:
      $ curl -X POST 'http://localhost:4200/_sql' -H 'Content-Type: application/json' -d '

      {"stmt":"SHOW TABLES"}

      '

      {"cols":["table_name"],"rows":[],"rowcount":0,"duration":6.013363}

      no new tables have been created.

      So, I subscribe to Lamp Luminosity (tutorial section: Aggregate Motion Sensor Count Events -> 2 Request):
      $ curl -iX POST \
      'http://localhost:1026/v2/subscriptions/' \
      -H 'Content-Type: application/json' \
      -H 'fiware-service: openiot' \
      -H 'fiware-servicepath: /' \
      -d '{
      "description": "Notify QuantumLeap on luminosity changes on any Lamp",
      "subject": {
      "entities": [

      { "idPattern": "Lamp.*" }

      ],
      "condition":

      { "attrs": [ "luminosity", "location" ] }

      },
      "notification": {
      "http":

      { "url": "http://quantumleap:8668/v2/notify" }

      ,
      "attrs": [
      "luminosity", "location"
      ],
      "metadata": ["dateCreated", "dateModified"]
      },
      "throttling": 1
      }'
      HTTP/1.1 201 Created
      Connection: Keep-Alive
      Content-Length: 0
      Location: /v2/subscriptions/6023b16df285be8854aeb2aa
      Fiware-Correlator: 682f63ba-6b88-11eb-8e86-0242c0a85005
      Date: Wed, 10 Feb 2021 10:11:57 GMT

      check the subscriptions:
      $ curl -X GET 'http://localhost:1026/v2/subscriptions/' -H 'fiware-service: openiot' -H 'fiware-servicepath: /'
      [{"id":"6023b32d944ff95b637d9cbd","description":"Notify QuantumLeap of count changes of any Motion Sensor","status":"active","subject":{"entities":[

      {"idPattern":"Motion.*"}

      ],"condition":{"attrs":["count"]}},"notification":{"timesSent":1,"lastNotification":"2021-02-10T10:19:25.000Z","attrs":["count"],"onlyChangedAttrs":false,"attrsFormat":"normalized","http":

      {"url":"http://quantumleap:8668/v2/notify"}

      ,"metadata":["dateCreated","dateModified"],"lastSuccess":"2021-02-10T10:19:25.000Z","lastSuccessCode":200},"throttling":1},
      {"id":"6023b499944ff95b637d9cbe","description":"Notify QuantumLeap on luminosity changes on any Lamp","status":"active","subject":{"entities":[

      {"idPattern":"Lamp.*"}

      ],"condition":{"attrs":["luminosity","location"]}},"notification":{"timesSent":1,"lastNotification":"2021-02-10T10:25:29.000Z","attrs":["luminosity","location"],"onlyChangedAttrs":false,"attrsFormat":"normalized","http":

      {"url":"http://quantumleap:8668/v2/notify"}

      ,"metadata":["dateCreated","dateModified"],"lastSuccess":"2021-02-10T10:25:31.000Z","lastSuccessCode":200},"throttling":1}]

      and check the tables again:
      $ curl -X POST 'http://localhost:4200/_sql' -H 'Content-Type: application/json' -d '

      {"stmt":"SHOW TABLES"}

      '

      {"cols":["table_name"],"rows":[["etlamp"],["md_ets_metadata"]],"rowcount":2,"duration":6.791097}

      and can clearly see a new table "etlamp" has been created.

      So why has the Motion table not been created?
      I am new to CrateDB and intend to use TimescaleDB but it is important to follow the tutorial to get a feel for how it works as TimescaleDB is not well documented for FIWARE use yet. However, I found a crate-db log (gc.log) and looked for "etlamp", "lamp", "motion" but find no reference. I haven't located logs in the quantum leap container - so I'm at a loss as to how to troubleshoot further. Please can you advise?

      Thank you
      Kind regards
      Taz

      Taz Lodder
      IT Lösningsspecialist och Förvaltningsledare Stöd till Forskning & Nyttiggörande | Chalmers IT-Avdelningen
      Technical Solution Specialist and Project Manager to Research & Utilisation | Chalmers IT Office
      taz@chalmers.se<taz@chalmers.se>
      Internal: 6636
      External: +46(0)31 772 6636

      Chalmers Tekniska Högskola | Chalmers University of Technology
      SE-412 96 Göteborg, Sweden
      http://www.chalmers.se

      __________________________________________________________________________________________

      You can get more information about our cookies and privacy policies on the following links:

      fiware-general-help mailing list
      fiware-general-help@lists.fiware.org

      To unsubscribe from fiware-general-help mailing list, go to the information page of the list at:
      https://lists.fiware.org/listinfo/fiware-general-help

      [Created via e-mail received from: Taz Lodder <taz@chalmers.se>]

        Activity

        Hide
        jason.fox Jason Fox added a comment - - edited

        Responded by eMail:

        I think I know the answer to your tutorial question.  It appears that you missed the instruction to  switch on the lamp and unlock the door.
        When an NGSI-v2 subscription is created it fires once. Therefore after you have created the subscriptions the system will be in the following state:

        check the subscription (tutorial section: Checking Subscriptions for QuantumLeap - 3 Request):
        $ curl -X GET 'http://localhost:1026/v2/subscriptions/' -H 'fiware-service: openiot' -H 'fiware-servicepath: /'

        {{
        [{"id":"6023b32d944ff95b637d9cbd","description":"Notify QuantumLeap of count changes of any Motion Sensor","status":"active","subject":{"entities":[

        {"idPattern":"Motion.*"}

        ],"condition":{"attrs":["count"]}},"notification":{"timesSent":1,"lastNotification":"2021-02-10T10:19:25.000Z","attrs":["count"],"onlyChangedAttrs":false,"attrsFormat":"normalized","http":

        {"url":"http://quantumleap:8668/v2/notify"}

        ,"metadata":["dateCreated","dateModified"],"lastSuccess":"2021-02-10T10:19:25.000Z","lastSuccessCode":200},"throttling":1}]
        }}

        timesSent is 1. For the motion sensors the count has not yet been properly defined. It is only once the context is changing that the queries to timescale will fire correctly. Obviously using an offset when there is no data present will result in. a not found response.

        run the following:

        docker logs fiware-orion

        you should see lines as follows:

        {{time=2021-02-10T13:51:16.821Z | lvl=INFO | corr=0be1d2d6-6ba7-11eb-b54d-0242ac120107 | trans=1612964843-360-00000000303 | from=172.18.1.8 | srv=openiot | subsrv=/ | comp=Orion | op=logTracing.cpp[130]:logInfoRequestWithPayload | msg=Request received: POST /v2/entities/Motion:001/attrs?type=Motion, request payload (997 bytes): {"count":{"type":"Integer","value":"1","metadata":{"TimeInstant":

        {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}

        }},"refStore":{"type":"Relationship","value":"urn:ngsi-ld:Store:001","metadata":{"TimeInstant":

        {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}

        }},"category":{"type":"Text","value":["sensor"],"metadata":{"TimeInstant":

        {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}

        }},"controlledProperty":{"type":"Text","value":"motion","metadata":{"TimeInstant":

        {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}

        }},"function":{"type":"Text","value":["sensing"],"metadata":{"TimeInstant":

        {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}

        }},"supportedProtocol":{"type":"Text","value":["ul20"],"metadata":{"TimeInstant":

        {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}

        }},"supportedUnits":}}{"type":"Text","value":"C62","metadata":{"TimeInstant":

        {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}

        }},"TimeInstant":{"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}}, response code: 204time=2021-02-10T13:51:16.864Z | lvl=INFO | corr=0bbcd3c8-6ba7-11eb-bfa5-0242ac120107; cbnotif=1 | trans=1612964843-360-00000000300 | from=172.18.1.8 | srv=openiot | subsrv=<none> | comp=Orion | op=logTracing.cpp[63]:logInfoNotification | msg=Notif delivered (subId: 6023e45597283d3c5d2d4fc5): POST quantumleap:8668/v2/notify, response code: 200}}

        the /v2/entities/Motion:001/attrs shows a motion sensor is being updatedthe POST quantumleap:8668/v2/notify, shows a subscription event has been raised.

             

        Show
        jason.fox Jason Fox added a comment - - edited Responded by eMail: I think I know the answer to your tutorial question.  It appears that you missed the instruction to  switch on the lamp and unlock the door. When an NGSI-v2 subscription is created it fires once. Therefore after you have created the subscriptions the system will be in the following state: check the subscription (tutorial section: Checking Subscriptions for QuantumLeap - 3 Request): $ curl -X GET 'http://localhost:1026/v2/subscriptions/' -H 'fiware-service: openiot' -H 'fiware-servicepath: /' {{ [{"id":"6023b32d944ff95b637d9cbd","description":"Notify QuantumLeap of count changes of any Motion Sensor","status":"active","subject":{"entities":[ {"idPattern":"Motion.*"} ],"condition":{"attrs": ["count"] }},"notification":{"timesSent":1,"lastNotification":"2021-02-10T10:19:25.000Z","attrs": ["count"] ,"onlyChangedAttrs":false,"attrsFormat":"normalized","http": {"url":"http://quantumleap:8668/v2/notify"} ,"metadata": ["dateCreated","dateModified"] ,"lastSuccess":"2021-02-10T10:19:25.000Z","lastSuccessCode":200},"throttling":1}] }} timesSent is 1. For the motion sensors the count has not yet been properly defined. It is only once the context is changing that the queries to timescale will fire correctly. Obviously using an offset when there is no data present will result in. a not found response. run the following: docker logs fiware-orion you should see lines as follows: {{time=2021-02-10T13:51:16.821Z | lvl=INFO | corr=0be1d2d6-6ba7-11eb-b54d-0242ac120107 | trans=1612964843-360-00000000303 | from=172.18.1.8 | srv=openiot | subsrv=/ | comp=Orion | op=logTracing.cpp [130] :logInfoRequestWithPayload | msg=Request received: POST /v2/entities/Motion:001/attrs?type=Motion, request payload (997 bytes): {"count":{"type":"Integer","value":"1","metadata":{"TimeInstant": {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"} }},"refStore":{"type":"Relationship","value":"urn:ngsi-ld:Store:001","metadata":{"TimeInstant": {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"} }},"category":{"type":"Text","value": ["sensor"] ,"metadata":{"TimeInstant": {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"} }},"controlledProperty":{"type":"Text","value":"motion","metadata":{"TimeInstant": {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"} }},"function":{"type":"Text","value": ["sensing"] ,"metadata":{"TimeInstant": {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"} }},"supportedProtocol":{"type":"Text","value": ["ul20"] ,"metadata":{"TimeInstant": {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"} }},"supportedUnits":}}{"type":"Text","value":"C62","metadata":{"TimeInstant": {"type":"DateTime","value":"2021-02-10T13:51:16.741Z"} }},"TimeInstant":{"type":"DateTime","value":"2021-02-10T13:51:16.741Z"}}, response code: 204time=2021-02-10T13:51:16.864Z | lvl=INFO | corr=0bbcd3c8-6ba7-11eb-bfa5-0242ac120107; cbnotif=1 | trans=1612964843-360-00000000300 | from=172.18.1.8 | srv=openiot | subsrv=<none> | comp=Orion | op=logTracing.cpp [63] :logInfoNotification | msg=Notif delivered (subId: 6023e45597283d3c5d2d4fc5): POST quantumleap:8668/v2/notify, response code: 200}} the /v2/entities/Motion:001/attrs shows a motion sensor is being updatedthe POST quantumleap:8668/v2/notify, shows a subscription event has been raised.      
        Hide
        jason.fox Jason Fox added a comment -

        Discussion over the internet - seems like the dummy devices were not connected properly. Fixed by downloading the latest version

        Show
        jason.fox Jason Fox added a comment - Discussion over the internet - seems like the dummy devices were not connected properly. Fixed by downloading the latest version

          People

          • Assignee:
            jason.fox Jason Fox
            Reporter:
            fw.ext.user FW External User
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: