Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-8762

[fiware-stackoverflow] Cygnus to Ckan - Runtime error (Don't know how to treat response code 409)

    Details

      Description

      Created question in FIWARE Q/A platform on 16-02-2016 at 14:02
      Please, ANSWER this question AT https://stackoverflow.com/questions/35433411/cygnus-to-ckan-runtime-error-dont-know-how-to-treat-response-code-409

      Question:
      Cygnus to Ckan - Runtime error (Don't know how to treat response code 409)

      Description:
      I am trying to use Cygnus with CKAN, but I get an error when I send the notification to the data is saved in CKAN.

      The Cygnus log:

      time=2016-02-17T10:01:11.601CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[209] : Starting transaction (1455699663-312-0000000000)
      time=2016-02-17T10:01:11.606CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[236] : Received data ({ "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8", "originator" : "localhost", "contextResponses" : [ { "contextElement" : { "attributes" : [

      { "name" : "temperature", "type" : "float", "value" : "26.5" }

      ], "type" : "Room", "isPattern" : "false", "id" : "Room1" }, "statusCode" :

      { "code" : "200", "reasonPhrase" : "OK" }

      } ]})
      time=2016-02-17T10:01:11.608CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[259] : Event put in the channel (id=1231327840, ttl=10)
      time=2016-02-17T10:01:34.311CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=processNewBatches | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[351] : Batch accumulation time reached, the batch will be processed as it is
      time=2016-02-17T10:01:34.319CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=persistAggregation | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionCKANSink[382] : [ckan-sink] Persisting data at OrionCKANSink (orgName=testjsb, pkgName=testjsb_testjsb, resName=room1_room, data=

      {"recvTimeTs": "1455699671","recvTime": "2016-02-17T09:01:11.609Z","fiwareServicePath": "testjsb","entityId": "Room1","entityType": "Room","attrName": "temperature","attrType": "float","attrValue": "26.5"}

      )
      time=2016-02-17T10:01:36.129CET | lvl=ERROR | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=processNewBatches | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[424] : Runtime error (Don't know how to treat response code 409)

      My cygnus agent config:

      cygnusagent.sources = http-source
      cygnusagent.sinks = ckan-sink
      cygnusagent.channels = ckan-channel

      #=============================================

      1. source configuration
      2. channel name where to write the notification events
        cygnusagent.sources.http-source.channels = ckan-channel
      3. source class, must not be changed
        cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
      4. listening port the Flume source will use for receiving incoming notifications
        cygnusagent.sources.http-source.port = 5050
      5. Flume handler that will parse the notifications, must not be changed
        cygnusagent.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.OrionRestHandler
      6. URL target
        cygnusagent.sources.http-source.handler.notification_target = /notify
      7. Default service (service semantic depends on the persistence sink)
        cygnusagent.sources.http-source.handler.default_service = testjsb
      8. Default service path (service path semantic depends on the persistence sink)
        cygnusagent.sources.http-source.handler.default_service_path = testjsb
      9. Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries)
        cygnusagent.sources.http-source.handler.events_ttl = 10
      10. Source interceptors, do not change
        cygnusagent.sources.http-source.interceptors = ts gi
      11. TimestampInterceptor, do not change
        cygnusagent.sources.http-source.interceptors.ts.type = timestamp
      12. GroupinInterceptor, do not change
        cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
      13. Grouping rules for the GroupingInterceptor, put the right absolute path to the file if necessary
      14. See the doc/design/interceptors document for more details
        cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf
      1. ============================================
      2. OrionCKANSink configuration
      3. channel name from where to read notification events
        cygnusagent.sinks.ckan-sink.channel = ckan-channel
      4. sink class, must not be changed
        cygnusagent.sinks.ckan-sink.type = com.telefonica.iot.cygnus.sinks.OrionCKANSink
        # true if the grouping feature is enabled for this sink, false otherwise
        cygnusagent.sinks.ckan-sink.enable_grouping = false
      5. the CKAN API key to use
        cygnusagent.sinks.ckan-sink.api_key = myapikey
      6. the FQDN/IP address for the CKAN API endpoint
        cygnusagent.sinks.ckan-sink.ckan_host = demo.ckan.org
      7. the port for the CKAN API endpoint
        cygnusagent.sinks.ckan-sink.ckan_port = 80
      8. Orion URL used to compose the resource URL with the convenience operation URL to query it
        cygnusagent.sinks.ckan-sink.orion_url = http://localhost:1026
      9. how the attributes are stored, either per row either per column (row, column)
        cygnusagent.sinks.ckan-sink.attr_persistence = row
      10. enable SSL for secure Http transportation; 'true' or 'false'
        cygnusagent.sinks.ckan-sink.ssl = false
      11. number of notifications to be included within a processing batch
        cygnusagent.sinks.ckan-sink.batch_size = 100
      12. timeout for batch accumulation
        cygnusagent.sinks.ckan-sink.batch_timeout = 30

      #=============================================

      1. ckan-channel configuration
      2. channel type (must not be changed)
        cygnusagent.channels.ckan-channel.type = memory
      3. capacity of the channel
        cygnusagent.channels.ckan-channel.capacity = 1000
      4. amount of bytes that can be sent per transaction
        cygnusagent.channels.ckan-channel.transactionCapacity = 100

      And the notification code:

      curl http://localhost:5050/notify v -s -S --header 'Content-Type: application/json' --header 'Accept: application/json' -d @ <<EOF
      {
      "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",
      "originator" : "localhost",
      "contextResponses" : [
      {
      "contextElement" : {
      "attributes" : [

      { "name" : "temperature", "type" : "float", "value" : "26.5" }

      ],
      "type" : "Room",
      "isPattern" : "false",
      "id" : "Room1"
      },
      "statusCode" :

      { "code" : "200", "reasonPhrase" : "OK" }

      }
      ]
      }
      EOF

      When cygnus persists data into demo.ckan.org, organization, dataset and resource are created correctyl but data are not loaded.

      http://demo.ckan.org/dataset/testjsb_testjsb/resource/0d9a8394-3d57-4450-b764-f3dbe2287127

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        2d 19h 5m 1 Fernando Lopez 25/May/17 10:10 AM
        In Progress In Progress Answered Answered
        2s 1 Fernando Lopez 25/May/17 10:10 AM
        Answered Answered Closed Closed
        2s 1 Fernando Lopez 25/May/17 10:11 AM
        fla Fernando Lopez made changes -
        Fix Version/s 2021 [ 12600 ]
        fla Fernando Lopez made changes -
        HD-Enabler Cosmos [ 10872 ]
        Description
        Created question in FIWARE Q/A platform on 16-02-2016 at 14:02
        {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/35433411/cygnus-to-ckan-runtime-error-dont-know-how-to-treat-response-code-409


        +Question:+
        Cygnus to Ckan - Runtime error (Don&#39;t know how to treat response code 409)

        +Description:+
        I am trying to use Cygnus with CKAN, but I get an error when I send the notification to the data is saved in CKAN.

        The Cygnus log:

        time=2016-02-17T10:01:11.601CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[209] : Starting transaction (1455699663-312-0000000000)
        time=2016-02-17T10:01:11.606CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[236] : Received data ({ "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8", "originator" : "localhost", "contextResponses" : [ { "contextElement" : { "attributes" : [ { "name" : "temperature", "type" : "float", "value" : "26.5" } ], "type" : "Room", "isPattern" : "false", "id" : "Room1" }, "statusCode" : { "code" : "200", "reasonPhrase" : "OK" } } ]})
        time=2016-02-17T10:01:11.608CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[259] : Event put in the channel (id=1231327840, ttl=10)
        time=2016-02-17T10:01:34.311CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=processNewBatches | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[351] : Batch accumulation time reached, the batch will be processed as it is
        time=2016-02-17T10:01:34.319CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=persistAggregation | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionCKANSink[382] : [ckan-sink] Persisting data at OrionCKANSink (orgName=testjsb, pkgName=testjsb_testjsb, resName=room1_room, data={"recvTimeTs": "1455699671","recvTime": "2016-02-17T09:01:11.609Z","fiwareServicePath": "testjsb","entityId": "Room1","entityType": "Room","attrName": "temperature","attrType": "float","attrValue": "26.5"})
        time=2016-02-17T10:01:36.129CET | lvl=ERROR | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=processNewBatches | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[424] : Runtime error (Don't know how to treat response code 409)


        My cygnus agent config:

        cygnusagent.sources = http-source
        cygnusagent.sinks = ckan-sink
        cygnusagent.channels = ckan-channel

        #=============================================
        # source configuration
        # channel name where to write the notification events
        cygnusagent.sources.http-source.channels = ckan-channel
        # source class, must not be changed
        cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
        # listening port the Flume source will use for receiving incoming notifications
        cygnusagent.sources.http-source.port = 5050
        # Flume handler that will parse the notifications, must not be changed
        cygnusagent.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.OrionRestHandler
        # URL target
        cygnusagent.sources.http-source.handler.notification_target = /notify
        # Default service (service semantic depends on the persistence sink)
        cygnusagent.sources.http-source.handler.default_service = testjsb
        # Default service path (service path semantic depends on the persistence sink)
        cygnusagent.sources.http-source.handler.default_service_path = testjsb
        # Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries)
        cygnusagent.sources.http-source.handler.events_ttl = 10
        # Source interceptors, do not change
        cygnusagent.sources.http-source.interceptors = ts gi
        # TimestampInterceptor, do not change
        cygnusagent.sources.http-source.interceptors.ts.type = timestamp
        # GroupinInterceptor, do not change
        cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
        # Grouping rules for the GroupingInterceptor, put the right absolute path to the file if necessary
        # See the doc/design/interceptors document for more details
        cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf

        # ============================================
        # OrionCKANSink configuration
        # channel name from where to read notification events
        cygnusagent.sinks.ckan-sink.channel = ckan-channel
        # sink class, must not be changed
        cygnusagent.sinks.ckan-sink.type = com.telefonica.iot.cygnus.sinks.OrionCKANSink
        # true if the grouping feature is enabled for this sink, false otherwise
        cygnusagent.sinks.ckan-sink.enable_grouping = false
        # the CKAN API key to use
        cygnusagent.sinks.ckan-sink.api_key = myapikey
        # the FQDN/IP address for the CKAN API endpoint
        cygnusagent.sinks.ckan-sink.ckan_host = demo.ckan.org
        # the port for the CKAN API endpoint
        cygnusagent.sinks.ckan-sink.ckan_port = 80
        # Orion URL used to compose the resource URL with the convenience operation URL to query it
        cygnusagent.sinks.ckan-sink.orion_url = http://localhost:1026
        # how the attributes are stored, either per row either per column (row, column)
        cygnusagent.sinks.ckan-sink.attr_persistence = row
        # enable SSL for secure Http transportation; 'true' or 'false'
        cygnusagent.sinks.ckan-sink.ssl = false
        # number of notifications to be included within a processing batch
        cygnusagent.sinks.ckan-sink.batch_size = 100
        # timeout for batch accumulation
        cygnusagent.sinks.ckan-sink.batch_timeout = 30

        #=============================================
        # ckan-channel configuration
        # channel type (must not be changed)
        cygnusagent.channels.ckan-channel.type = memory
        # capacity of the channel
        cygnusagent.channels.ckan-channel.capacity = 1000
        # amount of bytes that can be sent per transaction
        cygnusagent.channels.ckan-channel.transactionCapacity = 100


        And the notification code:

        curl http://localhost:5050/notify -v -s -S --header 'Content-Type: application/json' --header 'Accept: application/json' -d @- &lt;&lt;EOF
        {
            "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",
            "originator" : "localhost",
            "contextResponses" : [
                {
                    "contextElement" : {
                        "attributes" : [
                            {
                                "name" : "temperature",
                                "type" : "float",
                                "value" : "26.5"
                            }
                        ],
                        "type" : "Room",
                        "isPattern" : "false",
                        "id" : "Room1"
                    },
                    "statusCode" : {
                        "code" : "200",
                        "reasonPhrase" : "OK"
                    }
                }
            ]
        }
        EOF


        When cygnus persists data into demo.ckan.org, organization, dataset and resource are created correctyl but data are not loaded.

        http://demo.ckan.org/dataset/testjsb_testjsb/resource/0d9a8394-3d57-4450-b764-f3dbe2287127
        Created question in FIWARE Q/A platform on 16-02-2016 at 14:02
        {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/35433411/cygnus-to-ckan-runtime-error-dont-know-how-to-treat-response-code-409


        +Question:+
        Cygnus to Ckan - Runtime error (Don&#39;t know how to treat response code 409)

        +Description:+
        I am trying to use Cygnus with CKAN, but I get an error when I send the notification to the data is saved in CKAN.

        The Cygnus log:

        time=2016-02-17T10:01:11.601CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[209] : Starting transaction (1455699663-312-0000000000)
        time=2016-02-17T10:01:11.606CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[236] : Received data ({ "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8", "originator" : "localhost", "contextResponses" : [ { "contextElement" : { "attributes" : [ { "name" : "temperature", "type" : "float", "value" : "26.5" } ], "type" : "Room", "isPattern" : "false", "id" : "Room1" }, "statusCode" : { "code" : "200", "reasonPhrase" : "OK" } } ]})
        time=2016-02-17T10:01:11.608CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[259] : Event put in the channel (id=1231327840, ttl=10)
        time=2016-02-17T10:01:34.311CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=processNewBatches | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[351] : Batch accumulation time reached, the batch will be processed as it is
        time=2016-02-17T10:01:34.319CET | lvl=INFO | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=persistAggregation | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionCKANSink[382] : [ckan-sink] Persisting data at OrionCKANSink (orgName=testjsb, pkgName=testjsb_testjsb, resName=room1_room, data={"recvTimeTs": "1455699671","recvTime": "2016-02-17T09:01:11.609Z","fiwareServicePath": "testjsb","entityId": "Room1","entityType": "Room","attrName": "temperature","attrType": "float","attrValue": "26.5"})
        time=2016-02-17T10:01:36.129CET | lvl=ERROR | trans=1455699663-312-0000000000 | srv=testjsb | subsrv=testjsb | function=processNewBatches | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[424] : Runtime error (Don't know how to treat response code 409)


        My cygnus agent config:

        cygnusagent.sources = http-source
        cygnusagent.sinks = ckan-sink
        cygnusagent.channels = ckan-channel

        #=============================================
        # source configuration
        # channel name where to write the notification events
        cygnusagent.sources.http-source.channels = ckan-channel
        # source class, must not be changed
        cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
        # listening port the Flume source will use for receiving incoming notifications
        cygnusagent.sources.http-source.port = 5050
        # Flume handler that will parse the notifications, must not be changed
        cygnusagent.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.OrionRestHandler
        # URL target
        cygnusagent.sources.http-source.handler.notification_target = /notify
        # Default service (service semantic depends on the persistence sink)
        cygnusagent.sources.http-source.handler.default_service = testjsb
        # Default service path (service path semantic depends on the persistence sink)
        cygnusagent.sources.http-source.handler.default_service_path = testjsb
        # Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries)
        cygnusagent.sources.http-source.handler.events_ttl = 10
        # Source interceptors, do not change
        cygnusagent.sources.http-source.interceptors = ts gi
        # TimestampInterceptor, do not change
        cygnusagent.sources.http-source.interceptors.ts.type = timestamp
        # GroupinInterceptor, do not change
        cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
        # Grouping rules for the GroupingInterceptor, put the right absolute path to the file if necessary
        # See the doc/design/interceptors document for more details
        cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf

        # ============================================
        # OrionCKANSink configuration
        # channel name from where to read notification events
        cygnusagent.sinks.ckan-sink.channel = ckan-channel
        # sink class, must not be changed
        cygnusagent.sinks.ckan-sink.type = com.telefonica.iot.cygnus.sinks.OrionCKANSink
        # true if the grouping feature is enabled for this sink, false otherwise
        cygnusagent.sinks.ckan-sink.enable_grouping = false
        # the CKAN API key to use
        cygnusagent.sinks.ckan-sink.api_key = myapikey
        # the FQDN/IP address for the CKAN API endpoint
        cygnusagent.sinks.ckan-sink.ckan_host = demo.ckan.org
        # the port for the CKAN API endpoint
        cygnusagent.sinks.ckan-sink.ckan_port = 80
        # Orion URL used to compose the resource URL with the convenience operation URL to query it
        cygnusagent.sinks.ckan-sink.orion_url = http://localhost:1026
        # how the attributes are stored, either per row either per column (row, column)
        cygnusagent.sinks.ckan-sink.attr_persistence = row
        # enable SSL for secure Http transportation; 'true' or 'false'
        cygnusagent.sinks.ckan-sink.ssl = false
        # number of notifications to be included within a processing batch
        cygnusagent.sinks.ckan-sink.batch_size = 100
        # timeout for batch accumulation
        cygnusagent.sinks.ckan-sink.batch_timeout = 30

        #=============================================
        # ckan-channel configuration
        # channel type (must not be changed)
        cygnusagent.channels.ckan-channel.type = memory
        # capacity of the channel
        cygnusagent.channels.ckan-channel.capacity = 1000
        # amount of bytes that can be sent per transaction
        cygnusagent.channels.ckan-channel.transactionCapacity = 100


        And the notification code:

        curl http://localhost:5050/notify -v -s -S --header 'Content-Type: application/json' --header 'Accept: application/json' -d @- &lt;&lt;EOF
        {
            "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",
            "originator" : "localhost",
            "contextResponses" : [
                {
                    "contextElement" : {
                        "attributes" : [
                            {
                                "name" : "temperature",
                                "type" : "float",
                                "value" : "26.5"
                            }
                        ],
                        "type" : "Room",
                        "isPattern" : "false",
                        "id" : "Room1"
                    },
                    "statusCode" : {
                        "code" : "200",
                        "reasonPhrase" : "OK"
                    }
                }
            ]
        }
        EOF


        When cygnus persists data into demo.ckan.org, organization, dataset and resource are created correctyl but data are not loaded.

        http://demo.ckan.org/dataset/testjsb_testjsb/resource/0d9a8394-3d57-4450-b764-f3dbe2287127
        HD-Chapter Data [ 10838 ]
        fla Fernando Lopez made changes -
        Resolution Done [ 10000 ]
        Status Answered [ 10104 ] Closed [ 6 ]
        fla Fernando Lopez made changes -
        Status In Progress [ 3 ] Answered [ 10104 ]
        fla Fernando Lopez made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        fla Fernando Lopez made changes -
        Assignee Backlog Manager [ backlogmanager ]
        backlogmanager Backlog Manager made changes -
        Field Original Value New Value
        Component/s FIWARE-TECH-HELP [ 10278 ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-05-22 15:06|CREATED monitor | # answers= 0, accepted answer= False

        Show
        backlogmanager Backlog Manager added a comment - 2017-05-22 15:06|CREATED monitor | # answers= 0, accepted answer= False
        backlogmanager Backlog Manager created issue -

          People

          • Assignee:
            backlogmanager Backlog Manager
            Reporter:
            backlogmanager Backlog Manager
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: