Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-5442

FIWARE.Request.Tech.Data.CKAN.Problema al intentar grabar datos en CKAN desde Cygnus - CKAN side

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-TECH-HELP
    • Labels:
      None

      Description

      Buenas tardes.

      Estoy intentando grabar datos en CKAN a través de Cygnus, pero no consigo
      que funcione correctamente cuando el tipo del atributo es JSON. Lo primero
      que hago es la llamada a ContextBroker:

      Accept: application/json
      X-AUTH-TOKEN: <mytoken>
      Fiware-Service: PapelClubDemo
      Fiware-ServicePath: /events/leonliterario
      {
      "contextElements": [
      {
      "type": "Events",
      "isPattern": "false",
      "id": "thisweek",
      "attributes": [
      {
      "name": "schedule",
      "type": "json",
      "value": [

      {"title": "Presentación Viva Mi Gente","date": "2015-11-30","location": "Salón de actos del ICAL","url": "http://www.papel.club"}

      ]
      }
      ]
      }
      ],
      "updateAction": "APPEND"
      }

      Con la correspondiente suscripción de cygnus para esta entidad, obtengo
      esta entrada en el log de cygnus:

      01 Dec 2015 19:05:13,701 INFO [891993589@qtp-1988714671-0]
      (com.telefonica.iot.cygnus.handlers.OrionRestHandler.getEvents:232) -
      Received data ({"subscriptionId" : "565dd3497b72b7c7092d5a29",
      "originator" : "localhost", "contextResponses" : [ { "contextElement"
      : { "type" : "Events","isPattern" : "false", "id" : "thisweek",
      "attributes" : [ { "name" : "schedule", "type" : "json", "value" : [

      { "title" : "Presentación Viva Mi Gente", "date" : "2015-11-30", "location" : "Salón de actos del ICAL", "url" : "http://www.papel.club" }

      ,

      { "title" : "Presentación Viva Mi Gente2","date" : "2015-11-30", "location" : "Salón de actos del ICAL", "url" : "http://www.papel.club" }

      ] } ] }, "statusCode" :

      { "code" : "200","reasonPhrase" : "OK" }

      } ]})

      01 Dec 2015 19:05:13,702 INFO
      [891993589@qtp-1988714671-0](com.telefonica.iot.cygnus.handlers.OrionRestHandler.getEvents:255)

      • Event put in the channel (id=2134043204, ttl=10) 01 Dec 2015
        19:05:16,842 INFO[SinkRunner-PollingRunner-DefaultSinkProcessor]
        (com.telefonica.iot.cygnus.sinks.OrionCKANSink.persistOne:207) -
        [ckan-sink] Persisting data atOrionCKANSink (orgName=papelclubdemo,
        pkgName=papelclubdemo_events_leonliterario,
        resName=thisweek004_events,data=1448989513702,2015-12-01T17:05:13.702Z,thisweek,Events,schedule,json,[ {"title":"Presentación Viva MiGente","date":"2015-11-30","location":"Salón de actos del ICAL","url":"http://www.papel.club"}

        ,

        {"title":"Presentación Viva MiGente2","date":"2015-11-30","location":"Salón de actos del ICAL","url":"http://www.papel.club"}

        ],[])
        01 Dec 2015 19:05:19,479 ERROR
        [SinkRunner-PollingRunner-DefaultSinkProcessor]
        (com.telefonica.iot.cygnus.sinks.OrionSink.process:224) - Runtime
        error (Don't know how to treat response code 503)
        01 Dec 2015 19:05:19,480 INFO
        [SinkRunner-PollingRunner-DefaultSinkProcessor]
        (com.telefonica.iot.cygnus.sinks.OrionSink.process:233) - Finishing
        transaction (1448984542-601-0000000018)

      Esta es la configuración de mi fichero agent en cygnus:

      1. Flume handler that will parse the notifications, must not be changed
        cygnusagent.sources.http-source.handler =
        com.telefonica.iot.cygnus.handlers.OrionRestHandler
      2. URL target
        cygnusagent.sources.http-source.handler.notification_target = /notify
      3. Default service (service semantic depends on the persistence sink)
        cygnusagent.sources.http-source.handler.default_service = def_serv
      4. Default service path (service path semantic depends on the persistence sink)
        cygnusagent.sources.http-source.handler.default_service_path = def_servpath
      5. Number of channel re-injection retries before a Flume event is
        definitely discarded (-1 means infinite retries)
        cygnusagent.sources.http-source.handler.events_ttl = 10
      6. Source interceptors, do not change
        cygnusagent.sources.http-source.interceptors = ts gi
      7. TimestampInterceptor, do not change
        cygnusagent.sources.http-source.interceptors.ts.type = timestamp
      8. GroupinInterceptor, do not change
        cygnusagent.sources.http-source.interceptors.gi.type =
        com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
      9. Grouping rules for the GroupingInterceptor, put the right absolute
        path to the file if necessary
      10. See the doc/design/interceptors document for more details
        cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file
        = /usr/cygnus/conf/grouping_rules.conf
      1. ============================================
      2. OrionCKANSink configuration
      3. channel name from where to read notification events
        cygnusagent.sinks.ckan-sink.channel = ckan-channel
      4. sink class, must not be changed
        cygnusagent.sinks.ckan-sink.type = com.telefonica.iot.cygnus.sinks.OrionCKANSink
      5. the CKAN API key to use
        #cygnusagent.sinks.ckan-sink.api_key = <mykey>
      6. the FQDN/IP address for the CKAN API endpoint
        cygnusagent.sinks.ckan-sink.ckan_host = demo.ckan.org
      7. the port for the CKAN API endpoint
        cygnusagent.sinks.ckan-sink.ckan_port = 80
      8. Orion URL used to compose the resource URL with the convenience
        operation URL to query it
        cygnusagent.sinks.ckan-sink.orion_url = http://127.0.0.1:1026
      9. how the attributes are stored, either per row either per column (row, column)
        cygnusagent.sinks.ckan-sink.attr_persistence = row
      10. enable SSL for secure Http transportation; 'true' or 'false'
        cygnusagent.sinks.ckan-sink.ssl = false

      Cuando cygnus reciba la información la envía a ckan y este la recibe,
      creándome la organización (si no existe), el dataset y el recurso, pero
      cuando accedo a ver el contenido del recurso este está vacío. Estoy
      utilizando demo.ckan.org.

      Muchas gracias,

      Alfonso García Fernández

      Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
      Please, send your messages using the new domain (Fiware-tech-help@lists.fiware.org) instead of the old one.
      _______________________________________________
      Fiware-tech-help mailing list
      Fiware-tech-help@lists.fiware.org
      https://lists.fiware.org/listinfo/fiware-tech-help
      [Created via e-mail received from: =?UTF-8?Q?Alfonso_Garc=C3=ADa_Fern=C3=A1ndez?= <alfongf@gmail.com>]

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          14h 59m 1 Manuel Escriche 02/Dec/15 10:42 AM
          In Progress In Progress Closed Closed
          36s 1 Manuel Escriche 02/Dec/15 10:43 AM
          Closed Closed In Progress In Progress
          28m 6s 1 Manuel Escriche 02/Dec/15 11:11 AM
          In Progress In Progress Answered Answered
          4d 22h 34m 1 OKFN CKAN team 07/Dec/15 9:45 AM
          Answered Answered Closed Closed
          1s 1 OKFN CKAN team 07/Dec/15 9:45 AM

            People

            • Assignee:
              ckan-fiware-okfn OKFN CKAN team
              Reporter:
              fw.ext.user FW External User
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: