Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-4352

FIWARE.Request.Tech.Data.BigData-Analysis.CygnusError3

    Details

      Description

      Created question in FIWARE Q/A platform on 07-05-2015 at 14:05
      Please, ANSWER this question AT http://stackoverflow.com/questions/30101118/configuration-of-cygnus-conf

      Question:
      Configuration of cygnus.conf

      Description:
      I need to store data in cosmos with mysql format but I can't do it. I have checked that the data are stored on text file in cosmos, but when I enter in hive, there is not any table with my name.

      The following is the config of cygnus. Is it correct? Where is my error to store in mysql tables?

      cygnusagent.sources = http-source
      cygnusagent.sinks = hdfs-sink mysql-sink
      cygnusagent.channels = hdfs-channel mysql-channel

      cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
      cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
      cygnusagent.sources.http-source.port = 5050
      cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
      cygnusagent.sources.http-source.handler.notification_target = /notify
      cygnusagent.sources.http-source.handler.default_service = def_serv
      cygnusagent.sources.http-source.handler.default_service_path = def_servpath
      cygnusagent.sources.http-source.handler.events_ttl = 10
      cygnusagent.sources.http-source.interceptors = ts de
      cygnusagent.sources.http-source.interceptors.ts.type = timestamp
      cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder
      cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf

      cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
      cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
      cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
      cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
      cygnusagent.sinks.hdfs-sink.cosmos_default_username = cristina.albaladejo
      cygnusagent.sinks.hdfs-sink.cosmos_default_password = my_passw
      cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
      cygnusagent.sinks.hdfs-sink.attr_persistence = column
      cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
      cygnusagent.sinks.hdfs-sink.hive_port = 10000
      cygnusagent.sinks.hdfs-sink.krb5_auth = false
      cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username
      cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx
      cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf
      cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf

      1. OrionMySQLSink configuration
        cygnusagent.sinks.mysql-sink.channel = mysql-channel
        cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink
        cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46
        cygnusagent.sinks.mysql-sink.mysql_port = 3306
        cygnusagent.sinks.mysql-sink.mysql_username = root
        cygnusagent.sinks.mysql-sink.mysql_password = my_passw
        cygnusagent.sinks.mysql-sink.attr_persistence = column
      1. hdfs-channel configuration
        cygnusagent.channels.hdfs-channel.type = memory
        cygnusagent.channels.hdfs-channel.capacity = 1000
        cygnusagent.channels.hdfs-channel.transactionCapacity = 100
      1. mysql-channel configuration
        cygnusagent.channels.mysql-channel.type = memory
        cygnusagent.channels.mysql-channel.capacity = 1000
        cygnusagent.channels.mysql-channel.transactionCapacity = 100

        Activity

        Hide
        backlogmanager Backlog Manager added a comment -

        2015-05-24 15:05|CREATED monitor | # answers= 0, accepted answer= False

        Show
        backlogmanager Backlog Manager added a comment - 2015-05-24 15:05|CREATED monitor | # answers= 0, accepted answer= False
        Hide
        frb Francisco Romero added a comment -

        Closing since the original question comprised several subquestions that were fixed with the user either by personal email either through other SOF questions.

        Show
        frb Francisco Romero added a comment - Closing since the original question comprised several subquestions that were fixed with the user either by personal email either through other SOF questions.

          People

          • Assignee:
            frb Francisco Romero
            Reporter:
            backlogmanager Backlog Manager
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: