Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-9200

[fiware-stackoverflow] Configuration of cygnus.conf

    Details

      Description

      Created question in FIWARE Q/A platform on 07-05-2015 at 14:05
      Please, ANSWER this question AT https://stackoverflow.com/questions/30101118/configuration-of-cygnus-conf

      Question:
      Configuration of cygnus.conf

      Description:
      I need to store data in cosmos with mysql format but I can't do it. I have checked that the data are stored on text file in cosmos, but when I enter in hive, there is not any table with my name.

      The following is the config of cygnus. Is it correct? Where is my error to store in mysql tables?

      cygnusagent.sources = http-source
      cygnusagent.sinks = hdfs-sink mysql-sink
      cygnusagent.channels = hdfs-channel mysql-channel

      cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
      cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
      cygnusagent.sources.http-source.port = 5050
      cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
      cygnusagent.sources.http-source.handler.notification_target = /notify
      cygnusagent.sources.http-source.handler.default_service = def_serv
      cygnusagent.sources.http-source.handler.default_service_path = def_servpath
      cygnusagent.sources.http-source.handler.events_ttl = 10
      cygnusagent.sources.http-source.interceptors = ts de
      cygnusagent.sources.http-source.interceptors.ts.type = timestamp
      cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder
      cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf

      cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
      cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
      cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
      cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
      cygnusagent.sinks.hdfs-sink.cosmos_default_username = cristina.albaladejo
      cygnusagent.sinks.hdfs-sink.cosmos_default_password = my_passw
      cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
      cygnusagent.sinks.hdfs-sink.attr_persistence = column
      cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
      cygnusagent.sinks.hdfs-sink.hive_port = 10000
      cygnusagent.sinks.hdfs-sink.krb5_auth = false
      cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username
      cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx
      cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf
      cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf

      1. OrionMySQLSink configuration
        cygnusagent.sinks.mysql-sink.channel = mysql-channel
        cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink
        cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46
        cygnusagent.sinks.mysql-sink.mysql_port = 3306
        cygnusagent.sinks.mysql-sink.mysql_username = root
        cygnusagent.sinks.mysql-sink.mysql_password = my_passw
        cygnusagent.sinks.mysql-sink.attr_persistence = column
      1. hdfs-channel configuration
        cygnusagent.channels.hdfs-channel.type = memory
        cygnusagent.channels.hdfs-channel.capacity = 1000
        cygnusagent.channels.hdfs-channel.transactionCapacity = 100
      1. mysql-channel configuration
        cygnusagent.channels.mysql-channel.type = memory
        cygnusagent.channels.mysql-channel.capacity = 1000
        cygnusagent.channels.mysql-channel.transactionCapacity = 100

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        2h 59m 1 Backlog Manager 24/May/17 9:04 PM
        In Progress In Progress Answered Answered
        2h 59m 1 Backlog Manager 25/May/17 12:04 AM
        Answered Answered Closed Closed
        1d 10h 13m 1 Fernando Lopez 26/May/17 10:17 AM
        fla Fernando Lopez made changes -
        Fix Version/s 2021 [ 12600 ]
        fla Fernando Lopez made changes -
        Resolution Done [ 10000 ]
        Status Answered [ 10104 ] Closed [ 6 ]
        fla Fernando Lopez made changes -
        HD-Enabler Cosmos [ 10872 ]
        Description
        Created question in FIWARE Q/A platform on 07-05-2015 at 14:05
        {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/30101118/configuration-of-cygnus-conf


        +Question:+
        Configuration of cygnus.conf

        +Description:+
        I need to store data in cosmos with mysql format but I can't do it. I have checked that the data are stored on text file in cosmos, but when I enter in hive, there is not any table with my name.

        The following is the config of cygnus. Is it correct? Where is my error to store in mysql tables?

        cygnusagent.sources = http-source
        cygnusagent.sinks = hdfs-sink mysql-sink
        cygnusagent.channels = hdfs-channel mysql-channel

        cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
        cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
        cygnusagent.sources.http-source.port = 5050
        cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
        cygnusagent.sources.http-source.handler.notification_target = /notify
        cygnusagent.sources.http-source.handler.default_service = def_serv
        cygnusagent.sources.http-source.handler.default_service_path = def_servpath
        cygnusagent.sources.http-source.handler.events_ttl = 10
        cygnusagent.sources.http-source.interceptors = ts de
        cygnusagent.sources.http-source.interceptors.ts.type = timestamp
        cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder
        cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf

        cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
        cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
        cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
        cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
        cygnusagent.sinks.hdfs-sink.cosmos_default_username = cristina.albaladejo
        cygnusagent.sinks.hdfs-sink.cosmos_default_password = my_passw
        cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
        cygnusagent.sinks.hdfs-sink.attr_persistence = column
        cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
        cygnusagent.sinks.hdfs-sink.hive_port = 10000
        cygnusagent.sinks.hdfs-sink.krb5_auth = false
        cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username
        cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx
        cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf
        cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf

        # OrionMySQLSink configuration
        cygnusagent.sinks.mysql-sink.channel = mysql-channel
        cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink
        cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46
        cygnusagent.sinks.mysql-sink.mysql_port = 3306
        cygnusagent.sinks.mysql-sink.mysql_username = root
        cygnusagent.sinks.mysql-sink.mysql_password = my_passw
        cygnusagent.sinks.mysql-sink.attr_persistence = column

        # hdfs-channel configuration
        cygnusagent.channels.hdfs-channel.type = memory
        cygnusagent.channels.hdfs-channel.capacity = 1000
        cygnusagent.channels.hdfs-channel.transactionCapacity = 100

        # mysql-channel configuration
        cygnusagent.channels.mysql-channel.type = memory
        cygnusagent.channels.mysql-channel.capacity = 1000
        cygnusagent.channels.mysql-channel.transactionCapacity = 100

        Created question in FIWARE Q/A platform on 07-05-2015 at 14:05
        {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/30101118/configuration-of-cygnus-conf


        +Question:+
        Configuration of cygnus.conf

        +Description:+
        I need to store data in cosmos with mysql format but I can't do it. I have checked that the data are stored on text file in cosmos, but when I enter in hive, there is not any table with my name.

        The following is the config of cygnus. Is it correct? Where is my error to store in mysql tables?

        cygnusagent.sources = http-source
        cygnusagent.sinks = hdfs-sink mysql-sink
        cygnusagent.channels = hdfs-channel mysql-channel

        cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
        cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
        cygnusagent.sources.http-source.port = 5050
        cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
        cygnusagent.sources.http-source.handler.notification_target = /notify
        cygnusagent.sources.http-source.handler.default_service = def_serv
        cygnusagent.sources.http-source.handler.default_service_path = def_servpath
        cygnusagent.sources.http-source.handler.events_ttl = 10
        cygnusagent.sources.http-source.interceptors = ts de
        cygnusagent.sources.http-source.interceptors.ts.type = timestamp
        cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder
        cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf

        cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
        cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
        cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
        cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
        cygnusagent.sinks.hdfs-sink.cosmos_default_username = cristina.albaladejo
        cygnusagent.sinks.hdfs-sink.cosmos_default_password = my_passw
        cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
        cygnusagent.sinks.hdfs-sink.attr_persistence = column
        cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
        cygnusagent.sinks.hdfs-sink.hive_port = 10000
        cygnusagent.sinks.hdfs-sink.krb5_auth = false
        cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username
        cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx
        cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf
        cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf

        # OrionMySQLSink configuration
        cygnusagent.sinks.mysql-sink.channel = mysql-channel
        cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink
        cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46
        cygnusagent.sinks.mysql-sink.mysql_port = 3306
        cygnusagent.sinks.mysql-sink.mysql_username = root
        cygnusagent.sinks.mysql-sink.mysql_password = my_passw
        cygnusagent.sinks.mysql-sink.attr_persistence = column

        # hdfs-channel configuration
        cygnusagent.channels.hdfs-channel.type = memory
        cygnusagent.channels.hdfs-channel.capacity = 1000
        cygnusagent.channels.hdfs-channel.transactionCapacity = 100

        # mysql-channel configuration
        cygnusagent.channels.mysql-channel.type = memory
        cygnusagent.channels.mysql-channel.capacity = 1000
        cygnusagent.channels.mysql-channel.transactionCapacity = 100

        HD-Chapter Data [ 10838 ]
        fla Fernando Lopez made changes -
        Assignee Backlog Manager [ backlogmanager ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-05-25 00:05|UPDATED status: transition Answered| # answers= 1, accepted answer= False

        Show
        backlogmanager Backlog Manager added a comment - 2017-05-25 00:05|UPDATED status: transition Answered| # answers= 1, accepted answer= False
        backlogmanager Backlog Manager made changes -
        Status In Progress [ 3 ] Answered [ 10104 ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-05-24 21:05|UPDATED status: transition Answer| # answers= 1, accepted answer= False

        Show
        backlogmanager Backlog Manager added a comment - 2017-05-24 21:05|UPDATED status: transition Answer| # answers= 1, accepted answer= False
        backlogmanager Backlog Manager made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        backlogmanager Backlog Manager made changes -
        Field Original Value New Value
        Component/s FIWARE-TECH-HELP [ 10278 ]
        Hide
        backlogmanager Backlog Manager added a comment -

        2017-05-24 18:05|CREATED monitor | # answers= 1, accepted answer= False

        Show
        backlogmanager Backlog Manager added a comment - 2017-05-24 18:05|CREATED monitor | # answers= 1, accepted answer= False
        backlogmanager Backlog Manager created issue -

          People

          • Assignee:
            backlogmanager Backlog Manager
            Reporter:
            backlogmanager Backlog Manager
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: