Details
-
Type: Monitor
-
Status: Closed
-
Priority: Major
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: 2021
-
Component/s: FIWARE-TECH-HELP
-
Labels:
-
HD-Chapter:Data
-
HD-Enabler:Cosmos
Description
Created question in FIWARE Q/A platform on 07-05-2015 at 14:05
Please, ANSWER this question AT https://stackoverflow.com/questions/30101118/configuration-of-cygnus-conf
Question:
Configuration of cygnus.conf
Description:
I need to store data in cosmos with mysql format but I can't do it. I have checked that the data are stored on text file in cosmos, but when I enter in hive, there is not any table with my name.
The following is the config of cygnus. Is it correct? Where is my error to store in mysql tables?
cygnusagent.sources = http-source
cygnusagent.sinks = hdfs-sink mysql-sink
cygnusagent.channels = hdfs-channel mysql-channel
cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
cygnusagent.sources.http-source.port = 5050
cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
cygnusagent.sources.http-source.handler.notification_target = /notify
cygnusagent.sources.http-source.handler.default_service = def_serv
cygnusagent.sources.http-source.handler.default_service_path = def_servpath
cygnusagent.sources.http-source.handler.events_ttl = 10
cygnusagent.sources.http-source.interceptors = ts de
cygnusagent.sources.http-source.interceptors.ts.type = timestamp
cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder
cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf
cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
cygnusagent.sinks.hdfs-sink.cosmos_default_username = cristina.albaladejo
cygnusagent.sinks.hdfs-sink.cosmos_default_password = my_passw
cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
cygnusagent.sinks.hdfs-sink.attr_persistence = column
cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
cygnusagent.sinks.hdfs-sink.hive_port = 10000
cygnusagent.sinks.hdfs-sink.krb5_auth = false
cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username
cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx
cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf
cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf
- OrionMySQLSink configuration
cygnusagent.sinks.mysql-sink.channel = mysql-channel
cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink
cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46
cygnusagent.sinks.mysql-sink.mysql_port = 3306
cygnusagent.sinks.mysql-sink.mysql_username = root
cygnusagent.sinks.mysql-sink.mysql_password = my_passw
cygnusagent.sinks.mysql-sink.attr_persistence = column
- hdfs-channel configuration
cygnusagent.channels.hdfs-channel.type = memory
cygnusagent.channels.hdfs-channel.capacity = 1000
cygnusagent.channels.hdfs-channel.transactionCapacity = 100
- mysql-channel configuration
cygnusagent.channels.mysql-channel.type = memory
cygnusagent.channels.mysql-channel.capacity = 1000
cygnusagent.channels.mysql-channel.transactionCapacity = 100
Activity
Field | Original Value | New Value |
---|---|---|
Component/s | FIWARE-TECH-HELP [ 10278 ] |
Status | Open [ 1 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Answered [ 10104 ] |
Assignee | Backlog Manager [ backlogmanager ] |
HD-Enabler | Cosmos [ 10872 ] | |
Description |
Created question in FIWARE Q/A platform on 07-05-2015 at 14:05 {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/30101118/configuration-of-cygnus-conf +Question:+ Configuration of cygnus.conf +Description:+ I need to store data in cosmos with mysql format but I can't do it. I have checked that the data are stored on text file in cosmos, but when I enter in hive, there is not any table with my name. The following is the config of cygnus. Is it correct? Where is my error to store in mysql tables? cygnusagent.sources = http-source cygnusagent.sinks = hdfs-sink mysql-sink cygnusagent.channels = hdfs-channel mysql-channel cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource cygnusagent.sources.http-source.port = 5050 cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler cygnusagent.sources.http-source.handler.notification_target = /notify cygnusagent.sources.http-source.handler.default_service = def_serv cygnusagent.sources.http-source.handler.default_service_path = def_servpath cygnusagent.sources.http-source.handler.events_ttl = 10 cygnusagent.sources.http-source.interceptors = ts de cygnusagent.sources.http-source.interceptors.ts.type = timestamp cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf cygnusagent.sinks.hdfs-sink.channel = hdfs-channel cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46 cygnusagent.sinks.hdfs-sink.cosmos_port = 14000 cygnusagent.sinks.hdfs-sink.cosmos_default_username = cristina.albaladejo cygnusagent.sinks.hdfs-sink.cosmos_default_password = my_passw cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs cygnusagent.sinks.hdfs-sink.attr_persistence = column cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46 cygnusagent.sinks.hdfs-sink.hive_port = 10000 cygnusagent.sinks.hdfs-sink.krb5_auth = false cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf # OrionMySQLSink configuration cygnusagent.sinks.mysql-sink.channel = mysql-channel cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46 cygnusagent.sinks.mysql-sink.mysql_port = 3306 cygnusagent.sinks.mysql-sink.mysql_username = root cygnusagent.sinks.mysql-sink.mysql_password = my_passw cygnusagent.sinks.mysql-sink.attr_persistence = column # hdfs-channel configuration cygnusagent.channels.hdfs-channel.type = memory cygnusagent.channels.hdfs-channel.capacity = 1000 cygnusagent.channels.hdfs-channel.transactionCapacity = 100 # mysql-channel configuration cygnusagent.channels.mysql-channel.type = memory cygnusagent.channels.mysql-channel.capacity = 1000 cygnusagent.channels.mysql-channel.transactionCapacity = 100 |
Created question in FIWARE Q/A platform on 07-05-2015 at 14:05
{color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/30101118/configuration-of-cygnus-conf +Question:+ Configuration of cygnus.conf +Description:+ I need to store data in cosmos with mysql format but I can't do it. I have checked that the data are stored on text file in cosmos, but when I enter in hive, there is not any table with my name. The following is the config of cygnus. Is it correct? Where is my error to store in mysql tables? cygnusagent.sources = http-source cygnusagent.sinks = hdfs-sink mysql-sink cygnusagent.channels = hdfs-channel mysql-channel cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource cygnusagent.sources.http-source.port = 5050 cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler cygnusagent.sources.http-source.handler.notification_target = /notify cygnusagent.sources.http-source.handler.default_service = def_serv cygnusagent.sources.http-source.handler.default_service_path = def_servpath cygnusagent.sources.http-source.handler.events_ttl = 10 cygnusagent.sources.http-source.interceptors = ts de cygnusagent.sources.http-source.interceptors.ts.type = timestamp cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf cygnusagent.sinks.hdfs-sink.channel = hdfs-channel cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46 cygnusagent.sinks.hdfs-sink.cosmos_port = 14000 cygnusagent.sinks.hdfs-sink.cosmos_default_username = cristina.albaladejo cygnusagent.sinks.hdfs-sink.cosmos_default_password = my_passw cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs cygnusagent.sinks.hdfs-sink.attr_persistence = column cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46 cygnusagent.sinks.hdfs-sink.hive_port = 10000 cygnusagent.sinks.hdfs-sink.krb5_auth = false cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf # OrionMySQLSink configuration cygnusagent.sinks.mysql-sink.channel = mysql-channel cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46 cygnusagent.sinks.mysql-sink.mysql_port = 3306 cygnusagent.sinks.mysql-sink.mysql_username = root cygnusagent.sinks.mysql-sink.mysql_password = my_passw cygnusagent.sinks.mysql-sink.attr_persistence = column # hdfs-channel configuration cygnusagent.channels.hdfs-channel.type = memory cygnusagent.channels.hdfs-channel.capacity = 1000 cygnusagent.channels.hdfs-channel.transactionCapacity = 100 # mysql-channel configuration cygnusagent.channels.mysql-channel.type = memory cygnusagent.channels.mysql-channel.capacity = 1000 cygnusagent.channels.mysql-channel.transactionCapacity = 100 |
HD-Chapter | Data [ 10838 ] |
Resolution | Done [ 10000 ] | |
Status | Answered [ 10104 ] | Closed [ 6 ] |
Fix Version/s | 2021 [ 12600 ] |