Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-4353

FIWARE.Request.Tech.Data.BigData-Analysis.HiveTablesNotCreated

        Details

          Description

          Created question in FIWARE Q/A platform on 08-05-2015 at 13:05
          Please, ANSWER this question AT http://stackoverflow.com/questions/30122195/hive-tables-are-not-created

          Question:
          Hive tables are not created

          Description:
          I have checked that the data are stored on text file in cosmos as I show here:

          hadoop fs -cat /user/cristina.albaladejo/def_serv/def_servpath/sensorreading_1_sensorreading/sensorreading_1_sensorreading.txt

          {"recvTime":"2015-05-06T13:33:20.905","timestamp":"2015-04-15 11:10:00", "timestamp_md":[],"value":"-50.6", "value_md":[],"sensortypeid":"1", "sensortypeid_md":[],"nodeid":"1", "nodeid_md":[],"systemid":"1", "systemid_md":[]} {"recvTime":"2015-05-06T13:34:00.784","timestamp":"2015-04-15 11:10:00", "timestamp_md":[],"value":"-220.2", "value_md":[],"sensortypeid":"1", "sensortypeid_md":[],"nodeid":"1", "nodeid_md":[],"systemid":"1", "systemid_md":[]}

          But when I enter in hive, there is not any table with my name. Why not the hive tables are created?
          My file of configuration of Cygnus is the following:

          cygnusagent.sources = http-source
          cygnusagent.sinks = hdfs-sink mysql-sink
          cygnusagent.channels = hdfs-channel mysql-channel

          cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
          cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
          cygnusagent.sources.http-source.port = 5050
          cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
          cygnusagent.sources.http-source.handler.notification_target = /notify
          cygnusagent.sources.http-source.handler.default_service = def_serv
          cygnusagent.sources.http-source.handler.default_service_path = def_servpath
          cygnusagent.sources.http-source.handler.events_ttl = 10
          cygnusagent.sources.http-source.interceptors = ts de
          cygnusagent.sources.http-source.interceptors.ts.type = timestamp
          cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder
          cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf

          cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
          cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
          cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
          cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
          cygnusagent.sinks.hdfs-sink.cosmos_default_username = cristina.albaladejo
          cygnusagent.sinks.hdfs-sink.cosmos_default_password = my_passw
          cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
          cygnusagent.sinks.hdfs-sink.attr_persistence = column
          cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
          cygnusagent.sinks.hdfs-sink.hive_port = 10000
          cygnusagent.sinks.hdfs-sink.krb5_auth = false
          cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username
          cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx
          cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf
          cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf

          1. OrionMySQLSink configuration
            cygnusagent.sinks.mysql-sink.channel = mysql-channel
            cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink
            cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46
            cygnusagent.sinks.mysql-sink.mysql_port = 3306
            cygnusagent.sinks.mysql-sink.mysql_username = root
            cygnusagent.sinks.mysql-sink.mysql_password = my_passw
            cygnusagent.sinks.mysql-sink.attr_persistence = column
          1. hdfs-channel configuration
            cygnusagent.channels.hdfs-channel.type = memory
            cygnusagent.channels.hdfs-channel.capacity = 1000
            cygnusagent.channels.hdfs-channel.transactionCapacity = 100

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            backlogmanager Backlog Manager added a comment -

            2015-05-24 15:05|CREATED monitor | # answers= 0, accepted answer= False

            Show
            backlogmanager Backlog Manager added a comment - 2015-05-24 15:05|CREATED monitor | # answers= 0, accepted answer= False
            Hide
            Permalink
            frb Francisco Romero added a comment -

            Closing since this was answered by personal email.

            Show
            frb Francisco Romero added a comment - Closing since this was answered by personal email.
            Hide
            Permalink
            mev Manuel Escriche added a comment -

            Woudn't you provide that input in the QA platform?
            Other people might benefit from it. Thanks.

            Show
            mev Manuel Escriche added a comment - Woudn't you provide that input in the QA platform? Other people might benefit from it. Thanks.

              People

              • Assignee:
                frb Francisco Romero
                Reporter:
                backlogmanager Backlog Manager
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: