Subject: [FIWARE-JIRA] (HELP-12848) [Fiware-lab-help] Hive - JSonSerde library - Cygnus
From: "FW External User (JIRA)" <[email-masked]>
To: [email-masked]
Date: Mon, 30 Oct 2017 18:22:02 +0000 (GMT)

[ https://jira.fiware.org/browse/HELP-12848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

FW External User updated HELP-12848:
------------------------------------
    Attachment: [FIWARE-JIRA] (HELP-12848) [Fiware-lab-help] Hive - JSonSerde library - Cygnus

Comment by [email-masked] : 

Spam detection software, running on the system "fiware-forge-01", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't spam) or label
similar future email.  If you have any questions, see
the administrator of that system for details.

Content preview:  [ https://jira.fiware.org/browse/HELP-12848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
   ] FW External User updated HELP-12848: Attachment: [FIWARE-JIRA] (HELP-12848)
   [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-12870) [SPAM] Website Job
   Application [...] 

Content analysis details:   (7.7 points, 4.5 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 2.5 URIBL_DBL_SPAM         Contains a spam URL listed in the DBL blocklist
                            [URIs: kzarea.com]
 1.7 URIBL_BLACK            Contains an URL listed in the URIBL blacklist
                            [URIs: kzarea.com]
 0.0 FSL_HELO_NON_FQDN_1    FSL_HELO_NON_FQDN_1
 3.8 HELO_LOCALHOST         HELO_LOCALHOST
 1.6 DATE_IN_PAST_03_06     Date: is 3 to 6 hours before Received: date
-1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
                            [score: 0.0000]

> [Fiware-lab-help] Hive - JSonSerde library - Cygnus
> ---------------------------------------------------
>
>                 Key: HELP-12848
>                 URL: https://jira.fiware.org/browse/HELP-12848
>             Project: Help-Desk
>          Issue Type: extRequest
>          Components: FIWARE-LAB-HELP
>            Reporter: FW External User
>            Assignee: Joaquín Salvachúa 
>         Attachments: [FIWARE-JIRA] (HELP-12848) [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-12870) [SPAM] Website Job Application, [FIWARE-JIRA] (HELP-12848) [Fiware-lab-help] Hive - JSonSerde library - Cygnus
>
>
> Hi ,
> I have configured the Cygnus instance with the following configuration for Hive:
> cygnus-ngsi.sinks.hdfs-sink.hive.host = cosmos.lab.fiware.org
> cygnus-ngsi.sinks.hdfs-sink.hive.server_version = 2
> cygnus-ngsi.sinks.hdfs-sink.file_format = json-column
> cygnus-ngsi.sinks.hdfs-sink.csv_separator = ,
> cygnus-ngsi.sinks.hdfs-sink.hdfs_host = storage.cosmos.lab.fiware.org
> cygnus-ngsi.sinks.hdfs-sink.hdfs_port = 14000
> cygnus-ngsi.sinks.hdfs-sink.hdfs_username = <username>
> cygnus-ngsi.sinks.hdfs-sink.hdfs_password = <password>
> cygnus-ngsi.sinks.hdfs-sink.oauth2_token = <token>
> cygnus-ngsi.sinks.hdfs-sink.hive = true
> After table creation, I have received the below error message:
> Error message:
> SQL Error [1] [08S01]: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
> SQL Error [1] [08S01]: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
>   Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
>   Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
> Ddl hive table:
> create external table if not exists default.mellusom_defaultx002fx0053x0043x0053x002d004_x0053martx0043ityx0053ensor_column
> (
> recvx0054ime string,
> fiwarex0053ervicex0050ath string,
> entityx0049d string,
> entityx0054ype string,
> brightness string,
> brightness_md array<struct<name:string,type:string,value:string>>,
> city string,city_md array<struct<name:string,type:string,value:string>>,
> continent string,continent_md array<struct<name:string,type:string,value:string>>,
> country string,country_md array<struct<name:string,type:string,value:string>>,
> location string,location_md array<struct<name:string,type:string,value:string>>,
> owner string,owner_md array<struct<name:string,type:string,value:string>>,
> region string,region_md array<struct<name:string,type:string,value:string>>,
> temperature_c string,temperature_c_md array<struct<name:string,type:string,value:string>>,
> zip string,zip_md array<struct<name:string,type:string,value:string>>)
> row format serde 'org.openx.data.jsonserde.JsonSerDe' with serdeproperties ("dots.in.keys" = "true")
> location '/user/mellusom/default/SCS-004_SmartCitySensor'
> I solved the issue regarding the error message using this link:
> https://community.cloudera.com/t5/Batch-SQL-Apache-Hive/Where-to-put-json-serde-1-3-7-jar-with-dependencies-jar-file/td-p/48073<http://cp.mcafee.com/d/avndygsd3gOrhvKYY-U-NsTsSyyPtBBBVBMSyyOyevKUYyrhhp76zBWXVEVd79J55VNdNZcQszzrItxrWmHOSHr7PPZ1lGNcWuvaDiCrJm9DjPVkWkPuE9LK-qen-LOrz1EVd7bPTnKnjpujVxUsUOUVR4mlHFGTppVkffGhBrwqrhdI6XYCyOzsQsIL8I6XCOsVHkiP5RmUiCmWNZmQfzLbVd_1DpS9BNw1MGn45JIjYK5V6vM1ivNYLeFtpKpshFm1uDNgYKynpbzyUzIiVykOpt3RLwxYbi-xshS9tSh_Vd0KJz3RNP9IhHjV4orqluJIo_fUR1jIz6N_w09JAsYYyCr1vF6y0bOc-q82VEw0woH0o-a7Cy0-h2eNb6y0Kq818IhwTgSOedBjl2gIAwSd_>
> and after that I have tried to deploy on my local Hive istance the jar library recovered by http://www.congiu.net/hive-json-serde/<http://cp.mcafee.com/d/FZsS93gwrhvKYY-U-NsTsSyyPtBBBVBMSyyOyevKUYyrhhp76zBWXVEVd79J55VNdNZcQszzrItxrWmHOSHr7PPZ1lGNcWuvaDiCrJm9DjPVkWkPuE9LK-qen-LOrz1EVd7bPTnKnjpujVxUsUOUVR4mlHFGTppVkffGhBrwqrjdI6XYCyOzsQsIL8I6XCM0lGM8OrU6UjApYK8QH0LjUEvxIhHjV4orqluJIo_fUR1jIz6N_w09JAsYYyCr1vF6y0bOc-q82VEw0woH0o-a7Cy0-h2eNb6y0Kq818IhwTgSOedFPnyNnzG> (version 1.3.8)
> and now It's correctly working in local.
> Best regards.
> Francesco Paparella
> Ricerca e Sviluppo
> Corvallis S.p.A. a Socio Unico
> Attività di direzione e coordinamento Corvallis Holding SpA
> Gruppo Corvallis
> Viale Aldo Moro, 36
> 73100 Lecce
> Italia
> Telefono         +39 0832 069901
> Telefax   +39 0832 069911
> [email-masked]<mailto:[email-masked]>
> www.corvallis.it<http://www.corvallis.it/>
> Le informazioni contenute in questo messaggio di posta elettronica e negli eventuali allegati sono riservate e confidenziali e sono indirizzate esclusivamente al destinatario. Si prega di non fare copia, inoltrare a terzi o conservare tale messaggio se non si è il legittimo destinatario dello stesso. Qualora questo messaggio sia stato ricevuto per errore, si prega di rinviarlo al mittente e di cancellarlo permanentemente dal proprio computer.
> The information contained in this message and in any attachment is intended exclusively for the recipient. If you are not the intended recipient you are hereby notified not to copy, save, disclose, or distribute it to any third party. If you erroneously received this message you are kindly requested  to return it to the sender and eliminate it permanently from your computer.
> __________________________________________________________________________________________
> You can get more information about our cookies and privacy policies on the following links:
> - http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/FIWARE_Privacy_Policy
> - http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Cookies_Policy_FIWARE
> Fiware-lab-help mailing list
> [email-masked]
> https://lists.fiware.org/listinfo/fiware-lab-help
> [Created via e-mail received from: Paparella Francesco <[email-masked]>]

--
This message was sent by Atlassian JIRA
(v6.4.1#64016)