Details
-
Type:
Monitor
-
Status: Closed
-
Priority:
Major
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: 2021
-
Component/s: FIWARE-TECH-HELP
-
Labels:
-
HD-Chapter:Data
-
HD-Enabler:Cosmos
Description
Created question in FIWARE Q/A platform on 11-08-2015 at 11:08
Please, ANSWER this question AT https://stackoverflow.com/questions/31937580/cygnus-mysql-sink-error-attr-persistence-column
Question:
Cygnus MySQL sink error (attr_persistence = column)
Description:
I'm using cygnus to store data in both MySQL Database and Cosmos.
Storing the data in cosmos works ok, but when storing in MySQL, especifically with the attribute "cygnusagent.sinks.mysql-sink.attr_persistence = column" instead of "cygnusagent.sinks.mysql-sink.attr_persistence = row" the data is not stored and I'm getting some errors in flume log.
As the table needs to be created previously when using the column attribute(more info : https://github.com/telefonicaid/fiware-cygnus/blob/master/doc/design/OrionMySQLSink.md#important-notes-regarding-the-persistence-mode), I create the table:
CREATE TABLE def_servpath_sensorreading4_sensorreading(systemid int,value float, sensorid int, nodeid int);
DESCRIBE def_servpath_sensorreading4_sensorreading(systemid int,value float, sensorid int, nodeid int);
mysql> DESCRIBE def_servpath_sensorreading4_sensorreading;
------------------------------------+
| Field | Type | Null | Key | Default | Extra |
------------------------------------+
| systemid | int(11) | YES | NULL | ||
| value | float | YES | NULL | ||
| sensorid | int(11) | YES | NULL | ||
| nodeid | int(11) | YES | NULL |
------------------------------------+
4 rows in set (0.00 sec)
This is the flume log:
11 Aug 2015 11:40:31,977 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor] (com.telefonica.iot.cygnus.sinks.OrionMySQLSink.persist:240) - [mysql-sink] Persisting data at OrionMySQLSink. Database: def_serv, Table: def_servpath_sensorreading3_sensorreading, Timestamp: 2015-08-11T08:40:31.969, Data (attrs):
{systemid=1, value=29.2, sensorid=2, nodeid=1}, (metadata):
{sensorid_md=[], systemid_md=[], value_md=[], nodeid_md=[]}11 Aug 2015 11:40:31,978 WARN [SinkRunner-PollingRunner-DefaultSinkProcessor] (com.telefonica.iot.cygnus.sinks.OrionSink.process:182) - Bad context data (Unknown column 'systemid' in 'field list')
11 Aug 2015 11:40:31,978 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor] (com.telefonica.iot.cygnus.sinks.OrionSink.process:193) - Finishing transaction (1439278647-655-0000000005)
11 Aug 2015 11:40:32,369 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor] (com.telefonica.iot.cygnus.sinks.OrionHDFSSink.persist:356) - [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file (def_serv/def_servpath/sensorreading3_sensorreading/sensorreading3_sensorreading.txt), Data (
)
11 Aug 2015 11:40:32,858 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor] (com.telefonica.iot.cygnus.sinks.OrionSink.process:193) - Finishing transaction (1439278647-655-0000000005)
Where im getting the following error:
Bad context data (Unknown column 'systemid' in 'field list')
Do I need to create any more columns in the table related with the metadata? Am I creating the table correctly?
Thanks in advance.
Activity
2017-05-23 00:05|UPDATED status: transition Answer| # answers= 1, accepted answer= True
2017-05-23 03:05|UPDATED status: transition Answered| # answers= 1, accepted answer= True
2017-05-22 21:13|CREATED monitor | # answers= 1, accepted answer= True