Hi,

  I copy Francisco (Paco) Romero who should be able to provide some help on this.   I guess he won't be able to work on the matter until tomorrow since support is not granted out of office working hours.

  Nevertheless, I believe that it is rather important that you open an issue so that a ticket on our help-desk system is created and then follow-up of support is feasible.

  You may create a ticket in our helpdesk simply by sending an email to either fiware-lab-help (if it happens to be an issue which is mostly related to FIWARE Lab or the global COSMOS instance setup on the FIWARE Lab) or fiware-tech-help (if it happens to be an issue mostly related with the technology, i.e., with COSMOS in general and not the particular instance of COSMOS running on the FIWARE Lab).

  Best regards,

– Juanjo

Hi Juanjo,

Thanks for you feedback, we sent it today because we wanted you to answer
it tomorrow morning .

As you can see, a ticket has been created in Jira (
https://jira.fiware.org/browse/HELC-1025). I used the
fiware-finodex-coaching list, because we are a FINODEX project and they
told us to do so. Should we create another ticket in the fiware-tech-help
list also?

Best regards

–
Carlos Corrales
Cofounder Secmotic Innovation S.L.
http://www.secmotic.com

_______________________________________________
Fiware-finodex-coaching mailing list
Fiware-finodex-coaching@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-finodex-coaching-new

Hi Carlos,

I do not know if this will help. The same thing happened to us.
http://stackoverflow.com/questions/31310111/oauth2-in-cygnus/31312196#31312196

Regards

firma
*SensoWave
Ignacio Gómez Maqueda*
*
*
Director Técnico

móvil: + 34 644350613

_imaqueda@sensowave.es <candres@sensowave.es>_

@queronea

_______________________________________________
Fiware-finodex-coaching mailing list
Fiware-finodex-coaching@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-finodex-coaching-new

  I guess that the FINODEX coach will forward the ticket to the FIWARE GE owner (or eventually, if he hadn't know where to redirect it, he would redirect the original question to the fiware-tech-help mailing list)

  Point is that one way or the other, it would arrive at the inbox of the FIWARE GE owner who has to take care of the ticket.

  You may have sent the ticket directly to the fiware-tech-help mailing list ... however, I guess that the FINODEX coach team wanted to be informed to keep track of the evolution.

  Manuel, in copy, is the master of all the helpdesk processes and he can clarify what the actual workflow is.

  Best regards,

– Juanjo

dear carlos,

in fact, as an sme working in finodex you did correctly.
it will be your fiware coach (pasquale vitale in cc) that if not able to
solve himself the issue, and i guess this time he will not, he will turn
the ticket to the ge owner (francisco romero as indicated by juanjo).

ciao,
stefano

–
Stefano De Panfilis
Chief Innovation Officer
Engineering Ingegneria Informatica S.p.A.
via Riccardo Morandi 32
00148 Roma
Italy

tel (direct): +39-06-8759-4253
tel (secr.): +39-068307-4513
fax: +39-068307-4200
cell: +39-335-7542-567
skype: depa01
twitter: @depa01

_______________________________________________
Fiware-finodex-coaching mailing list
Fiware-finodex-coaching@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-finodex-coaching-new

see the helc 1026

Hi everyone,

Thanks for your responses, we have been testing the whole day and trying to
find out the issue. It is not a problem with neither the password, nor the
oauth2 token. We are able to access using ssh user/password and the curl
command proposed by Pasquale with Oauth2 token. This is what I received
using the Oauth2 token curl proposed by Pasquale

curl -X GET "
http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/idelgado?op=liststatus&user.name=idelgado"
-H "X-Auth-Token: rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i"
{"FileStatuses":{"FileStatus":[

,

,

,

,

,

,

,

]}}

Everything seems to be correct. However, using the same token and updating
ORION, I receive the following error in Cygnus:

time=2015-09-14T18:10:44.419CEST | lvl=ERROR |
trans=1442246992-282-0000000000 | function=process | comp=Cygnus |
msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[140] :
Persistence error (The org42/idelgado_tablanull directory could not be
created in HDFS. HttpFS response: 401 Unauthorized)
time=2015-09-14T18:10:44.419CEST | lvl=WARN |
trans=1442246992-282-0000000000 | function=process | comp=Cygnus |
msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[153] : The event
TTL has expired, it is no more re-injected in the channel (id=1444248959,
ttl=0)

However, if I try to access the directory shown in the error, I'm able to
do so

ssh idelgado@cosmos.lab.fi-ware.org
hadoop fs -ls /user/idelgado/org42/idelgado_tablanull
Found 1 items
rw-r---- 3 idelgado idelgado 1416 2015-07-01 13:11
/user/idelgado/org42/idelgado_tablanull/idelgado_tablanull.txt

My cygnus.conf file content is:

#
============================================

1. OrionHDFSSink
   configuration

1. channel name from where to read notification
   events
   cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
2. sink class, must not be
   changed

cygnusagent.sinks.hdfs-sink.type =
es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink

1. Comma-separated list of FQDN/IP address regarding the Cosmos Namenode
   endpoints
   cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
2. port of the Cosmos service listening for persistence operations; 14000
   for httpfs, 50070 for webhdfs and free c\
   hoice for
   inifinty

cygnusagent.sinks.hdfs-sink.cosmos_port = 14000

1. default username allowed to write in
   HDFS
   cygnusagent.sinks.hdfs-sink.cosmos_default_username = idelgado
2. default password for the default
   username

#cygnusagent.sinks.hdfs-sink.cosmos_default_password = XXXXXXX

cygnusagent.sinks.hdfs-sink.oauth2_token = rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i

1. HDFS backend type (webhdfs, httpfs or
   infinity)
   cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
2. how the attributes are stored, either per row either per column (row,
   column)
   cygnusagent.sinks.hdfs-sink.attr_persistence = column
3. prefix for the database and table names, empty if no prefix is
   desired
   cygnusagent.sinks.hdfs-sink.naming_prefix = idelgado_tabla
4. Hive FQDN/IP address of the Hive
   server

cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46

1. Hive port for Hive external table
   provisioning

cygnusagent.sinks.hdfs-sink.hive_port = 10000

Could you please help us?

Thanks in advance!

2015-09-14 10:26 GMT+02:00 MANUEL ESCRICHE VICENTE <

–
Carlos Corrales
Cofounder Secmotic Innovation S.L.
http://www.secmotic.com

_______________________________________________
Fiware-finodex-coaching mailing list
Fiware-finodex-coaching@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-finodex-coaching-new

One more thing, Francisco, how can I find the Cygnus version?

Best regards

–
Carlos Corrales
Cofounder Secmotic Innovation S.L.
http://www.secmotic.com

_______________________________________________
Fiware-finodex-coaching mailing list
Fiware-finodex-coaching@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-finodex-coaching-new

Hi everyone again,

Could it be related to the Fiware-Service and Fiware-ServicePath headers?
We didn't use them at the beginning, then we tried to use them, and now, we
notice that if we didn't use them the directory used by Cygnus to write in
Cosmos is /org42/idelgado_tablanull, while if we use for instance

--header 'Fiware-Service: unknownT' --header 'Fiware-ServicePath: /'

the directory written is /unknownt/idelgado_tablanull.

Any clues?

Best regards

–
Carlos Corrales
Cofounder Secmotic Innovation S.L.
http://www.secmotic.com

_______________________________________________
Fiware-finodex-coaching mailing list
Fiware-finodex-coaching@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-finodex-coaching-new

The issue has been emailed:

• Time sent: 15/Sep/15 1:08 PM
• To: Fiware-finodex-coaching@lists.fiware.org
• with subject: *(HELC-1025) [Fiware-finodex-coaching] Problems with COSMOS persistance *
  

Dear Carlos,
Fiware-Service and Fiware-ServicePath headers are important when you want to use Cygnus with a Database. They are the name of table in the database.

Best regards,
Pasquale

From: FRANCISCO ROMERO BUENO
Sent: martes, 15 de septiembre de 2015 9:12
To: Carlos Corrales; MANUEL ESCRICHE VICENTE
Cc: JUAN JOSE HIERRO SUREDA; fiware-finodex-coaching@lists.fi-ware.org; jose@secmotic.com
Subject: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

Hi Carlos,

I will aggregate through this email all the other emails you sent.

Regarding the version, it is printed in the log at the very beginning of the execution. In addition, if you installed via RPM you can check for the installed version by typing:

$ rpm –qa | grep cygnus

Nevertheless, looking at your configuration, it seems you are using a very old version of Cygnus, or you are mixing parameters from different versions. I mean:
Current version is 0.8.2
The "cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink” part is previous to 0.8.0. From 0.8.0 it must be “com.telefonica.iot.cygnus.sinks.OrionCKANSink”
The OAuth2 token was introduced in 0.8.2
My recommendation is you install the latest version from RPM (https://github.com/telefonicaid/fiware-cygnus/tree/master#section4.1) and you configure Cygnus following the guidelines appearing in the master or release-0.8.2 branches (they are synchronized):
https://github.com/telefonicaid/fiware-cygnus/blob/master/README.md
https://github.com/telefonicaid/fiware-cygnus/blob/release/0.8.2/README.md

Regarding the fiware-service and fiware-servicePath values, if they are not notified as http headers, then the default configured values in Cygnus’ HTTPSource are used. Again, it seems you are not using/configuring these values.

Regards,
Francisco

Hi Francisco,

Yes, we were using cygnus 0.5-91... It is the default version when using
the image available in FIWARE lab (orion-psb-image-R4.2
<https://cloud.lab.fiware.org/#nova/images/7dd7a0cc-3e3b-4eb3-a50c-4a2e3e498df3>
).

After updating the version and configuring it properly, everything is
working again.

Thank you very much!

2015-09-15 9:45 GMT+02:00 MANUEL ESCRICHE VICENTE <

–
Carlos Corrales
Cofounder Secmotic Innovation S.L.
http://www.secmotic.com

_______________________________________________
Fiware-finodex-coaching mailing list
Fiware-finodex-coaching@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-finodex-coaching-new

Hi Carlos,

I will aggregate through this email all the other emails you sent.

Regarding the version, it is printed in the log at the very beginning of the execution. In addition, if you installed via RPM you can check for the installed version by typing:

$ rpm –qa | grep cygnus

Nevertheless, looking at your configuration, it seems you are using a very old version of Cygnus, or you are mixing parameters from different versions. I mean:

• Current version is 0.8.2
• The "cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink” part is previous to 0.8.0. From 0.8.0 it must be “com.telefonica.iot.cygnus.sinks.OrionCKANSink”
• The OAuth2 token was introduced in 0.8.2

My recommendation is you install the latest version from RPM (https://github.com/telefonicaid/fiware-cygnus/tree/master#section4.1) and you configure Cygnus following the guidelines appearing in the master or release-0.8.2 branches (they are synchronized):
https://github.com/telefonicaid/fiware-cygnus/blob/master/README.md
https://github.com/telefonicaid/fiware-cygnus/blob/release/0.8.2/README.md<https://github.com/telefonicaid/fiware-cygnus/tree/release/0.8.2>

Regarding the fiware-service and fiware-servicePath values, if they are not notified as http headers, then the default configured values in Cygnus’ HTTPSource are used. Again, it seems you are not using/configuring these values.

Regards,
Francisco

De: Carlos Corrales <carlos@secmotic.com<carlos@secmotic.com>>
Fecha: lunes, 14 de septiembre de 2015, 18:22
Para: MANUEL ESCRICHE VICENTE <manuel.escrichevicente@telefonica.com<manuel.escrichevicente@telefonica.com>>
CC: JUAN JOSE HIERRO SUREDA <juanjose.hierro@telefonica.com<juanjose.hierro@telefonica.com>>, "fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>" <fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>>, "jose@secmotic.com<jose@secmotic.com>" <jose@secmotic.com<jose@secmotic.com>>, Francisco Romero Bueno <francisco.romerobueno@telefonica.com<francisco.romerobueno@telefonica.com>>
Asunto: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

Hi everyone,

Thanks for your responses, we have been testing the whole day and trying to find out the issue. It is not a problem with neither the password, nor the oauth2 token. We are able to access using ssh user/password and the curl command proposed by Pasquale with Oauth2 token. This is what I received using the Oauth2 token curl proposed by Pasquale

curl -X GET "http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/idelgado?op=liststatus&user.name=idelgado" -H "X-Auth-Token: rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i"
{"FileStatuses":{"FileStatus":[

,

,

,

,

,

,

,

]}}

Everything seems to be correct. However, using the same token and updating ORION, I receive the following error in Cygnus:

time=2015-09-14T18:10:44.419CEST | lvl=ERROR | trans=1442246992-282-0000000000 | function=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[140] : Persistence error (The org42/idelgado_tablanull directory could not be created in HDFS. HttpFS response: 401 Unauthorized)
time=2015-09-14T18:10:44.419CEST | lvl=WARN | trans=1442246992-282-0000000000 | function=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[153] : The event TTL has expired, it is no more re-injected in the channel (id=1444248959, ttl=0)

However, if I try to access the directory shown in the error, I'm able to do so

ssh idelgado@cosmos.lab.fi-ware.org<idelgado@cosmos.lab.fi-ware.org>
hadoop fs -ls /user/idelgado/org42/idelgado_tablanull
Found 1 items
rw-r---- 3 idelgado idelgado 1416 2015-07-01 13:11 /user/idelgado/org42/idelgado_tablanull/idelgado_tablanull.txt

My cygnus.conf file content is:

1. ============================================
2. OrionHDFSSink configuration
3. channel name from where to read notification events
   cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
4. sink class, must not be changed
   cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
5. Comma-separated list of FQDN/IP address regarding the Cosmos Namenode endpoints
   cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
6. port of the Cosmos service listening for persistence operations; 14000 for httpfs, 50070 for webhdfs and free c\
   hoice for inifinty
   cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
7. default username allowed to write in HDFS
   cygnusagent.sinks.hdfs-sink.cosmos_default_username = idelgado
8. default password for the default username
   #cygnusagent.sinks.hdfs-sink.cosmos_default_password = XXXXXXX
   cygnusagent.sinks.hdfs-sink.oauth2_token = rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i
9. HDFS backend type (webhdfs, httpfs or infinity)
   cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
10. how the attributes are stored, either per row either per column (row, column)
    cygnusagent.sinks.hdfs-sink.attr_persistence = column
11. prefix for the database and table names, empty if no prefix is desired
    cygnusagent.sinks.hdfs-sink.naming_prefix = idelgado_tabla
12. Hive FQDN/IP address of the Hive server
    cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
13. Hive port for Hive external table provisioning
    cygnusagent.sinks.hdfs-sink.hive_port = 10000

Could you please help us?

Thanks in advance!

2015-09-14 10:26 GMT+02:00 MANUEL ESCRICHE VICENTE <manuel.escrichevicente@telefonica.com<manuel.escrichevicente@telefonica.com>>:
In this case I see both Pasquale (Coach) and Francisco (GE Owner) have reacted to the request.
In the normal flow GE owners enter into action upon request from Coaches.
Kind regards,
Manuel

From: JUAN JOSE HIERRO SUREDA
Sent: lunes, 14 de septiembre de 2015 0:37
To: Carlos Corrales
Cc: fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>; jose@secmotic.com<jose@secmotic.com>; FRANCISCO ROMERO BUENO; MANUEL ESCRICHE VICENTE; JUAN JOSE HIERRO SUREDA
Subject: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

I guess that the FINODEX coach will forward the ticket to the FIWARE GE owner (or eventually, if he hadn't know where to redirect it, he would redirect the original question to the fiware-tech-help mailing list)

Point is that one way or the other, it would arrive at the inbox of the FIWARE GE owner who has to take care of the ticket.

You may have sent the ticket directly to the fiware-tech-help mailing list ... however, I guess that the FINODEX coach team wanted to be informed to keep track of the evolution.

Manuel, in copy, is the master of all the helpdesk processes and he can clarify what the actual workflow is.

Best regards,

– Juanjo

Great to hear that!

Regarding the image containing the bundle Orion+Cygnus, it must be updated for sure. Adding it to our backlog.

Regards,
Francisco

De: Carlos Corrales <carlos@secmotic.com<carlos@secmotic.com>>
Fecha: miércoles, 16 de septiembre de 2015, 11:11
Para: MANUEL ESCRICHE VICENTE <manuel.escrichevicente@telefonica.com<manuel.escrichevicente@telefonica.com>>
CC: Francisco Romero Bueno <francisco.romerobueno@telefonica.com<francisco.romerobueno@telefonica.com>>, JUAN JOSE HIERRO SUREDA <juanjose.hierro@telefonica.com<juanjose.hierro@telefonica.com>>, "fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>" <fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>>, "jose@secmotic.com<jose@secmotic.com>" <jose@secmotic.com<jose@secmotic.com>>
Asunto: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

Hi Francisco,

Yes, we were using cygnus 0.5-91... It is the default version when using the image available in FIWARE lab (orion-psb-image-R4.2<https://cloud.lab.fiware.org/#nova/images/7dd7a0cc-3e3b-4eb3-a50c-4a2e3e498df3>).

After updating the version and configuring it properly, everything is working again.

Thank you very much!

2015-09-15 9:45 GMT+02:00 MANUEL ESCRICHE VICENTE <manuel.escrichevicente@telefonica.com<manuel.escrichevicente@telefonica.com>>:
Be aware there exist this issue in JIRA for this exchange: https://jira.fiware.org/browse/HELC-1025

From: FRANCISCO ROMERO BUENO
Sent: martes, 15 de septiembre de 2015 9:12
To: Carlos Corrales; MANUEL ESCRICHE VICENTE
Cc: JUAN JOSE HIERRO SUREDA; fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>; jose@secmotic.com<jose@secmotic.com>

Subject: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

Hi Carlos,

I will aggregate through this email all the other emails you sent.

Regarding the version, it is printed in the log at the very beginning of the execution. In addition, if you installed via RPM you can check for the installed version by typing:

$ rpm –qa | grep cygnus

Nevertheless, looking at your configuration, it seems you are using a very old version of Cygnus, or you are mixing parameters from different versions. I mean:

• Current version is 0.8.2
• The "cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink” part is previous to 0.8.0. From 0.8.0 it must be “com.telefonica.iot.cygnus.sinks.OrionCKANSink”
• The OAuth2 token was introduced in 0.8.2
  My recommendation is you install the latest version from RPM (https://github.com/telefonicaid/fiware-cygnus/tree/master#section4.1) and you configure Cygnus following the guidelines appearing in the master or release-0.8.2 branches (they are synchronized):
  https://github.com/telefonicaid/fiware-cygnus/blob/master/README.md
  https://github.com/telefonicaid/fiware-cygnus/blob/release/0.8.2/README.md<https://github.com/telefonicaid/fiware-cygnus/tree/release/0.8.2>

Regarding the fiware-service and fiware-servicePath values, if they are not notified as http headers, then the default configured values in Cygnus’ HTTPSource are used. Again, it seems you are not using/configuring these values.

Regards,
Francisco

De: Carlos Corrales <carlos@secmotic.com<carlos@secmotic.com>>
Fecha: lunes, 14 de septiembre de 2015, 18:22
Para: MANUEL ESCRICHE VICENTE <manuel.escrichevicente@telefonica.com<manuel.escrichevicente@telefonica.com>>
CC: JUAN JOSE HIERRO SUREDA <juanjose.hierro@telefonica.com<juanjose.hierro@telefonica.com>>, "fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>" <fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>>, "jose@secmotic.com<jose@secmotic.com>" <jose@secmotic.com<jose@secmotic.com>>, Francisco Romero Bueno <francisco.romerobueno@telefonica.com<francisco.romerobueno@telefonica.com>>
Asunto: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

Hi everyone,
Thanks for your responses, we have been testing the whole day and trying to find out the issue. It is not a problem with neither the password, nor the oauth2 token. We are able to access using ssh user/password and the curl command proposed by Pasquale with Oauth2 token. This is what I received using the Oauth2 token curl proposed by Pasquale

curl -X GET "http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/idelgado?op=liststatus&user.name=idelgado" -H "X-Auth-Token: rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i"
{"FileStatuses":{"FileStatus":[

,

,

,

,

,

,

,

]}}

Everything seems to be correct. However, using the same token and updating ORION, I receive the following error in Cygnus:

time=2015-09-14T18:10:44.419CEST | lvl=ERROR | trans=1442246992-282-0000000000 | function=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[140] : Persistence error (The org42/idelgado_tablanull directory could not be created in HDFS. HttpFS response: 401 Unauthorized)
time=2015-09-14T18:10:44.419CEST | lvl=WARN | trans=1442246992-282-0000000000 | function=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[153] : The event TTL has expired, it is no more re-injected in the channel (id=1444248959, ttl=0)
However, if I try to access the directory shown in the error, I'm able to do so

ssh idelgado@cosmos.lab.fi-ware.org<idelgado@cosmos.lab.fi-ware.org>
hadoop fs -ls /user/idelgado/org42/idelgado_tablanull
Found 1 items
rw-r---- 3 idelgado idelgado 1416 2015-07-01 13:11 /user/idelgado/org42/idelgado_tablanull/idelgado_tablanull.txt
My cygnus.conf file content is:

1. ============================================
2. OrionHDFSSink configuration
3. channel name from where to read notification events
   cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
4. sink class, must not be changed
   cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
5. Comma-separated list of FQDN/IP address regarding the Cosmos Namenode endpoints
   cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
6. port of the Cosmos service listening for persistence operations; 14000 for httpfs, 50070 for webhdfs and free c\
   hoice for inifinty
   cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
7. default username allowed to write in HDFS
   cygnusagent.sinks.hdfs-sink.cosmos_default_username = idelgado
8. default password for the default username
   #cygnusagent.sinks.hdfs-sink.cosmos_default_password = XXXXXXX
   cygnusagent.sinks.hdfs-sink.oauth2_token = rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i
9. HDFS backend type (webhdfs, httpfs or infinity)
   cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
10. how the attributes are stored, either per row either per column (row, column)
    cygnusagent.sinks.hdfs-sink.attr_persistence = column
11. prefix for the database and table names, empty if no prefix is desired
    cygnusagent.sinks.hdfs-sink.naming_prefix = idelgado_tabla
12. Hive FQDN/IP address of the Hive server
    cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
13. Hive port for Hive external table provisioning
    cygnusagent.sinks.hdfs-sink.hive_port = 10000
    Could you please help us?
    Thanks in advance!

2015-09-14 10:26 GMT+02:00 MANUEL ESCRICHE VICENTE <manuel.escrichevicente@telefonica.com<manuel.escrichevicente@telefonica.com>>:
In this case I see both Pasquale (Coach) and Francisco (GE Owner) have reacted to the request.
In the normal flow GE owners enter into action upon request from Coaches.
Kind regards,
Manuel

From: JUAN JOSE HIERRO SUREDA
Sent: lunes, 14 de septiembre de 2015 0:37
To: Carlos Corrales
Cc: fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>; jose@secmotic.com<jose@secmotic.com>; FRANCISCO ROMERO BUENO; MANUEL ESCRICHE VICENTE; JUAN JOSE HIERRO SUREDA
Subject: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

I guess that the FINODEX coach will forward the ticket to the FIWARE GE owner (or eventually, if he hadn't know where to redirect it, he would redirect the original question to the fiware-tech-help mailing list)

Point is that one way or the other, it would arrive at the inbox of the FIWARE GE owner who has to take care of the ticket.

You may have sent the ticket directly to the fiware-tech-help mailing list ... however, I guess that the FINODEX coach team wanted to be informed to keep track of the evolution.

Manuel, in copy, is the master of all the helpdesk processes and he can clarify what the actual workflow is.

Best regards,

– Juanjo

In this case I see both Pasquale (Coach) and Francisco (GE Owner) have reacted to the request.
In the normal flow GE owners enter into action upon request from Coaches.
Kind regards,
Manuel

From: JUAN JOSE HIERRO SUREDA
Sent: lunes, 14 de septiembre de 2015 0:37
To: Carlos Corrales
Cc: fiware-finodex-coaching@lists.fi-ware.org; jose@secmotic.com; FRANCISCO ROMERO BUENO; MANUEL ESCRICHE VICENTE; JUAN JOSE HIERRO SUREDA
Subject: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

I guess that the FINODEX coach will forward the ticket to the FIWARE GE owner (or eventually, if he hadn't know where to redirect it, he would redirect the original question to the fiware-tech-help mailing list)

Point is that one way or the other, it would arrive at the inbox of the FIWARE GE owner who has to take care of the ticket.

You may have sent the ticket directly to the fiware-tech-help mailing list ... however, I guess that the FINODEX coach team wanted to be informed to keep track of the evolution.

Manuel, in copy, is the master of all the helpdesk processes and he can clarify what the actual workflow is.

Best regards,

– Juanjo

Be aware there exist this issue in JIRA for this exchange: https://jira.fiware.org/browse/HELC-1025

From: FRANCISCO ROMERO BUENO
Sent: martes, 15 de septiembre de 2015 9:12
To: Carlos Corrales; MANUEL ESCRICHE VICENTE
Cc: JUAN JOSE HIERRO SUREDA; fiware-finodex-coaching@lists.fi-ware.org; jose@secmotic.com
Subject: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

Hi Carlos,

I will aggregate through this email all the other emails you sent.

Regarding the version, it is printed in the log at the very beginning of the execution. In addition, if you installed via RPM you can check for the installed version by typing:

$ rpm -qa | grep cygnus

Nevertheless, looking at your configuration, it seems you are using a very old version of Cygnus, or you are mixing parameters from different versions. I mean:

• Current version is 0.8.2
• The "cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink" part is previous to 0.8.0. From 0.8.0 it must be "com.telefonica.iot.cygnus.sinks.OrionCKANSink"
• The OAuth2 token was introduced in 0.8.2
  My recommendation is you install the latest version from RPM (https://github.com/telefonicaid/fiware-cygnus/tree/master#section4.1) and you configure Cygnus following the guidelines appearing in the master or release-0.8.2 branches (they are synchronized):
  https://github.com/telefonicaid/fiware-cygnus/blob/master/README.md
  https://github.com/telefonicaid/fiware-cygnus/blob/release/0.8.2/README.md<https://github.com/telefonicaid/fiware-cygnus/tree/release/0.8.2>

Regarding the fiware-service and fiware-servicePath values, if they are not notified as http headers, then the default configured values in Cygnus' HTTPSource are used. Again, it seems you are not using/configuring these values.

Regards,
Francisco

De: Carlos Corrales <carlos@secmotic.com<carlos@secmotic.com>>
Fecha: lunes, 14 de septiembre de 2015, 18:22
Para: MANUEL ESCRICHE VICENTE <manuel.escrichevicente@telefonica.com<manuel.escrichevicente@telefonica.com>>
CC: JUAN JOSE HIERRO SUREDA <juanjose.hierro@telefonica.com<juanjose.hierro@telefonica.com>>, "fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>" <fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>>, "jose@secmotic.com<jose@secmotic.com>" <jose@secmotic.com<jose@secmotic.com>>, Francisco Romero Bueno <francisco.romerobueno@telefonica.com<francisco.romerobueno@telefonica.com>>
Asunto: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

Hi everyone,
Thanks for your responses, we have been testing the whole day and trying to find out the issue. It is not a problem with neither the password, nor the oauth2 token. We are able to access using ssh user/password and the curl command proposed by Pasquale with Oauth2 token. This is what I received using the Oauth2 token curl proposed by Pasquale

curl -X GET "http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/idelgado?op=liststatus&user.name=idelgado" -H "X-Auth-Token: rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i"
{"FileStatuses":{"FileStatus":[

,

,

,

,

,

,

,

]}}

Everything seems to be correct. However, using the same token and updating ORION, I receive the following error in Cygnus:

time=2015-09-14T18:10:44.419CEST | lvl=ERROR | trans=1442246992-282-0000000000 | function=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[140] : Persistence error (The org42/idelgado_tablanull directory could not be created in HDFS. HttpFS response: 401 Unauthorized)
time=2015-09-14T18:10:44.419CEST | lvl=WARN | trans=1442246992-282-0000000000 | function=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[153] : The event TTL has expired, it is no more re-injected in the channel (id=1444248959, ttl=0)
However, if I try to access the directory shown in the error, I'm able to do so

ssh idelgado@cosmos.lab.fi-ware.org<idelgado@cosmos.lab.fi-ware.org>
hadoop fs -ls /user/idelgado/org42/idelgado_tablanull
Found 1 items
rw-r---- 3 idelgado idelgado 1416 2015-07-01 13:11 /user/idelgado/org42/idelgado_tablanull/idelgado_tablanull.txt
My cygnus.conf file content is:

1. ============================================
2. OrionHDFSSink configuration
3. channel name from where to read notification events
   cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
4. sink class, must not be changed
   cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
5. Comma-separated list of FQDN/IP address regarding the Cosmos Namenode endpoints
   cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
6. port of the Cosmos service listening for persistence operations; 14000 for httpfs, 50070 for webhdfs and free c\
   hoice for inifinty
   cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
7. default username allowed to write in HDFS
   cygnusagent.sinks.hdfs-sink.cosmos_default_username = idelgado
8. default password for the default username
   #cygnusagent.sinks.hdfs-sink.cosmos_default_password = XXXXXXX
   cygnusagent.sinks.hdfs-sink.oauth2_token = rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i
9. HDFS backend type (webhdfs, httpfs or infinity)
   cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
10. how the attributes are stored, either per row either per column (row, column)
    cygnusagent.sinks.hdfs-sink.attr_persistence = column
11. prefix for the database and table names, empty if no prefix is desired
    cygnusagent.sinks.hdfs-sink.naming_prefix = idelgado_tabla
12. Hive FQDN/IP address of the Hive server
    cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
13. Hive port for Hive external table provisioning
    cygnusagent.sinks.hdfs-sink.hive_port = 10000
    Could you please help us?
    Thanks in advance!

2015-09-14 10:26 GMT+02:00 MANUEL ESCRICHE VICENTE <manuel.escrichevicente@telefonica.com<manuel.escrichevicente@telefonica.com>>:
In this case I see both Pasquale (Coach) and Francisco (GE Owner) have reacted to the request.
In the normal flow GE owners enter into action upon request from Coaches.
Kind regards,
Manuel

From: JUAN JOSE HIERRO SUREDA
Sent: lunes, 14 de septiembre de 2015 0:37
To: Carlos Corrales
Cc: fiware-finodex-coaching@lists.fi-ware.org<fiware-finodex-coaching@lists.fi-ware.org>; jose@secmotic.com<jose@secmotic.com>; FRANCISCO ROMERO BUENO; MANUEL ESCRICHE VICENTE; JUAN JOSE HIERRO SUREDA
Subject: Re: [Fiware-finodex-coaching] Problems with COSMOS persistance

I guess that the FINODEX coach will forward the ticket to the FIWARE GE owner (or eventually, if he hadn't know where to redirect it, he would redirect the original question to the fiware-tech-help mailing list)

Point is that one way or the other, it would arrive at the inbox of the FIWARE GE owner who has to take care of the ticket.

You may have sent the ticket directly to the fiware-tech-help mailing list ... however, I guess that the FINODEX coach team wanted to be informed to keep track of the evolution.

Manuel, in copy, is the master of all the helpdesk processes and he can clarify what the actual workflow is.

Best regards,

– Juanjo