Dear Andrea,

As you have noticed, this is a problem regarding the Hadoop version
(pretty old) we are running at the global instance. This will be fixed
once we release the new global instance (end of the year/start of the new
one), running a modern version of Hadoop based on Hortonworks.

Thanks for the feedback.

Regards,
Francisco

El 1/12/15 17:05, "Manuel Escriche (JIRA)" <jira-help-desk@fi-ware.org>

________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
Please, send your messages using the new domain (Fiware-tech-help@lists.fiware.org) instead of the old one.
_______________________________________________
Fiware-tech-help mailing list
Fiware-tech-help@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-tech-help

Thanks Francisco,

If you do not mind, I would like to discuss another point.

We decided to develop a Spark pipeline of tasks to process our data.
Currently we are using our private Hadoop single node instance to execute
the Spark jobs. Unfortunately, only recently we found out that Spark is not
deployed on Cosmos, but we need a full integration of the Cosmos GE within
our architecture. I'm not sure that making our product viable by using a
plain private Hadoop will be enough.

I found that Apache Oozie can schedule
<https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html> Spark
jobs and I was wondering if in the next Hadoop deployment on Cosmos also
Spark will be installed.

Thanks in advance for your collaboration.

Regards,
Andrea

2015-12-02 11:39 GMT+01:00 FRANCISCO ROMERO BUENO <

Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
Please, send your messages using the new domain (Fiware-tech-help@lists.fiware.org) instead of the old one.
_______________________________________________
Fiware-tech-help mailing list
Fiware-tech-help@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-tech-help

Dear Andrea, Dear Francisco,

I don't see progress on this request for more than 10 days.
Is the issue over? Can we close it?

Andrea, if the question posed is not related to this request, please set another questions so that we can close this request.

Thanks in advance!
Manuel

Dear Manuel,

the issue is not over since it will be fixed with a new deployment of
Cosmos. I am actually waiting for it with a modern version of Hadoop. I
hope also Spark will be included in the new deployment.

Best Regards,
Andrea

Dear Andrea,
Thanks for feedback!

Dear Francisco,
When do you foresee to deploy again? Release5.1?
Thanks in advance!

Kind regards,
Manuel

The deployment started several months ago, this is a very ambitious task.
We are in the latest steps, most probably it will be ready after the Xmas
holidays.

Regards,
Francisco

El 16/12/15 8:38, "Manuel Escriche (JIRA)" <jira-help-desk@fi-ware.org>

________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
Please, send your messages using the new domain (Fiware-tech-help@lists.fiware.org) instead of the old one.
_______________________________________________
Fiware-tech-help mailing list
Fiware-tech-help@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-tech-help

Adding impediment: the user is waiting for the new cluster deployment.

Hi all,

Regarding this issue, it must be said the new cluster for storage is
finally working. This new cluster is based on a modern Hadoop and the
problem regarding offsets in WebHDFS should be fixed.

As said, this cluster is only suited for storage, not computing (a second
cluster specifically addressing computing will be released in a few
weeks). So, if your use case is about storing data in HDFS and consuming
it through WebHDFS, with no Hive, no MapReduce, then you are ready to move
from the ³old² cluster (cosmos.lab.fiware.org) to the new one
(storage.cosmos.lab.fiware.org). If you are interested, let me now and
I¹ll provision the account and I¹ll migrate the data from one cluster to
the other.

Regards,
Francisco

El 16/12/15 8:56, "FRANCISCO ROMERO BUENO"

________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
Please, send your messages using the new domain (Fiware-tech-help@lists.fiware.org) instead of the old one.
_______________________________________________
Fiware-tech-help mailing list
Fiware-tech-help@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-tech-help

Any update? still impeded? Thanks

Hi all,

This issue may be considered fixed once the new storage cluster has been
announced. I¹m copying the announce:

‹‹‹‹‹

Dear FIWARE Lab user,
Few months ago our Big Data team started working on deploying a new global
instance of Cosmos. The current "old" one is reaching its limit, both in
storage and computing capabilities, so moving to a large and powerful
cluster was a need.
Regarding this new instance, we have decided to split storage and
computing into two different clusters. That's why I'm glad to announce the
storage one is ready to be used. Nevertheless, some remarks must be done:

• It must be considered as a beta. Errors and missconfigurations may still
  arise.

• It is only suited for storage purposes. The software stack only
  comprises HDFS, not MapReduce nor any other computing tool.

• No ssh access has been enabled. From the current "old" cluster we have
  learn most of people used the ssh access to install their own software or
  allocating vast ammounts of data in the local file system, which is a very
  bad behaviour since the clusters are always shared. Thus, the only allowed
  access is through WebHDFS REST API, which is protected by OAuth2.

This cluster, and the upcoming computing one, will replace the current
"old" cluster in the short time, based on a modern Hadoop distribution.
Nevertheless, for now it is not mandatory to use it. Moving to this
cluster is only recommended:

• If you are interested on pure storage (i.e. no analytics on the data,
  for instance, if you are using Cygnus for building historics about Orion's
  context data)

• and, as much, the consumption of your data is done through WebHDFS REST
  API.

It must be said those of you moving right now to this cluster will enjoy a
larger HDFS quota. If you are interested, please sent an email to
francisco.romerobueno@telefonica.com
Any doubt you may have, do not hesitate to contact us.
Best regards,
Cosmos Big Data team

El 17/2/16 10:51, "Manuel Escriche (JIRA)" <jira-help-desk@fi-ware.org>

________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
Please, send your messages using the new domain (Fiware-tech-help@lists.fiware.org) instead of the old one.
_______________________________________________
Fiware-tech-help mailing list
Fiware-tech-help@lists.fiware.org
https://lists.fiware.org/listinfo/fiware-tech-help