Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-5605

FIWARE.Request.Tech.Data.BigData-Analysis.Medbravo - Cosmos GE HDFS Quota

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-TECH-HELP
    • Labels:
      None

      Description

      Dear All,

      Medbravo is continuously processing big-data on clinical research on
      cancer.
      In order to accommodate the increasing Medbravo data processing size
      in the resources
      provided in COSMOS Global Instance we would kindly ask you to for a larger
      quota in HDFS.
      Specifically *our current quota of 5 GB *is limiting us both input and
      output data size of Medbravo Hadoop MapReduce Jobs. Our current input for
      big data processing is now 8.5 GB and in the future it would grow bigger.
      Our output is larger as it is composed of input data plus additional
      information.
      Therefore we are asking you to increase our quota in HDFS. We need *at
      least 30 GB* to safely store input + output + possible intermediate data
      during processing.

      We will really appreciate your favourable response,

      We wish you a Merry Christmas and a Prosperous New Year,
      Aurelia

      Aurelia Bustos MD
      Cofounder

      tel: (+34) 618 453 214
      www.medbravo.org

      Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
      Please, send your messages using the new domain (Fiware-fiche-coaching@lists.fiware.org) instead of the old one.
      _______________________________________________
      Fiware-fiche-coaching mailing list
      Fiware-fiche-coaching@lists.fiware.org
      https://lists.fiware.org/listinfo/fiware-fiche-coaching
      [Created via e-mail received from: Aurelia Bustos <aurelia@medbravo.org>]

        Issue Links

          Activity

          Hide
          alfopietro Pietropaolo Alfonso added a comment -

          Dear Spain Team,
          would you please increase user's resources as asked within the ticket?
          Thanks.

          Show
          alfopietro Pietropaolo Alfonso added a comment - Dear Spain Team, would you please increase user's resources as asked within the ticket? Thanks.
          Hide
          frb Francisco Romero added a comment -

          Dear All,

          Medbravo is continuously processing big-data on clinical research on
          cancer.
          In order to accommodate the increasing Medbravo data processing size
          in the resources
          provided in COSMOS Global Instance we would kindly ask you to for a larger
          quota in HDFS.
          Specifically *our current quota of 5 GB *is limiting us both input and
          output data size of Medbravo Hadoop MapReduce Jobs. Our current input for
          big data processing is now 8.5 GB and in the future it would grow bigger.
          Our output is larger as it is composed of input data plus additional
          information.
          Therefore we are asking you to increase our quota in HDFS. We need *at
          least 30 GB* to safely store input + output + possible intermediate data
          during processing.

          We will really appreciate your favourable response,

          We wish you a Merry Christmas and a Prosperous New Year,
          Aurelia

          Aurelia Bustos MD
          Cofounder

          tel: (+34) 618 453 214
          www.medbravo.org

          Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
          Please, send your messages using the new domain (Fiware-fiche-coaching@lists.fiware.org) instead of the old one.
          _______________________________________________
          Fiware-fiche-coaching mailing list
          Fiware-fiche-coaching@lists.fiware.org
          https://lists.fiware.org/listinfo/fiware-fiche-coaching
          [Created via e-mail received from: Aurelia Bustos <aurelia@medbravo.org>]

          Show
          frb Francisco Romero added a comment - Dear All, Medbravo is continuously processing big-data on clinical research on cancer. In order to accommodate the increasing Medbravo data processing size in the resources provided in COSMOS Global Instance we would kindly ask you to for a larger quota in HDFS. Specifically *our current quota of 5 GB *is limiting us both input and output data size of Medbravo Hadoop MapReduce Jobs. Our current input for big data processing is now 8.5 GB and in the future it would grow bigger. Our output is larger as it is composed of input data plus additional information. Therefore we are asking you to increase our quota in HDFS. We need *at least 30 GB* to safely store input + output + possible intermediate data during processing. We will really appreciate your favourable response, We wish you a Merry Christmas and a Prosperous New Year, Aurelia – Aurelia Bustos MD Cofounder tel: (+34) 618 453 214 www.medbravo.org Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost. Please, send your messages using the new domain (Fiware-fiche-coaching@lists.fiware.org) instead of the old one. _______________________________________________ Fiware-fiche-coaching mailing list Fiware-fiche-coaching@lists.fiware.org https://lists.fiware.org/listinfo/fiware-fiche-coaching [Created via e-mail received from: Aurelia Bustos <aurelia@medbravo.org>]
          Hide
          fw.ext.user FW External User added a comment -

          Dear Aurelia,

          The global instance of Cosmos in FIWARE Lab is an environment addressed
          for testing and pre-production. It is relatively a small cluster and it is
          shared among several FIWARE users (in the sense the full HDFS space is
          divided into private namespaces). That¹s why the quota is limited to 5 GB.

          We could increase the quota a little bit, let¹s say 10 GB, but the cluster
          is pretty close to the 100% of storage capacity. Let me know if 10GB is
          enough for the moment, and let think what we can do for the future.

          Regards,
          Francisco

          ________________________________

          Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

          The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

          Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
          Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
          Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one.
          _______________________________________________
          Fiware-lab-help mailing list
          Fiware-lab-help@lists.fiware.org
          https://lists.fiware.org/listinfo/fiware-lab-help

          Show
          fw.ext.user FW External User added a comment - Dear Aurelia, The global instance of Cosmos in FIWARE Lab is an environment addressed for testing and pre-production. It is relatively a small cluster and it is shared among several FIWARE users (in the sense the full HDFS space is divided into private namespaces). That¹s why the quota is limited to 5 GB. We could increase the quota a little bit, let¹s say 10 GB, but the cluster is pretty close to the 100% of storage capacity. Let me know if 10GB is enough for the moment, and let think what we can do for the future. Regards, Francisco ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost. Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one. _______________________________________________ Fiware-lab-help mailing list Fiware-lab-help@lists.fiware.org https://lists.fiware.org/listinfo/fiware-lab-help
          Hide
          fw.ext.user FW External User added a comment -

          Dear Francisco,

          Thank you for the information. Yes, we would really appreciate if you
          increase our quota to 10 GB. Although this would still impose limits to us

          • for instance using HIVE - we can find a solution by modifying our design
            to split the processing load.

          As you say - the cluster is close to 100% capacity. When we were testing
          software last month it failed some times because both lack of space on hdfs
          but mostly because lack of storage for data between map and reduce on
          processing nodes.

          Bests,
          Aurelia

          On Fri, Jan 8, 2016 at 8:29 AM, FRANCISCO ROMERO BUENO <

          Aurelia Bustos MD
          Cofounder

          tel: (+34) 618 453 214
          www.medbravo.org

          Sent with MailTrack
          <https://mailtrack.io/install?source=signature&lang=en&referral=aureliabustos@gmail.com&idSignature=22>

          Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
          Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one.
          _______________________________________________
          Fiware-lab-help mailing list
          Fiware-lab-help@lists.fiware.org
          https://lists.fiware.org/listinfo/fiware-lab-help

          Show
          fw.ext.user FW External User added a comment - Dear Francisco, Thank you for the information. Yes, we would really appreciate if you increase our quota to 10 GB. Although this would still impose limits to us for instance using HIVE - we can find a solution by modifying our design to split the processing load. As you say - the cluster is close to 100% capacity. When we were testing software last month it failed some times because both lack of space on hdfs but mostly because lack of storage for data between map and reduce on processing nodes. Bests, Aurelia On Fri, Jan 8, 2016 at 8:29 AM, FRANCISCO ROMERO BUENO < – Aurelia Bustos MD Cofounder tel: (+34) 618 453 214 www.medbravo.org Sent with MailTrack < https://mailtrack.io/install?source=signature&lang=en&referral=aureliabustos@gmail.com&idSignature=22 > Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost. Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one. _______________________________________________ Fiware-lab-help mailing list Fiware-lab-help@lists.fiware.org https://lists.fiware.org/listinfo/fiware-lab-help
          Hide
          flegall Franck Le Gall added a comment -

          Dear Francisco

          Can this issue be processed and close ?

          Many thanks
          Franck

          Show
          flegall Franck Le Gall added a comment - Dear Francisco Can this issue be processed and close ? Many thanks Franck
          Hide
          fw.ext.user FW External User added a comment -

          Hi Franck,

          This issue is related to this other one
          https://jira.fiware.org/browse/HELP-5555, for which I'm still waiting for
          new feedback.

          Regards,
          Francisco

          El 13/1/16 15:14, "Franck Le Gall (JIRA)" <jira-help-desk@fi-ware.org>

          ________________________________

          Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

          The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

          Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição

          Show
          fw.ext.user FW External User added a comment - Hi Franck, This issue is related to this other one https://jira.fiware.org/browse/HELP-5555 , for which I'm still waiting for new feedback. Regards, Francisco El 13/1/16 15:14, "Franck Le Gall (JIRA)" <jira-help-desk@fi-ware.org> ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
          Hide
          frb Francisco Romero added a comment -

          Quota increased, closing.

          Show
          frb Francisco Romero added a comment - Quota increased, closing.

            People

            • Assignee:
              frb Francisco Romero
              Reporter:
              fw.ext.user FW External User
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: