Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-5850

FIWARE.Request.Lab.Budapest.Re: Information object storare

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-LAB-HELP
    • Labels:
      None
    • HD-Node:
      Budapest

      Description

      From: Massimo Corna massimo.corna@krill.eu
      Sent: Tuesday, February 09, 2016 12:42 PM
      To: Chulani, Ilknur
      Subject: Information object storare

      Good morning

      Sorry to trouble, however, we have a problem of slowness with object storage. Our application provides you load of more than 20 Mb file the problem is just in the time it takes to load them in the object storage. From the tests of a 18 MB file the average load time is 7 minutes.
      I wanted to know if it is due to the Object storage on budapest node and because fiware lab is a development platform, because for testing we can also expect these times but in production these times are impossible for our application.
      I do not know if you can help me and if I was clear, waiting for an answer thank.

      Massimo Corna


      http://www.e-levelcom.com/elevel/krill.png
      Massimo Corna
      Developer

      Krill Srl
      Via Casale, 5
      20144 Milano (MI)
      Tel: +39 02 8907 5413
      Fax: +39 02 3206 6978
      Mob: +39 393 911 3793
      Skype: massimo.corna
      http://www.<http://www.e-levelcom.com/>krill.eu<http://krill.eu/>

      ______________________________________________
      Fiware-fiadopt-coaching mailing list
      Fiware-fiadopt-coaching@lists.fiware.org
      https://lists.fiware.org/listinfo/fiware-fiadopt-coaching

        Issue Links

          Activity

          Hide
          burak Karaboga, Burak added a comment -

          Hi Massimo,

          No problem, we are here to help.

          We will be looking forward for your updates.

          Regards,

          Burak

          Show
          burak Karaboga, Burak added a comment - Hi Massimo, No problem, we are here to help. We will be looking forward for your updates. Regards, Burak
          Hide
          ichulani ilknur chulani added a comment -

          Dear Massimo,

          Were you able produce a test case?

          We will close the ticket if we do not hear back from you until next Tuesday.

          Kind regards,

          ilknur

          Show
          ichulani ilknur chulani added a comment - Dear Massimo, Were you able produce a test case? We will close the ticket if we do not hear back from you until next Tuesday. Kind regards, ilknur
          Hide
          fw.ext.user FW External User added a comment -

          Good morning
          I apologize for the delay but I'm very busy on the various projects. I did
          a further test using the code you find below developed with Node Js
          technology. I loaded in object storage a 1.6 MB file in 36 seconds and a
          4.1 MB file in 1 minute and 40 seconds.

          It seems to me than when I had made test ,the situation has greatly
          improved.

          This is the code that I used for testing

          function uploadobject(usr, dir, filename, response) {
          var init = new Date().getTime();
          console.log("Start");
          var body = JSON.stringify({
          auth: {
          tenantId: tenantId,
          passwordCredentials:

          {username: user, password: psw}

          },
          });
          var options = {
          hostname: 'cloud.lab.fi-ware.org',
          port: 4730,
          path: '/v2.0/tokens',
          method: 'POST',
          headers:

          { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Content-length': body.length }

          };
          var req2 = http.request(options, function (res2) {
          var data = '';
          res2.setEncoding('utf8');
          res2.on('data', function (chunk)

          { data += chunk; }

          );
          res2.on('end', function () {
          if (res2.statusCode == 200) {
          var resu = JSON.parse(data);
          var token = resu.access.token.id;
          //upload file to objectstorage
          var file = fs.createReadStream(dir + '/' + filename)
          .pipe(request.put({
          url: '
          http://148.6.80.5:8080/v1/AUTH_6b5e128ba10b47cc905b1ad66fd01c4f/' + usr +
          "/" + filename,
          headers:

          {'Content-Type': 'application/zip', 'X-Auth-Token': token}

          }, function (err, res, body) {
          if (err)

          { logger.error("routesobj. Function uploadobject error " + err); //console.log('error', err); }

          else {
          //console.log('status', res.statusCode);
          if (res.statusCode === 201) {
          console.log("Finish");
          var finit = new Date().getTime();
          console.log(finit-init);
          //elimino il file zip nella directory
          temporanea
          fs.unlink(dir + '/' + filename,
          function ()

          { response.redirect('/objlist'); }

          );
          } else

          { logger.error("routesobj. Function uploadobject return res code from fiware " + res.statusCode); }

          }
          }));

          } else

          { logger.error("routesobj. Function uploadobject return res2 code from fiware " + res2.statusCode); //console.log("Error"); }

          });

          });
          req2.write(body);
          req2.end();
          }

          For me now, the situation would be fine.

          Thank you so much for your time and help

          Massimo Corna

          Massimo Corna
          Developer

          Krill Srl
          Via Copernico, 38
          20125 Milano
          Tel: +39 02 8907 5413
          Fax: +39 02 3206 6978
          Mob: +39 393 911 3793
          Skype: massimo.corna
          http://www. <http://www.e-levelcom.com/>krill.eu

          Please consider the environment before printing this e-mail.
          This email and any attachments are confidential to the intended recipient
          and may also be privileged. If you are not the intended recipient please
          delete it from your system and notify the sender. You should not copy it or
          use it for any purpose nor disclose or distribute its contents to any other
          person. Any improper use of this email and its attachments will lead to the
          person responsible having to deal with all the associated civil and
          criminal consequences of their actions (Legislative Decree 196/03 and
          Directive 2002/58/EC). The sender shall not be liable for the improper or
          incomplete transmission of the information contained in this communication
          nor for any delay in its receipt or damage to your system. The sender does
          not guarantee that the integrity of this communication has been maintained
          nor that this communication is free of viruses, interceptions or
          interference.

          Show
          fw.ext.user FW External User added a comment - Good morning I apologize for the delay but I'm very busy on the various projects. I did a further test using the code you find below developed with Node Js technology. I loaded in object storage a 1.6 MB file in 36 seconds and a 4.1 MB file in 1 minute and 40 seconds. It seems to me than when I had made test ,the situation has greatly improved. This is the code that I used for testing function uploadobject(usr, dir, filename, response) { var init = new Date().getTime(); console.log("Start"); var body = JSON.stringify({ auth: { tenantId: tenantId, passwordCredentials: {username: user, password: psw} }, }); var options = { hostname: 'cloud.lab.fi-ware.org', port: 4730, path: '/v2.0/tokens', method: 'POST', headers: { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Content-length': body.length } }; var req2 = http.request(options, function (res2) { var data = ''; res2.setEncoding('utf8'); res2.on('data', function (chunk) { data += chunk; } ); res2.on('end', function () { if (res2.statusCode == 200) { var resu = JSON.parse(data); var token = resu.access.token.id; //upload file to objectstorage var file = fs.createReadStream(dir + '/' + filename) .pipe(request.put({ url: ' http://148.6.80.5:8080/v1/AUTH_6b5e128ba10b47cc905b1ad66fd01c4f/ ' + usr + "/" + filename, headers: {'Content-Type': 'application/zip', 'X-Auth-Token': token} }, function (err, res, body) { if (err) { logger.error("routesobj. Function uploadobject error " + err); //console.log('error', err); } else { //console.log('status', res.statusCode); if (res.statusCode === 201) { console.log("Finish"); var finit = new Date().getTime(); console.log(finit-init); //elimino il file zip nella directory temporanea fs.unlink(dir + '/' + filename, function () { response.redirect('/objlist'); } ); } else { logger.error("routesobj. Function uploadobject return res code from fiware " + res.statusCode); } } })); } else { logger.error("routesobj. Function uploadobject return res2 code from fiware " + res2.statusCode); //console.log("Error"); } }); }); req2.write(body); req2.end(); } For me now, the situation would be fine. Thank you so much for your time and help Massimo Corna – Massimo Corna Developer Krill Srl Via Copernico, 38 20125 Milano Tel: +39 02 8907 5413 Fax: +39 02 3206 6978 Mob: +39 393 911 3793 Skype: massimo.corna http://www . < http://www.e-levelcom.com/ >krill.eu Please consider the environment before printing this e-mail. This email and any attachments are confidential to the intended recipient and may also be privileged. If you are not the intended recipient please delete it from your system and notify the sender. You should not copy it or use it for any purpose nor disclose or distribute its contents to any other person. Any improper use of this email and its attachments will lead to the person responsible having to deal with all the associated civil and criminal consequences of their actions (Legislative Decree 196/03 and Directive 2002/58/EC). The sender shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system. The sender does not guarantee that the integrity of this communication has been maintained nor that this communication is free of viruses, interceptions or interference.
          Hide
          ichulani ilknur chulani added a comment -

          Hi Massimo,

          Thank you for the test case. I am not very clear about the next steps though. Do you mean to say you are ok now with the performance of the Object storage? Should we close the ticket? or do you still need further investigation?

          Show
          ichulani ilknur chulani added a comment - Hi Massimo, Thank you for the test case. I am not very clear about the next steps though. Do you mean to say you are ok now with the performance of the Object storage? Should we close the ticket? or do you still need further investigation?
          Hide
          fw.ext.user FW External User added a comment -

          Thankyou for me now is ok for the performance of the Object storage you
          close the ticket.

          Thankyou very much

          Bye

          Ing.Massimo Corna

          Massimo Corna
          Developer

          Krill Srl
          Via Copernico, 38
          20125 Milano
          Tel: +39 02 8907 5413
          Fax: +39 02 3206 6978
          Mob: +39 393 911 3793
          Skype: massimo.corna
          http://www. <http://www.e-levelcom.com/>krill.eu

          Please consider the environment before printing this e-mail.
          This email and any attachments are confidential to the intended recipient
          and may also be privileged. If you are not the intended recipient please
          delete it from your system and notify the sender. You should not copy it or
          use it for any purpose nor disclose or distribute its contents to any other
          person. Any improper use of this email and its attachments will lead to the
          person responsible having to deal with all the associated civil and
          criminal consequences of their actions (Legislative Decree 196/03 and
          Directive 2002/58/EC). The sender shall not be liable for the improper or
          incomplete transmission of the information contained in this communication
          nor for any delay in its receipt or damage to your system. The sender does
          not guarantee that the integrity of this communication has been maintained
          nor that this communication is free of viruses, interceptions or
          interference.

          Show
          fw.ext.user FW External User added a comment - Thankyou for me now is ok for the performance of the Object storage you close the ticket. Thankyou very much Bye Ing.Massimo Corna – Massimo Corna Developer Krill Srl Via Copernico, 38 20125 Milano Tel: +39 02 8907 5413 Fax: +39 02 3206 6978 Mob: +39 393 911 3793 Skype: massimo.corna http://www . < http://www.e-levelcom.com/ >krill.eu Please consider the environment before printing this e-mail. This email and any attachments are confidential to the intended recipient and may also be privileged. If you are not the intended recipient please delete it from your system and notify the sender. You should not copy it or use it for any purpose nor disclose or distribute its contents to any other person. Any improper use of this email and its attachments will lead to the person responsible having to deal with all the associated civil and criminal consequences of their actions (Legislative Decree 196/03 and Directive 2002/58/EC). The sender shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt or damage to your system. The sender does not guarantee that the integrity of this communication has been maintained nor that this communication is free of viruses, interceptions or interference.

            People

            • Assignee:
              WIGNER Budapest Node Helpdesk
              Reporter:
              ichulani ilknur chulani
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: