Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-5366

FIWARE.Request.Tech.Apps.ApplicationMashup.ProblemAccessingOrionUsingHTTPS

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-TECH-HELP
    • Labels:
      None

      Description

      Hi,

      I haven't used my mashup application in the last few weeks, but now my
      NGSI query operator stopped working. After some digging and debugging,
      it seems like the NGSI connection is passing the url of my context
      broker through this url "https://mashup.lab.fiware.org/proxy/<cb_url>"
      and here seems to be the problem. The context broker itself is working
      fine when used from other applications.

      The same problem seems to occur, when using the NGSI browser widget from
      the marketplace.

      What do I have to do to fix this?

      My source looks like this:

      this.ngsi_server = MashupPlatform.prefs.get('ngsi_server');
      this.ngsi_proxy = MashupPlatform.prefs.get('ngsi_proxy');

      var request_headers = {};

      if (MashupPlatform.prefs.get('use_owner_credentials'))

      { request_headers['X-FI-WARE-OAuth-Token'] = 'true'; request_headers['X-FI-WARE-OAuth-Header-Name'] = 'X-Auth-Token'; request_headers['x-FI-WARE-OAuth-Source'] = 'workspaceowner'; }

      this.connection = new NGSI.Connection(this.ngsi_server,

      { use_user_fiware_token: MashupPlatform.prefs.get('use_user_fiware_token'), request_headers: request_headers, ngsi_proxy_url: this.ngsi_proxy }

      );

      var types = MashupPlatform.prefs.get('ngsi_entities').split(new
      RegExp(',
      s*'));

      var entityIdList = [];
      var attributeList = [];
      var entityId;
      for (var i = 0; i < types.length; i++) {
      entityId =

      { id: '.*', type: types[i], isPattern: true }

      ;
      entityIdList.push(entityId);
      }

      var options = {
      flat: true,
      limit: 1000,
      offset: 0,
      details: true,
      onSuccess: function (data, details) {

      for (var entityId in data)

      { MashupPlatform.wiring.pushEvent("entityOutput", JSON.stringify(data[entityId])); }

      if (details.count > options.offset + options.limit)

      { options.offset += options.limit; this.connection.query(entityIdList, attributeList, options); }

      }.bind(this),
      onFailure: function (e)

      { MashupPlatform.operator.log("Error querying entities from context broker server: " + e.message); }

      };
      this.connection.query(entityIdList, attributeList, options);


      Artur Löwen
      CEO
      Gridhound UG (haftungsbeschränkt)
      +49 179 1449189
      aloewen@gridhound.de
      www.gridhound.de
      Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
      Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one.
      _______________________________________________
      Fiware-lab-help mailing list
      Fiware-lab-help@lists.fiware.org
      https://lists.fiware.org/listinfo/fiware-lab-help
      [Created via e-mail received from: =?UTF-8?Q?Artur_L=C3=B6wen?= <aloewen@gridhound.de>]

        Activity

        Hide
        fermin Fermín Galán added a comment -

        I have been added to this thread. However, after the

        thank you, it is working now.

        I understand that the issue is solved and no input from side is need. However, if I'm wrong, please ask whatever Orion specific question using StackOverflow ("fiware-orion" tag). Thanks!

        Show
        fermin Fermín Galán added a comment - I have been added to this thread. However, after the thank you, it is working now. I understand that the issue is solved and no input from side is need. However, if I'm wrong, please ask whatever Orion specific question using StackOverflow ("fiware-orion" tag). Thanks!
        Hide
        aarranz Álvaro Arranz added a comment -

        Álvaro,
        thank you, it is working now.

        Great

        But does this mean, you would have to add every intermediate certificate
        now, to get it to work? Let's say I would try to use Let's Encrypt or
        any other of the bigger providers. This problem would raise with every
        one of them, right?

        Hmm, yes and no. I have added the intermediate certificate into the Mashup portal to let you continue working without having to wait more time, but https servers MUST serve the full certificate chain, so the correct fix for this problem is configuring your orion context broker to provide the full certificate chain allowing everybody to trust your server (see this link for more details). Probably, the orion context broker needs to be modified for adding support for providing the certificate chain, I don't know because I don't work on the orion context broker (that was the point of adding Fermin to CC). Another option is to deploy your context broker using a front-end web server supporting the full https stack (e.g. Apache, nginx, ...)

        So every time, someone tries to use it, they run
        with their head against the wall, with no meaningful error for the
        source of the problem. And it takes quite some time to sort it out, as
        in my case, until the error is found and the correct certificate is
        added to the Mashup Portal.

        Well, you have been the firsts with those problems. So thanks for being our testers , and sorry for the waste of time .

        Anyway, we have improved the response from the WireCloud proxy, that now provides a "better" error message:

        {"description": "SSL Error", "details": "(\"bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)\",)"}

        It's not perfect, but is easier now to understand that the certificate is not valid. Also take into account that the error description comes directly from python, as the SSL connection can fail for other reasons.

        It's truth that after this modification we can provide a better error message for such cases in the "NGSI Source" operator, so I have created a ticket for improving this aspect .

        Do you have any other suggestion for improving this scenario for the next ones?

        Best regards
        Artur

        Best regards,
        Álvaro

        Show
        aarranz Álvaro Arranz added a comment - Álvaro, thank you, it is working now. Great But does this mean, you would have to add every intermediate certificate now, to get it to work? Let's say I would try to use Let's Encrypt or any other of the bigger providers. This problem would raise with every one of them, right? Hmm, yes and no. I have added the intermediate certificate into the Mashup portal to let you continue working without having to wait more time, but https servers MUST serve the full certificate chain, so the correct fix for this problem is configuring your orion context broker to provide the full certificate chain allowing everybody to trust your server (see this link for more details). Probably, the orion context broker needs to be modified for adding support for providing the certificate chain, I don't know because I don't work on the orion context broker (that was the point of adding Fermin to CC). Another option is to deploy your context broker using a front-end web server supporting the full https stack (e.g. Apache, nginx, ...) So every time, someone tries to use it, they run with their head against the wall, with no meaningful error for the source of the problem. And it takes quite some time to sort it out, as in my case, until the error is found and the correct certificate is added to the Mashup Portal. Well, you have been the firsts with those problems. So thanks for being our testers , and sorry for the waste of time . Anyway, we have improved the response from the WireCloud proxy, that now provides a "better" error message: {"description": "SSL Error", "details": "(\"bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)\",)"} It's not perfect, but is easier now to understand that the certificate is not valid. Also take into account that the error description comes directly from python, as the SSL connection can fail for other reasons. It's truth that after this modification we can provide a better error message for such cases in the "NGSI Source" operator, so I have created a ticket for improving this aspect . Do you have any other suggestion for improving this scenario for the next ones? Best regards Artur Best regards, Álvaro
        Hide
        fw.ext.user FW External User added a comment -

        Álvaro,

        thank you, it is working now.

        But does this mean, you would have to add every intermediate certificate
        now, to get it to work? Let's say I would try to use Let's Encrypt or
        any other of the bigger providers. This problem would raise with every
        one of them, right? So every time, someone tries to use it, they run
        with their head against the wall, with no meaningful error for the
        source of the problem. And it takes quite some time to sort it out, as
        in my case, until the error is found and the correct certificate is
        added to the Mashup Portal.

        Best regards

        Artur

        Show
        fw.ext.user FW External User added a comment - Álvaro, thank you, it is working now. But does this mean, you would have to add every intermediate certificate now, to get it to work? Let's say I would try to use Let's Encrypt or any other of the bigger providers. This problem would raise with every one of them, right? So every time, someone tries to use it, they run with their head against the wall, with no meaningful error for the source of the problem. And it takes quite some time to sort it out, as in my case, until the error is found and the correct certificate is added to the Mashup Portal. Best regards Artur
        Hide
        aarranz Álvaro Arranz added a comment -

        Hi Artur,

        I have added the certificate for "COMODO RSA Domain Validation Secure Server CA" into the Mashup portal, so you dashboard should be working now.

        Is there a way, that enables my server, the Orion context broker, to
        provide the full chain? If that is the problem. I tried to use the file
        with the full chain, but Orion wouldn't load it. The loaded file is
        limited by Orion to 2048 Bytes, which doesn't allow more than one
        certificate in a file

        I have added Fermin to CC because I don't know if it's possible to configure orion to provide the full certificate chain.

        Best regards.
        Álvaro

        Show
        aarranz Álvaro Arranz added a comment - Hi Artur, I have added the certificate for "COMODO RSA Domain Validation Secure Server CA" into the Mashup portal, so you dashboard should be working now. Is there a way, that enables my server, the Orion context broker, to provide the full chain? If that is the problem. I tried to use the file with the full chain, but Orion wouldn't load it. The loaded file is limited by Orion to 2048 Bytes, which doesn't allow more than one certificate in a file I have added Fermin to CC because I don't know if it's possible to configure orion to provide the full certificate chain. Best regards. Álvaro
        Hide
        fw.ext.user FW External User added a comment -

        Álvaro,

        sadly my mashup still fails to use the NGSI Query Operator to collect
        data from the Orion context broker using the address
        https://fiware.gridhound.de:1026/
        The error is the same as before: 01/27/16 05:15:11 PM Error querying
        entities from context broker server: Connection Error

        Is there a way, that enables my server, the Orion context broker, to
        provide the full chain? If that is the problem. I tried to use the file
        with the full chain, but Orion wouldn't load it. The loaded file is
        limited by Orion to 2048 Bytes, which doesn't allow more than one
        certificate in a file.

        Cheers

        Artur

        Show
        fw.ext.user FW External User added a comment - Álvaro, sadly my mashup still fails to use the NGSI Query Operator to collect data from the Orion context broker using the address https://fiware.gridhound.de:1026/ The error is the same as before: 01/27/16 05:15:11 PM Error querying entities from context broker server: Connection Error Is there a way, that enables my server, the Orion context broker, to provide the full chain? If that is the problem. I tried to use the file with the full chain, but Orion wouldn't load it. The loaded file is limited by Orion to 2048 Bytes, which doesn't allow more than one certificate in a file. Cheers Artur
        Hide
        aarranz Álvaro Arranz added a comment -

        The problem seems to be that your server is not providing the full certificate chain (See this link for more info). I have tested adding your intermediate CA (COMODO RSA Domain Validation Secure Server CA) to the trusted certificates and it works , so I think that there are nothing more to fix .

        Best regards,
        Álvaro

        Show
        aarranz Álvaro Arranz added a comment - The problem seems to be that your server is not providing the full certificate chain (See this link for more info). I have tested adding your intermediate CA (COMODO RSA Domain Validation Secure Server CA) to the trusted certificates and it works , so I think that there are nothing more to fix . Best regards, Álvaro
        Hide
        fw.ext.user FW External User added a comment -

        Álvaro,

        I did check my configuration, although the suggested site doesn't work
        with port 1026 for the context broker. Other checking tools like
        https://sslanalyzer.comodoca.com/ recognize the certificate but complain
        about a missing local issuer certificate. But please check for your self
        and tell me, what I can do.

        You can find my context broker at fiware.gridhound.de:1026.

        Thanks

        Artur

        Show
        fw.ext.user FW External User added a comment - Álvaro, I did check my configuration, although the suggested site doesn't work with port 1026 for the context broker. Other checking tools like https://sslanalyzer.comodoca.com/ recognize the certificate but complain about a missing local issuer certificate. But please check for your self and tell me, what I can do. You can find my context broker at fiware.gridhound.de:1026. Thanks Artur
        Hide
        aarranz Álvaro Arranz added a comment -

        Hi Artur,

        You can check your https configuration using the following link . Anyway, if you provide me your DNS, I can make some tests from the Mashup portal to see if there is any other problem.

        Best Regards,
        Álvaro

        Show
        aarranz Álvaro Arranz added a comment - Hi Artur, You can check your https configuration using the following link . Anyway, if you provide me your DNS, I can make some tests from the Mashup portal to see if there is any other problem. Best Regards, Álvaro
        Hide
        fw.ext.user FW External User added a comment -

        Hello Álvaro,

        sadly I wasn't able to fix the problem. I created a DNS entry for my
        server IP, as SSL Certificates are not possible for an IP and created an
        SSL certificate for that entry from Let's Encrypt as well as comodo.com,
        but neither of them solve the problem. I still get a connection error.
        As I have no way to figure out what the exact problem is, I kindly ask
        for further help to identify the issue.

        Best regards,

        Artur

        Show
        fw.ext.user FW External User added a comment - Hello Álvaro, sadly I wasn't able to fix the problem. I created a DNS entry for my server IP, as SSL Certificates are not possible for an IP and created an SSL certificate for that entry from Let's Encrypt as well as comodo.com, but neither of them solve the problem. I still get a connection error. As I have no way to figure out what the exact problem is, I kindly ask for further help to identify the issue. Best regards, Artur
        Hide
        aarranz Álvaro Arranz added a comment -

        Hi Artur,

        I'm going to close this ticket as we have identified the problem.

        Anyway, If you don't find how to obtain a valid HTTPS certificate, we can also add your self-signed certificated into the thrusted cert lists used by the Mashup portal server. But take into account that your current cert was generated for localhost so it won't be valid for accessing your server using the IP address: 130.206.112.223. Don't hesitate to contact me again if you create another self-signed cert for 130.206.112.223 and you want me to add it to the trusted list of certificates .

        Best regards,
        Álvaro

        Show
        aarranz Álvaro Arranz added a comment - Hi Artur, I'm going to close this ticket as we have identified the problem. Anyway, If you don't find how to obtain a valid HTTPS certificate, we can also add your self-signed certificated into the thrusted cert lists used by the Mashup portal server. But take into account that your current cert was generated for localhost so it won't be valid for accessing your server using the IP address: 130.206.112.223. Don't hesitate to contact me again if you create another self-signed cert for 130.206.112.223 and you want me to add it to the trusted list of certificates . Best regards, Álvaro
        Hide
        fw.ext.user FW External User added a comment -

        Hi Álvaro,

        thank you for your hint. I will try it and report back.

        Cheers

        Artur


        Artur Löwen
        Geschäftsführer
        Gridhound UG (haftungsbeschränkt)
        +49 179 1449189
        aloewen@gridhound.de
        www.gridhound.de

        Show
        fw.ext.user FW External User added a comment - Hi Álvaro, thank you for your hint. I will try it and report back. Cheers Artur – Artur Löwen Geschäftsführer Gridhound UG (haftungsbeschränkt) +49 179 1449189 aloewen@gridhound.de www.gridhound.de
        Hide
        aarranz Álvaro Arranz added a comment -

        Hi Artur,

        the error seems related to the use of a self-signed ssl certificated in your orion context broker instance. I recommend you to give a try to "Let’s Encrypt " for obtaining valid ssl certificates.

        Best regards,
        Álvaro

        Show
        aarranz Álvaro Arranz added a comment - Hi Artur, the error seems related to the use of a self-signed ssl certificated in your orion context broker instance. I recommend you to give a try to " Let’s Encrypt " for obtaining valid ssl certificates. Best regards, Álvaro
        Hide
        mev Manuel Escriche added a comment -

        Dear Artur, Dear Alvaro,

        I don't see any progress for the last 10 days.

        Artur, I kind of understand Alvaro is awaiting some input from you. Is that possible? Thanks.

        Kind regards,
        Manuel

        Show
        mev Manuel Escriche added a comment - Dear Artur, Dear Alvaro, I don't see any progress for the last 10 days. Artur, I kind of understand Alvaro is awaiting some input from you. Is that possible? Thanks. Kind regards, Manuel
        Hide
        aarranz Álvaro Arranz added a comment -

        Dear Artur,

        I was researching and I was unable to find the problem because I don't have enough information. WireCloud sends an email with info about the detected errors, but this mechanism is currently not working when the error is raised while processing a proxy request . I'm going to fix that but in the mean time, could you send me the information (privately to aarranz <at> conwet.com) about the request that is failing?

        Best regards,
        Álvaro

        Show
        aarranz Álvaro Arranz added a comment - Dear Artur, I was researching and I was unable to find the problem because I don't have enough information. WireCloud sends an email with info about the detected errors, but this mechanism is currently not working when the error is raised while processing a proxy request . I'm going to fix that but in the mean time, could you send me the information (privately to aarranz <at> conwet.com) about the request that is failing? Best regards, Álvaro
        Hide
        fw.ext.user FW External User added a comment -

        Dear Álvaro,

        it was the first thing I did, I used the context broker without the PEP
        proxy.
        In my original situation I had the mashup workspace URL registered as an
        application in the identy manager. There I also entered the PEP proxy
        username and oauth credentials.
        Now I created a new mashup workspace that is not registered as an
        application, not using the PEP proxy at all, but the problem remains.

        The original URL for the request is
        "https://<ip>:<non-proxy-port>/v1/queryContext" which gets translated by
        the NGSI API to
        "https://mashup.lab.fiware.org/proxy/https/<ip>:<non-proxy-port>/v1/queryContext"
        The body is:
        "{
        "entities": [

        { "id": ".*", "isPattern": "true", "type": "<existing type" }

        ]
        }"
        The error message returned by the request is Status: 500, Internal
        Server Error, "

        {"description": "Error processing proxy request: __str__ returned non-string (type Error)"}

        "

        The same query with the original URL, executed by curl, works fine. But
        as I get no entries in the contextBroker.log about that request, I guess
        it is never received there.

        Regards,

        Artur

        Show
        fw.ext.user FW External User added a comment - Dear Álvaro, it was the first thing I did, I used the context broker without the PEP proxy. In my original situation I had the mashup workspace URL registered as an application in the identy manager. There I also entered the PEP proxy username and oauth credentials. Now I created a new mashup workspace that is not registered as an application, not using the PEP proxy at all, but the problem remains. The original URL for the request is "https://<ip>:<non-proxy-port>/v1/queryContext" which gets translated by the NGSI API to "https://mashup.lab.fiware.org/proxy/https/<ip>:<non-proxy-port>/v1/queryContext" The body is: "{ "entities": [ { "id": ".*", "isPattern": "true", "type": "<existing type" } ] }" The error message returned by the request is Status: 500, Internal Server Error, " {"description": "Error processing proxy request: __str__ returned non-string (type Error)"} " The same query with the original URL, executed by curl, works fine. But as I get no entries in the contextBroker.log about that request, I guess it is never received there. Regards, Artur
        Hide
        aarranz Álvaro Arranz added a comment -

        Dear Artur,

        I don't see any problem with your code and the Mashup platform is working with other context broker instances (e.g. the global orion instance).

        Anyway, there are people having problems with the PEP proxy and the network of the Cloud infrastructures, you can test if this is your problem by making available your context broker without the PEP proxy. See this Stack Overflow question for more details. I've also added Álvaro Alonso to this thread so it can help us with this.

        Best regards,
        Álvaro Arranz (WireCloud's lead developer)

        Show
        aarranz Álvaro Arranz added a comment - Dear Artur, I don't see any problem with your code and the Mashup platform is working with other context broker instances (e.g. the global orion instance). Anyway, there are people having problems with the PEP proxy and the network of the Cloud infrastructures, you can test if this is your problem by making available your context broker without the PEP proxy. See this Stack Overflow question for more details. I've also added Álvaro Alonso to this thread so it can help us with this. Best regards, Álvaro Arranz (WireCloud's lead developer)
        Hide
        gcossu Giuseppe Cossu added a comment -

        Hi,
        I've forwarded your ticket to the Application Mashup GE owner.

        Regards,
        Giuseppe

        Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
        Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one.
        _______________________________________________
        Fiware-lab-help mailing list
        Fiware-lab-help@lists.fiware.org
        https://lists.fiware.org/listinfo/fiware-lab-help

        Show
        gcossu Giuseppe Cossu added a comment - Hi, I've forwarded your ticket to the Application Mashup GE owner. Regards, Giuseppe Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost. Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one. _______________________________________________ Fiware-lab-help mailing list Fiware-lab-help@lists.fiware.org https://lists.fiware.org/listinfo/fiware-lab-help
        Hide
        gcossu Giuseppe Cossu added a comment -

        On Fri, Nov 20, 2015 at 6:23 PM, Artur Löwen <aloewen@gridhound.de> wrote:
        Hi Giuseppe,

        I'm using the FIWARE Lab Mashup instance and my own instances of the Orion Context Broker (v0.21) and PEP Proxy (R4.3).
        As only the mashup instance might have changed, because I didn't update my instances, I checked the changelog of that, but I'm can't see something, that might be my problem.
        I do follow the suggested way for the authentication according to the documentation.

        Can you verify the NGSI browser widget, available from the marketplace, is working for you in the Mashup instance of the FIWARE Lab using https Orion and PEP Proxy instances in a separate VM?
        To me it seems the same problem is happening there, too.

        Thanks

        Artur

        Show
        gcossu Giuseppe Cossu added a comment - On Fri, Nov 20, 2015 at 6:23 PM, Artur Löwen <aloewen@gridhound.de> wrote: Hi Giuseppe, I'm using the FIWARE Lab Mashup instance and my own instances of the Orion Context Broker (v0.21) and PEP Proxy (R4.3). As only the mashup instance might have changed, because I didn't update my instances, I checked the changelog of that, but I'm can't see something, that might be my problem. I do follow the suggested way for the authentication according to the documentation. Can you verify the NGSI browser widget, available from the marketplace, is working for you in the Mashup instance of the FIWARE Lab using https Orion and PEP Proxy instances in a separate VM? To me it seems the same problem is happening there, too. Thanks Artur
        Hide
        gcossu Giuseppe Cossu added a comment -

        Hi,
        are you using some GE or fireware-lab component? If yes, my suggestion is
        to look at the changelogs of the new versions.

        Of course I can't check your code.

        Regards,
        Giuseppe

        Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
        Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one.
        _______________________________________________
        Fiware-lab-help mailing list
        Fiware-lab-help@lists.fiware.org
        https://lists.fiware.org/listinfo/fiware-lab-help

        Show
        gcossu Giuseppe Cossu added a comment - Hi, are you using some GE or fireware-lab component? If yes, my suggestion is to look at the changelogs of the new versions. Of course I can't check your code. Regards, Giuseppe Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost. Please, send your messages using the new domain (Fiware-lab-help@lists.fiware.org) instead of the old one. _______________________________________________ Fiware-lab-help mailing list Fiware-lab-help@lists.fiware.org https://lists.fiware.org/listinfo/fiware-lab-help

          People

          • Assignee:
            aarranz Álvaro Arranz
            Reporter:
            fw.ext.user FW External User
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: