Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-2296

FIWARE.Request.Lab.Budapest.Connectivity problems in some regions cause Kurento to fail.

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-LAB-HELP
    • Labels:
      None
    • HD-Node:
      Budapest

      Description

      We have been receiving several reports of Kurento (Stream Oriented GEri) failing to communicate in the last few weeks. After digging into the problem we have found out the cause, which is described bellow:

      Some FIWARE Lab regions are having strange connectivy behavior. In particular, we have been able to measure it in the Budapest instance. This behaviour makes the network to work in "burst", so that when an instance wants to get traffic towards the internet, the first packets take a very long delay to get out (around 6 seconds). You can reproduce this with a single ping.

      This delay is not compatible with the real-time communication protocols of Kurento, which expect sub-second latencies. Due to this, we are having many timeouts to fire and Kurento not being able to establish communication with the external world.

      We have detected what is causing your problem. The point is libnice (the library implementing the STUN protocol) has a timer of 1.8 seconds for gathering all the STUN candidates. If candidates don't arrive in that time, they are discarded. We have detected FIWARE Lab instances in many regions requiring more than 6 to enable connectivity (for some reason, connectivity works in bursts). With this type of behavior of the network you will not be able to make work any kind of application requiring real-time. I recommend you to follow this procedure:

      1. Ping from your instante to the IP of your STUN server. If the first ping takes more than 1.8 seconds, then you are hitting this problem.
      2. Try to move to another region not having such connectivity problems. For example, we are testing in Spain and we are successfully executing Kurento.

      I will fire a Jira issue for reporting about this problem.

      Best and thank you for your patience.

        Activity

        Hide
        mev Manuel Escriche added a comment -

        Hi Luis,
        I'd appreciate knowing what regions made you experience the problems in order to create the corresponding issues.
        Thanks

        Show
        mev Manuel Escriche added a comment - Hi Luis, I'd appreciate knowing what regions made you experience the problems in order to create the corresponding issues. Thanks
        Hide
        llopez NaevaTeC Development Team added a comment -

        Problems were detected in the Budapest instance.

        Show
        llopez NaevaTeC Development Team added a comment - Problems were detected in the Budapest instance.
        Hide
        WIGNER Budapest Node Helpdesk added a comment -

        We have tested it yesterday, and also experienced large packet drops at the beginning of the communication. We started investigating the issue. You will be informed about the results...

        Best,
        Sandor

        Show
        WIGNER Budapest Node Helpdesk added a comment - We have tested it yesterday, and also experienced large packet drops at the beginning of the communication. We started investigating the issue. You will be informed about the results... Best, Sandor
        Hide
        sandor Sándor Laki added a comment -

        problem dissapeared in Budapest2. It was related to OVS. Somehow table lookup took significant amount of time.

        Show
        sandor Sándor Laki added a comment - problem dissapeared in Budapest2. It was related to OVS. Somehow table lookup took significant amount of time.

          People

          • Assignee:
            WIGNER Budapest Node Helpdesk
            Reporter:
            llopez NaevaTeC Development Team
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: