Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-2214

FIWARE.Request.Lab.Budapest.unable to access our machines VIA SSH.

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-LAB-HELP
    • Labels:
      None

      Description

      Hello,

      We are experiencing difficulties in accessing our machines via SSH.
      It is returning a time out "no route to host" error. We tried accessing
      from three different computers using different connections.

      We checked them in FiwareLab interface and they are there, running.
      However, the interface itself took some time to display the machines
      (something that didn't happen before).

      Regards.

      _______________________________________________
      Fiware-creatifi-coaching mailing list
      Fiware-creatifi-coaching@lists.fi-ware.org
      https://lists.fi-ware.org/listinfo/fiware-creatifi-coaching

      [Created via e-mail received from: Xavier Carol Rossell <xavier.carol@i2cat.net>]

        Activity

        Hide
        fw.ext.user FW External User added a comment -

        Hi,

        Developer added more information to this issue.

        This afternoon machines were accessible again, but after doing some
        iptables operations, servers are unavailable again. This is what we've done:

        1. We have one server, lb.alquimia.io, with a floating IP and a public
        internal
        2. We have a couple of backend servers with public internal
        3. We pass on traffic to some ports on the frontend to the backend servers.
        To simulate this behavior, we are redirecting some ssh traffic on the head
        to the ssh on the backends.
        4. This is what we do:

        root@sf-appserver1:~# echo "1" > /proc/sys/net/ipv4/ip_forward
        root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22022 -j
        DNAT --to-destination 10.10.13.102:22
        root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22023 -j
        DNAT --to-destination 10.10.13.105:22
        root@sf-appserver1:~# iptables -t nat -A POSTROUTING -j MASQUERADE

        where 10.10.13.102 and 105 are the backends.

        I have been able to work for five minutes on the servers, but then ssh
        client has became unresponsive, and servers are NOT appearing again on the
        cloud.lab webpage. In fact, NOTHING is being displayed on the cloud.lab
        portal: no images, no floating IPs, no keypairs, no security groups.

        I would say that there is a link between the 'iptables' commands and the
        connectivity problems, but I cannot even imagine what is the problem. I
        have done this 'iptables' trick in other virtualized environments without
        problems :/

        Thanks,

        i-

        2015-02-26 9:21 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>:

        > Hello,
        >
        > We are experiencing difficulties in accessing our machines via SSH.
        > It is returning a time out "no route to host" error. We tried accessing
        > from three different computers using different connections.
        >
        > We checked them in FiwareLab interface and they are there, running.
        > However, the interface itself took some time to display the machines
        > (something that didn't happen before).
        >
        > Regards.
        >
        >
        >

        _______________________________________________
        Fiware-creatifi-coaching mailing list
        Fiware-creatifi-coaching@lists.fi-ware.org
        https://lists.fi-ware.org/listinfo/fiware-creatifi-coaching

        Show
        fw.ext.user FW External User added a comment - Hi, Developer added more information to this issue. This afternoon machines were accessible again, but after doing some iptables operations, servers are unavailable again. This is what we've done: 1. We have one server, lb.alquimia.io, with a floating IP and a public internal 2. We have a couple of backend servers with public internal 3. We pass on traffic to some ports on the frontend to the backend servers. To simulate this behavior, we are redirecting some ssh traffic on the head to the ssh on the backends. 4. This is what we do: root@sf-appserver1:~# echo "1" > /proc/sys/net/ipv4/ip_forward root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22022 -j DNAT --to-destination 10.10.13.102:22 root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22023 -j DNAT --to-destination 10.10.13.105:22 root@sf-appserver1:~# iptables -t nat -A POSTROUTING -j MASQUERADE where 10.10.13.102 and 105 are the backends. I have been able to work for five minutes on the servers, but then ssh client has became unresponsive, and servers are NOT appearing again on the cloud.lab webpage. In fact, NOTHING is being displayed on the cloud.lab portal: no images, no floating IPs, no keypairs, no security groups. I would say that there is a link between the 'iptables' commands and the connectivity problems, but I cannot even imagine what is the problem. I have done this 'iptables' trick in other virtualized environments without problems :/ Thanks, i- 2015-02-26 9:21 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>: > Hello, > > We are experiencing difficulties in accessing our machines via SSH. > It is returning a time out "no route to host" error. We tried accessing > from three different computers using different connections. > > We checked them in FiwareLab interface and they are there, running. > However, the interface itself took some time to display the machines > (something that didn't happen before). > > Regards. > > > _______________________________________________ Fiware-creatifi-coaching mailing list Fiware-creatifi-coaching@lists.fi-ware.org https://lists.fi-ware.org/listinfo/fiware-creatifi-coaching
        Hide
        fw.ext.user FW External User added a comment -

        The developer i s working on the Budapest node.

        KR

        2015-02-27 9:18 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>:

        > Hi,
        >
        > Developer added more information to this issue.
        >
        >
        > This afternoon machines were accessible again, but after doing some
        > iptables operations, servers are unavailable again. This is what we've done:
        >
        > 1. We have one server, lb.alquimia.io, with a floating IP and a public
        > internal
        > 2. We have a couple of backend servers with public internal
        > 3. We pass on traffic to some ports on the frontend to the backend
        > servers. To simulate this behavior, we are redirecting some ssh traffic on
        > the head to the ssh on the backends.
        > 4. This is what we do:
        >
        > root@sf-appserver1:~# echo "1" > /proc/sys/net/ipv4/ip_forward
        > root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22022
        > -j DNAT --to-destination 10.10.13.102:22
        > root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22023
        > -j DNAT --to-destination 10.10.13.105:22
        > root@sf-appserver1:~# iptables -t nat -A POSTROUTING -j MASQUERADE
        >
        > where 10.10.13.102 and 105 are the backends.
        >
        > I have been able to work for five minutes on the servers, but then ssh
        > client has became unresponsive, and servers are NOT appearing again on the
        > cloud.lab webpage. In fact, NOTHING is being displayed on the cloud.lab
        > portal: no images, no floating IPs, no keypairs, no security groups.
        >
        > I would say that there is a link between the 'iptables' commands and the
        > connectivity problems, but I cannot even imagine what is the problem. I
        > have done this 'iptables' trick in other virtualized environments without
        > problems :/
        >
        > Thanks,
        >
        > i-
        >
        >
        >
        >
        > 2015-02-26 9:21 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>:
        >
        >> Hello,
        >>
        >> We are experiencing difficulties in accessing our machines via SSH.
        >> It is returning a time out "no route to host" error. We tried accessing
        >> from three different computers using different connections.
        >>
        >> We checked them in FiwareLab interface and they are there, running.
        >> However, the interface itself took some time to display the machines
        >> (something that didn't happen before).
        >>
        >> Regards.
        >>
        >>
        >>
        >

        _______________________________________________
        Fiware-creatifi-coaching mailing list
        Fiware-creatifi-coaching@lists.fi-ware.org
        https://lists.fi-ware.org/listinfo/fiware-creatifi-coaching

        Show
        fw.ext.user FW External User added a comment - The developer i s working on the Budapest node. KR 2015-02-27 9:18 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>: > Hi, > > Developer added more information to this issue. > > > This afternoon machines were accessible again, but after doing some > iptables operations, servers are unavailable again. This is what we've done: > > 1. We have one server, lb.alquimia.io, with a floating IP and a public > internal > 2. We have a couple of backend servers with public internal > 3. We pass on traffic to some ports on the frontend to the backend > servers. To simulate this behavior, we are redirecting some ssh traffic on > the head to the ssh on the backends. > 4. This is what we do: > > root@sf-appserver1:~# echo "1" > /proc/sys/net/ipv4/ip_forward > root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22022 > -j DNAT --to-destination 10.10.13.102:22 > root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22023 > -j DNAT --to-destination 10.10.13.105:22 > root@sf-appserver1:~# iptables -t nat -A POSTROUTING -j MASQUERADE > > where 10.10.13.102 and 105 are the backends. > > I have been able to work for five minutes on the servers, but then ssh > client has became unresponsive, and servers are NOT appearing again on the > cloud.lab webpage. In fact, NOTHING is being displayed on the cloud.lab > portal: no images, no floating IPs, no keypairs, no security groups. > > I would say that there is a link between the 'iptables' commands and the > connectivity problems, but I cannot even imagine what is the problem. I > have done this 'iptables' trick in other virtualized environments without > problems :/ > > Thanks, > > i- > > > > > 2015-02-26 9:21 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>: > >> Hello, >> >> We are experiencing difficulties in accessing our machines via SSH. >> It is returning a time out "no route to host" error. We tried accessing >> from three different computers using different connections. >> >> We checked them in FiwareLab interface and they are there, running. >> However, the interface itself took some time to display the machines >> (something that didn't happen before). >> >> Regards. >> >> >> > _______________________________________________ Fiware-creatifi-coaching mailing list Fiware-creatifi-coaching@lists.fi-ware.org https://lists.fi-ware.org/listinfo/fiware-creatifi-coaching
        Hide
        fw.ext.user FW External User added a comment -

        Hi,

        The developer has updated this issue:

        "Just to mention: our machines are back again, altough network connectivity
        isn't as good as it should be. I have opened a service request (ticket #93
        <http://techsupport.creatifi.eu/issues/93>)

        I was thinking on the issue... How is the floating IP assigned to the node?
        doing a simple ifconfig does not show the public IP on the node. So there
        must be an upper layer (OpenStack Neuron, I guess) who redirects the
        traffic from the public floating to the inner private... We are surely
        needing some port redirection on our project, and we should know if we are
        going to face any kind of restrictions this way to rethink our strategy.

        Thank you,"

        2015-02-27 10:34 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>:

        > The developer i s working on the Budapest node.
        >
        > KR
        >
        > 2015-02-27 9:18 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>:
        >
        >> Hi,
        >>
        >> Developer added more information to this issue.
        >>
        >>
        >> This afternoon machines were accessible again, but after doing some
        >> iptables operations, servers are unavailable again. This is what we've done:
        >>
        >> 1. We have one server, lb.alquimia.io, with a floating IP and a public
        >> internal
        >> 2. We have a couple of backend servers with public internal
        >> 3. We pass on traffic to some ports on the frontend to the backend
        >> servers. To simulate this behavior, we are redirecting some ssh traffic on
        >> the head to the ssh on the backends.
        >> 4. This is what we do:
        >>
        >> root@sf-appserver1:~# echo "1" > /proc/sys/net/ipv4/ip_forward
        >> root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22022
        >> -j DNAT --to-destination 10.10.13.102:22
        >> root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22023
        >> -j DNAT --to-destination 10.10.13.105:22
        >> root@sf-appserver1:~# iptables -t nat -A POSTROUTING -j MASQUERADE
        >>
        >> where 10.10.13.102 and 105 are the backends.
        >>
        >> I have been able to work for five minutes on the servers, but then ssh
        >> client has became unresponsive, and servers are NOT appearing again on the
        >> cloud.lab webpage. In fact, NOTHING is being displayed on the cloud.lab
        >> portal: no images, no floating IPs, no keypairs, no security groups.
        >>
        >> I would say that there is a link between the 'iptables' commands and the
        >> connectivity problems, but I cannot even imagine what is the problem. I
        >> have done this 'iptables' trick in other virtualized environments without
        >> problems :/
        >>
        >> Thanks,
        >>
        >> i-
        >>
        >>
        >>
        >>
        >> 2015-02-26 9:21 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>:
        >>
        >>> Hello,
        >>>
        >>> We are experiencing difficulties in accessing our machines via SSH.
        >>> It is returning a time out "no route to host" error. We tried accessing
        >>> from three different computers using different connections.
        >>>
        >>> We checked them in FiwareLab interface and they are there, running.
        >>> However, the interface itself took some time to display the machines
        >>> (something that didn't happen before).
        >>>
        >>> Regards.
        >>>
        >>>
        >>>
        >>
        >

        _______________________________________________
        Fiware-creatifi-coaching mailing list
        Fiware-creatifi-coaching@lists.fi-ware.org
        https://lists.fi-ware.org/listinfo/fiware-creatifi-coaching

        Show
        fw.ext.user FW External User added a comment - Hi, The developer has updated this issue: "Just to mention: our machines are back again, altough network connectivity isn't as good as it should be. I have opened a service request (ticket #93 < http://techsupport.creatifi.eu/issues/93 >) I was thinking on the issue... How is the floating IP assigned to the node? doing a simple ifconfig does not show the public IP on the node. So there must be an upper layer (OpenStack Neuron, I guess) who redirects the traffic from the public floating to the inner private... We are surely needing some port redirection on our project, and we should know if we are going to face any kind of restrictions this way to rethink our strategy. Thank you," 2015-02-27 10:34 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>: > The developer i s working on the Budapest node. > > KR > > 2015-02-27 9:18 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>: > >> Hi, >> >> Developer added more information to this issue. >> >> >> This afternoon machines were accessible again, but after doing some >> iptables operations, servers are unavailable again. This is what we've done: >> >> 1. We have one server, lb.alquimia.io, with a floating IP and a public >> internal >> 2. We have a couple of backend servers with public internal >> 3. We pass on traffic to some ports on the frontend to the backend >> servers. To simulate this behavior, we are redirecting some ssh traffic on >> the head to the ssh on the backends. >> 4. This is what we do: >> >> root@sf-appserver1:~# echo "1" > /proc/sys/net/ipv4/ip_forward >> root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22022 >> -j DNAT --to-destination 10.10.13.102:22 >> root@sf-appserver1:~# iptables -t nat -A PREROUTING -p tcp --dport 22023 >> -j DNAT --to-destination 10.10.13.105:22 >> root@sf-appserver1:~# iptables -t nat -A POSTROUTING -j MASQUERADE >> >> where 10.10.13.102 and 105 are the backends. >> >> I have been able to work for five minutes on the servers, but then ssh >> client has became unresponsive, and servers are NOT appearing again on the >> cloud.lab webpage. In fact, NOTHING is being displayed on the cloud.lab >> portal: no images, no floating IPs, no keypairs, no security groups. >> >> I would say that there is a link between the 'iptables' commands and the >> connectivity problems, but I cannot even imagine what is the problem. I >> have done this 'iptables' trick in other virtualized environments without >> problems :/ >> >> Thanks, >> >> i- >> >> >> >> >> 2015-02-26 9:21 GMT+01:00 Xavier Carol Rossell <xavier.carol@i2cat.net>: >> >>> Hello, >>> >>> We are experiencing difficulties in accessing our machines via SSH. >>> It is returning a time out "no route to host" error. We tried accessing >>> from three different computers using different connections. >>> >>> We checked them in FiwareLab interface and they are there, running. >>> However, the interface itself took some time to display the machines >>> (something that didn't happen before). >>> >>> Regards. >>> >>> >>> >> > _______________________________________________ Fiware-creatifi-coaching mailing list Fiware-creatifi-coaching@lists.fi-ware.org https://lists.fi-ware.org/listinfo/fiware-creatifi-coaching
        Hide
        WIGNER Budapest Node Helpdesk added a comment -

        Dear Xavier,

        Unfortunately, our site is overloaded, and we are in the middle of migration to a new environment where the computational resources will be doubled. This will solve this performance issue. However, we try to free some resources to solve your performance problems. We will come back with the answer even this week.

        Sorry for the inconvenience...

        Best,
        Sandor

        Show
        WIGNER Budapest Node Helpdesk added a comment - Dear Xavier, Unfortunately, our site is overloaded, and we are in the middle of migration to a new environment where the computational resources will be doubled. This will solve this performance issue. However, we try to free some resources to solve your performance problems. We will come back with the answer even this week. Sorry for the inconvenience... Best, Sandor
        Hide
        sandor Sándor Laki added a comment -

        Please use Budapest2 with increased capacity.

        Show
        sandor Sándor Laki added a comment - Please use Budapest2 with increased capacity.

          People

          • Assignee:
            WIGNER Budapest Node Helpdesk
            Reporter:
            fw.ext.user FW External User
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: