Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-13047

FIWARE.Request.Lab.Genoa.fiware-lab - fi-health error: test_deploy_instance_with_new_network_and_e2e_snat_connection.

    Details

      Description

      Fiware-lab Genoa node Sanity Check recently sometimes failed because error

      test_deploy_instance_with_new_network_and_e2e_snat_connection

      The error seems related to an access into a test VM by fi-health procedure (see here below log chunk)

      Our side, we did not any specific change. Any idea about the reason of this failure?

      BTW, the error appears "randomly" and at the moment the sanity check is green.

      Thanks
      giorgio

      {{
      test_deploy_instance_with_new_network_and_e2e_snat_connection: exceptions.AssertionError
      Details
      PhoneHome request not received from VM 'e58544f9-5541-46f0-a587-0945b33e7b83'
      -------------------- >> begin captured logging << --------------------
      2017-12-12 11:53:12,358 UTC - TestCase - DEBUG - Loading userdata from file 'resources/templates/userdata/test.snat.phonehome.template'
      2017-12-12 11:53:12,362 UTC - TestCase - DEBUG - Userdata content: #cloud-config

      1. phone_home: if this dictionary is present, then the phone_home
      2. cloud-config module will post specified data back to given url
        #
        phone_home:
        url: http://fi-health.lab.fiware.org:8081/phonehome?TransactionId=07b2fb76-df33-11e7-b7d1-5254002b31dd
        post: [ hostname, instance_id ]
        tries: 10
      1. Capture all subprocess output into a logfile
      2. Useful for troubleshooting cloud-init issues
        output: {all: '| tee -a /var/log/cloud-init-output.log'}
      1. final_message
      2. default: cloud-init boot finished at $TIMESTAMP. Up $UPTIME seconds
      3. this message is written by cloud-final when the system is finished
      4. its first boot
        final_message: "The system is finally up, after $UPTIME seconds"
        2017-12-12 11:53:13,245 UTC - TestCase - DEBUG - Created router a32dda15-cb17-4473-ad10-72404841013d
        2017-12-12 11:53:13,245 UTC - TestCase - DEBUG - {u'status': u'ACTIVE', u'external_gateway_info':
        Unknown macro: {u'network_id'}

        , u'name': u'testing_router_snat_20171212115312', u'admin_state_up': True, u'tenant_id': u'8368995edf4042da9df4fb3ddcd45d89', u'routes': [], u'id': u'a32dda15-cb17-4473-ad10-72404841013d'}
        2017-12-12 11:53:13,428 UTC - TestCase - DEBUG - Created network af3132cf-c8eb-48a8-b992-e8897af9a384
        2017-12-12 11:53:13,690 UTC - TestCase - DEBUG - Created subnet sub-testing_network_20171212115312
        2017-12-12 11:53:13,690 UTC - TestCase - DEBUG - {u'status': u'ACTIVE', u'subnets': [], u'name': u'testing_network_20171212115312', u'admin_state_up': True, u'subnet':

        Unknown macro: {u'name'}

        , u'tenant_id': u'8368995edf4042da9df4fb3ddcd45d89', u'mtu': 1500, u'router:external': False, u'shared': False, u'id': u'af3132cf-c8eb-48a8-b992-e8897af9a384'}
        2017-12-12 11:53:14,060 UTC - TestCase - DEBUG - Linked subnet '9948a9b2-f1c8-4f8f-b465-5c9de3057919' to router a32dda15-cb17-4473-ad10-72404841013d
        2017-12-12 11:53:14,061 UTC - novaclient.v2.client - DEBUG - REQ: curl -g -i -X GET http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/flavors/detail -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token:

        {SHA1}a7960845e63059353d4249adeacb58067244c3de"
        2017-12-12 11:53:14,172 UTC - novaclient.v2.client - DEBUG - RESP: [200] Content-Type: application/json Content-Length: 2630 X-Compute-Request-Id: req-8cfab652-ffc0-4f47-9920-584471de4d21 Date: Tue, 12 Dec 2017 11:41:16 GMT Connection: close
        RESP BODY: {"flavors": [{"name": "m1.tiny", "links": [{"href": "http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/flavors/1", "rel": "self"}, {"href": "http://130.251.135.65:8774/8368995edf4042da9df4fb3ddcd45d89/flavors/1", "rel": "bookmark"}], "ram": 512, "OS-FLV-DISABLED:disabled": false, "vcpus": 1, "swap": "", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1, "id": "1"}, {"name": "m1.small", "links": [{"href": "http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/flavors/2", "rel": "self"}, {"href": "http://130.251.135.65:8774/8368995edf4042da9df4fb3ddcd45d89/flavors/2", "rel": "bookmark"}], "ram": 2048, "OS-FLV-DISABLED:disabled": false, "vcpus": 1, "swap": "", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 20, "id": "2"}, {"name": "m1.medium", "links": [{"href": "http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/flavors/3", "rel": "self"}, {"href": "http://130.251.135.65:8774/8368995edf4042da9df4fb3ddcd45d89/flavors/3", "rel": "bookmark"}], "ram": 4096, "OS-FLV-DISABLED:disabled": false, "vcpus": 2, "swap": "", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 40, "id": "3"}, {"name": "m1.large", "links": [{"href": "http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/flavors/4", "rel": "self"}, {"href": "http://130.251.135.65:8774/8368995edf4042da9df4fb3ddcd45d89/flavors/4", "rel": "bookmark"}], "ram": 8192, "OS-FLV-DISABLED:disabled": false, "vcpus": 4, "swap": "", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 80, "id": "4"}, {"name": "m1.xlarge", "links": [{"href": "http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/flavors/5", "rel": "self"}, {"href": "http://130.251.135.65:8774/8368995edf4042da9df4fb3ddcd45d89/flavors/5", "rel": "bookmark"}], "ram": 16384, "OS-FLV-DISABLED:disabled": false, "vcpus": 8, "swap": "", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 160, "id": "5"}, {"name": "m1.micro", "links": [{"href": "http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/flavors/5944c310-e348-4225-a646-5ddcba8d4483", "rel": "self"}, {"href": "http://130.251.135.65:8774/8368995edf4042da9df4fb3ddcd45d89/flavors/5944c310-e348-4225-a646-5ddcba8d4483", "rel": "bookmark"}], "ram": 64, "OS-FLV-DISABLED:disabled": false, "vcpus": 1, "swap": "", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 0, "id": "5944c310-e348-4225-a646-5ddcba8d4483"}]}

        2017-12-12 11:53:14,173 UTC - novaclient.v2.client - DEBUG - GET call to compute for http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/flavors/detail used request id req-8cfab652-ffc0-4f47-9920-584471de4d21
        2017-12-12 11:53:14,174 UTC - novaclient.v2.client - DEBUG - REQ: curl -g -i -X GET http://130.251.135.65:8774/v2/8368995edf4042da9df4fb3ddcd45d89/images -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}

        a7960845e63059353d4249adeacb58067244c3de"
        2017-12-12 11:53:15,382 UTC - novaclient.v2.client - DEBUG - RESP: [200] Content-Type: application/json Content-Length: 37035 X-Compute-Request-Id: req-6c4550f8-b055-4372-a6ba-a8ad2d05b3b1 Date: Tue, 12 Dec 2017 11:41:17 GMT Connection: close
        RESP BODY: {"images": [

        Unknown macro: {"id"}

        Activity

        Hide
        tntlab Genoa Node Support Team added a comment -

        One possible reason of this fi-health failure:
        some public IP addresses in range assigned by Genoa node
        have been "stolen" by unknown users in our department.
        We are investigating on it.

        The issue happens at random and today the issue does not happen again, so I close the issue (by now).

        giorgio

        Show
        tntlab Genoa Node Support Team added a comment - One possible reason of this fi-health failure: some public IP addresses in range assigned by Genoa node have been "stolen" by unknown users in our department. We are investigating on it. The issue happens at random and today the issue does not happen again, so I close the issue (by now). giorgio
        Hide
        fw.ext.user FW External User added a comment -

        Comment by jira-help-desk@jira.fiware.org :

        Spam detection software, running on the system "fiware-forge-01", has
        identified this incoming email as possible spam. The original message
        has been attached to this so you can view it (if it isn't spam) or label
        similar future email. If you have any questions, see
        the administrator of that system for details.

        Content preview: [ https://jira.fiware.org/browse/HELP-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
        ] FW External User updated HELP-13047: > [Fiware-general-help] [SPAM] [FIWARE-JIRA]
        (HELP-13082) [SPAM] Website Job Application > > > Key: HELP-13047 > URL:
        https://jira.fiware.org/browse/HELP-13047 > Project: Help-Desk > Issue Type:
        extRequest > Reporter: FW External User > Attachments: [FIWARE-JIRA] (HELP-13082)
        [Fiware-general-help] [SPAM] Website Job Application > > > Spam detection
        software, running on the system "fiware-forge-01", has > identified this
        incoming email as possible spam. The original message > has been attached
        to this so you can view it (if it isn't spam) or label > similar future email.
        If you have any questions, see > the administrator of that system for details.
        > Content preview: [ https://jira.fiware.org/browse/HELP-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
        > ] FW External User updated HELP-13082: > [Fiware-general-help] [SPAM] Website
        > Job Application > > > Key: HELP-13082 > URL: https://jira.fiware.org/browse/HELP-13082
        > > Project: Help-Desk > Issue Type: extRequest > Components: SPAM > Reporter:
        > FW External User > Attachments: [FIWARE-JIRA] (HELP-12876) [Fiware-general-help]
        > [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application, [FIWARE-JIRA]
        > (HELP-12876) [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM]
        > Website Job Application, [FIWARE-JIRA] (HELP-12876) [Fiware-general-help]
        > [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application, [FIWARE-JIRA]
        > (HELP-12877) [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM]
        > Website Job Application, [FIWARE-JIRA] (HELP-12877) [Fiware-general-help]
        > [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application, [FIWARE-JIRA]
        > (HELP-12877) [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM]
        > Website Job Application, [FIWARE-JIRA] (HELP-12878) [Fiware-general-help]
        > [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application, [FIWARE-JIRA]
        > (HELP-12878) [Fiware-general-help] [SPAM] [...]

        Content analysis details: (6.9 points, 4.5 required)

        pts rule name description
        ---- ---------------------- --------------------------------------------------
        1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist
        [URIs: latedeliver.com]
        2.5 URIBL_DBL_SPAM Contains a spam URL listed in the DBL blocklist
        [URIs: latedeliver.com]
        0.0 FSL_HELO_NON_FQDN_1 FSL_HELO_NON_FQDN_1
        3.8 HELO_LOCALHOST HELO_LOCALHOST
        -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
        [score: 0.0000]
        0.8 RDNS_NONE Delivered to internal network by a host with no rDNS

        Show
        fw.ext.user FW External User added a comment - Comment by jira-help-desk@jira.fiware.org : Spam detection software, running on the system "fiware-forge-01", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: [ https://jira.fiware.org/browse/HELP-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FW External User updated HELP-13047 : > [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application > > > Key: HELP-13047 > URL: https://jira.fiware.org/browse/HELP-13047 > Project: Help-Desk > Issue Type: extRequest > Reporter: FW External User > Attachments: [FIWARE-JIRA] (HELP-13082) [Fiware-general-help] [SPAM] Website Job Application > > > Spam detection software, running on the system "fiware-forge-01", has > identified this incoming email as possible spam. The original message > has been attached to this so you can view it (if it isn't spam) or label > similar future email. If you have any questions, see > the administrator of that system for details. > Content preview: [ https://jira.fiware.org/browse/HELP-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > ] FW External User updated HELP-13082: > [Fiware-general-help] [SPAM] Website > Job Application > > > Key: HELP-13082 > URL: https://jira.fiware.org/browse/HELP-13082 > > Project: Help-Desk > Issue Type: extRequest > Components: SPAM > Reporter: > FW External User > Attachments: [FIWARE-JIRA] (HELP-12876) [Fiware-general-help] > [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application, [FIWARE-JIRA] > (HELP-12876) [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] > Website Job Application, [FIWARE-JIRA] (HELP-12876) [Fiware-general-help] > [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application, [FIWARE-JIRA] > (HELP-12877) [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] > Website Job Application, [FIWARE-JIRA] (HELP-12877) [Fiware-general-help] > [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application, [FIWARE-JIRA] > (HELP-12877) [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] > Website Job Application, [FIWARE-JIRA] (HELP-12878) [Fiware-general-help] > [SPAM] [FIWARE-JIRA] (HELP-13082) [SPAM] Website Job Application, [FIWARE-JIRA] > (HELP-12878) [Fiware-general-help] [SPAM] [...] Content analysis details: (6.9 points, 4.5 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist [URIs: latedeliver.com] 2.5 URIBL_DBL_SPAM Contains a spam URL listed in the DBL blocklist [URIs: latedeliver.com] 0.0 FSL_HELO_NON_FQDN_1 FSL_HELO_NON_FQDN_1 3.8 HELO_LOCALHOST HELO_LOCALHOST -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] 0.8 RDNS_NONE Delivered to internal network by a host with no rDNS

          People

          • Assignee:
            tntlab Genoa Node Support Team
            Reporter:
            tntlab Genoa Node Support Team
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: