Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-2242

FIWARE.Request.Lab.Gent.VM keep on "deleting" (ERROR status) and I cannot delete it

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-LAB-HELP
    • Labels:
      None
    • HD-Node:
      Gent

      Description

      Hi!

      As you can see in the report http://fi-health.lab.fi-ware.eu/RegionSanityCheck,
      some of the Gent sanity checks are failing due to this error:

      Quota exceeded for ram: Requested 2048, but already used 2048 of 3072 ram (HTTP 413)
      

      I can see that there is a VM with "deleting" status in the Tenant used for launching that test cases and I cannot delete it.

      • User: e.cloud10
      • TenantId: 00000000000000000000000000011233

      Could you please take a look to that VM?

      • Name: testing_instance_06
      • ID: d2b2b82b-9f8d-4908-aa61-756bfefd86f4
      • Status: ERROR
      • Image Name: CentOS-6.3init

      Thanks!

      BR,
      Javier.

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        20h 54m 1 Vicent Borja Torres (Inactive) 03/Mar/15 2:08 PM
        In Progress In Progress Answered Answered
        5m 37s 1 Vicent Borja Torres (Inactive) 03/Mar/15 2:14 PM
        Answered Answered Done Done
        1h 3m 1 Javier Fernández (Inactive) 03/Mar/15 3:18 PM
        Done Done Closed Closed
        314d 17h 52m 1 Manuel Escriche 12/Jan/16 9:10 AM
        fla Fernando Lopez made changes -
        Fix Version/s 2021 [ 12600 ]
        backlogmanager Backlog Manager made changes -
        Summary FIWARE.Request.Lab.VM keep on "deleting" (ERROR status) and I cannot delete it FIWARE.Request.Lab.Gent.VM keep on "deleting" (ERROR status) and I cannot delete it
        mev Manuel Escriche made changes -
        HD-Node Gent [ 10916 ]
        backlogmanager Backlog Manager made changes -
        Summary VM keep on "deleting" (ERROR status) and I cannot delete it FIWARE.Request.Lab.VM keep on "deleting" (ERROR status) and I cannot delete it
        mev Manuel Escriche made changes -
        Workflow FW eRequest Workflow [ 24401 ] FW extRequest Workflow-II [ 41115 ]
        Status Done [ 10000 ] Closed [ 6 ]
        Issue Type eRequest [ 10101 ] extRequest [ 10104 ]
        mev Manuel Escriche made changes -
        Component/s FIWARE-LAB-HELP [ 10279 ]
        cloudtid Javier Fernández (Inactive) made changes -
        Resolution Done [ 10000 ]
        Status Answered [ 10104 ] Done [ 10000 ]
        Hide
        Vicent Vicent Borja Torres (Inactive) added a comment -

        ERROR Status is because is trying to delete a VM that is already deleted at the domain level (compute), and the script just raise an exception with and exit before finish. And never manage to clean the database info.

        Show
        Vicent Vicent Borja Torres (Inactive) added a comment - ERROR Status is because is trying to delete a VM that is already deleted at the domain level (compute), and the script just raise an exception with and exit before finish. And never manage to clean the database info.
        Vicent Vicent Borja Torres (Inactive) made changes -
        Status In Progress [ 3 ] Answered [ 10104 ]
        Hide
        Vicent Vicent Borja Torres (Inactive) added a comment - - edited

        VM was deleted from compute but the database was not ware of it. Traced the messages until an error exception, that seems database never received delete confirmation. As follows:

        2015-03-02 10:41:21.598 AUDIT nova.compute.manager [req-c646a76b-084d-4c17-8f0e-43100dc8c9f1 e-dot-cloud10 00000000000000000000000000011233] [instance: d2b2b82b-9f8d-4908-aa61-756bfefd86f4] Terminating instance
        2015-03-02 10:41:22.924 ERROR nova.openstack.common.rpc.amqp [req-c646a76b-084d-4c17-8f0e-43100dc8c9f1 e-dot-cloud10 00000000000000000000000000011233] Exception during message handling

        RabbitMQ crashes and can miss messages, while the service is auto-recovering.

        Problem solved, restoring database manually.

        Show
        Vicent Vicent Borja Torres (Inactive) added a comment - - edited VM was deleted from compute but the database was not ware of it. Traced the messages until an error exception, that seems database never received delete confirmation. As follows: 2015-03-02 10:41:21.598 AUDIT nova.compute.manager [req-c646a76b-084d-4c17-8f0e-43100dc8c9f1 e-dot-cloud10 00000000000000000000000000011233] [instance: d2b2b82b-9f8d-4908-aa61-756bfefd86f4] Terminating instance 2015-03-02 10:41:22.924 ERROR nova.openstack.common.rpc.amqp [req-c646a76b-084d-4c17-8f0e-43100dc8c9f1 e-dot-cloud10 00000000000000000000000000011233] Exception during message handling RabbitMQ crashes and can miss messages, while the service is auto-recovering. Problem solved, restoring database manually.
        Vicent Vicent Borja Torres (Inactive) made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        Vicent Vicent Borja Torres (Inactive) added a comment -

        We will check it and report with more details.

        Show
        Vicent Vicent Borja Torres (Inactive) added a comment - We will check it and report with more details.
        Vicent Vicent Borja Torres (Inactive) made changes -
        Field Original Value New Value
        Assignee Gent Node Helpdesk [ iminds ] Vicent Borja Torres [ vicent ]
        cloudtid Javier Fernández (Inactive) created issue -

          People

          • Assignee:
            Vicent Vicent Borja Torres (Inactive)
            Reporter:
            cloudtid Javier Fernández (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: