Hi Henar, yesterday I was able to solve the issue. I had to reduce the MTU size of eth0 on the VM that was launched through Blueprint. We had already other connectivity issues wrt MTU size and the reason seem that we are using GRE tunneling. Now I've configured the dhcp-option-force=26,1456 for the dnsmasq. This will set the MTU size of the interfaces from new VMs to this value and this seems to solve the connectivity issues.

Thanks again for your support.

Hi Henar, I gave you an additional public IP (193.175.132.57), please delete it if it is not needed anymore.

Hi

My skype is henarmunoz. My ID is 00000000000000000000000000000081
Regards,
Henar

Unfortunately we're not allowed to give third parties SSH access our servers. Please let us know the log files you're interested in and we'll provide the required information asap. We could also have together a live debugging session via GotoMeeting or Skype.

Please provide your tenant ID and I'll assign a floating IP.

in addition I will need a floating ip to be able to access and check it.

Hi

yes I am testing and it seems problems with the infrastructure. I think the problem was that the metadata service was not working corretly and it avoid the start of the VM. Could you make available the logs so that it is possible to access to VM logs from outside? It will help a lot.
Regards,
Henar

Hi Henar, did you had the chance to further investigate? Is there anything we can help in solving this issue?

Hi
I am doing some tests with the same image, with blueprint and without. The only difference is the user_data used in contextualization. I will continue testing to know why it is not working in Berlin2, while it is working in other regions
Regards,
Henar

Hi Henar,

could you please advise what exactly needs to be checked?

From a regular VM I can access the metadata service.
root@tgu-test:~# curl http://169.254.169.254
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04

If you have a look at the comments 2 days ago, you'll see that I already tried to connect to the VM through the namespace which was failing.

VM created with Blueprint:
root@xifi-juno-ctrl:~# ip netns exec qrouter-aab3f734-ec00-4702-9eee-6e99838e9d65 ping -c 1 192.168.6.34
PING 192.168.6.34 (192.168.6.34) 56(84) bytes of data.

— 192.168.6.34 ping statistics —
1 packets transmitted, 0 received, 100% packet loss, time 0ms

VM created without Blueprint
root@xifi-juno-ctrl:~# ip netns exec qrouter-aab3f734-ec00-4702-9eee-6e99838e9d65 ping -c 1 192.168.6.29
PING 192.168.6.29 (192.168.6.29) 56(84) bytes of data.
64 bytes from 192.168.6.29: icmp_seq=1 ttl=64 time=0.401 ms

— 192.168.6.29 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.401/0.401/0.401/0.000 ms

root@tgu-test:~# ip a l eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether fa:16:3e:9c:6d:78 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.29/24 brd 192.168.6.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe9c:6d78/64 scope link
valid_lft forever preferred_lft forever

As you can see above the second host is able to reach the metadata service.

Hi

Having a look with the console http://193.175.132.6:6080/vnc_auto.html?token=b3433ba0-a7ad-45f4-886b-697c01d2768e, it seems that metadata are not working. Anyway, I cannot access.
As you are administrator, you can debug by using ip netns in your network controller. Anyway, I will need to have increase my floatingip quota to check it.
Regards,
Henar

A reboot doesn't changed the situation. I'm still not able to connect to the VM either through VNC nor ping or SSH.

The console-log is not really helpful:

root@xifi-juno-ctrl:~# nova console-log 63e1a8f6-8d37-42dd-ae8f-f240194b9709
?

I created my own Blueprint template with an Ubuntu14.04 Image. When I launch the blueprint it shows Status: Installing at Blueprint instances, and on instance view it show active and running. If I connect to the VNC I see the deployment of my SSH Key but no login.

If I launch the same Image from Glance repository everything is working fine. I'm able to connect to VM without any problem.

Therefore I assume that there is some general issue with blueprints on the Berlin node. Something seems to go wrong during the launch process. What are the difference between blueprint and glance image launch?

I'm not able to connect to the instance.

root@xifi-juno-ctrl:/home/ngniadm# ip netns exec qrouter-d02d3b92-ef67-42a4-82fe-f63a543635bc ping 192.168.1.154
PING 192.168.1.154 (192.168.1.154) 56(84) bytes of data.
^C
— 192.168.1.154 ping statistics —
4 packets transmitted, 0 received, 100% packet loss, time 2999ms

The instance probably didn't booted correctly. If I connect via VNC I see the CentOS boot prompt.
See attachment.

I'll try to reboot the instance.

hi

it seems that your VM has not access to the chef-server. Could you enter in the VM and check that the VM has IP and it can connect to chef-server.lab.fiware.org?
REgards,
Henar

Now the status changes from installing to error:

Success: Blueprint Instance test-2 status.

Description: Create environment test-2

Status: ERROR

Error: Error installing a product. Description:com.telefonica.euro_iaas.paasmanager.exception.ProductInstallatorException: Error installing product orion-0.19.0 Error invokg SDC to Install Productorion-0.19.0 0.19.0 SDCException. com.telefonica.euro_iaas.sdc.exception.SdcRuntimeException: com.telefonica.euro_iaas.sdc.exception.CanNotCallChefException: Node test-2-orion-1-003233 is not registered in ChefServer: com.telefonica.euro_iaas.sdc.exception.SdcRuntimeException: com.telefonica.euro_iaas.sdc.exception.CanNotCallChefException: Node test-2-orion-1-003233 is not registered in ChefServer. com.telefonica.euro_iaas.sdc.exception.SdcRuntimeException: com.telefonica.euro_iaas.sdc.exception.CanNotCallChefException: Node test-2-orion-1-003233 is not registered in ChefServer

Hi
Same error. createSecurityGroups | msg=[InfrastructureException] It is not possible to create the security group sg_00000000000000000000000000003233_CBinstance-orion-1-003233 Error performing post on the resource

I guess you have achieved the security group quota. Could you check it?
Regards,
Henar

I used the same template as yesterday and it shows again the same error.

Success: Blueprint Instance CBinstance status.
Description: Create environment CBinstance
Status: ERROR
Error: The Environment CBinstance is Invalid

In the nova-api.log I see the following ERROR:
2015-06-19 10:03:25.389 5464 ERROR nova.network.security_group.neutron_driver [req-3ce97239-520f-4247-a5a5-1e2200faae55 None] Neutron Error adding rules to security group sg_00000000000000000000000000003233_CBinstance-orion-1-003233

I see that the security rule is existing for the user:
tgu@potemkin:~$ nova secgroup-list
--------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------

But it seems that because of the currently configured neutron quota, the user is not able to add a security rules. Therefore I tried to force the issue through CLI.

tgu@potemkin:~$ nova secgroup-list-rules aed8b799-a662-4e38-a36e-0edc6a31b763
-------------------------------------------------

-------------------------------------------------
-------------------------------------------------
tgu@potemkin:~$ nova secgroup-add-rule aed8b799-a662-4e38-a36e-0edc6a31b763 tcp 22 22 0.0.0.0/0
ERROR: Quota exceeded for resources: ['security_group_rule'] (HTTP 403) (Request-ID: req-805a10d2-4afb-4935-8a3a-4c2692d98da8)

Then I verfied the configured neutron quota:
tgu@potemkin:~$ neutron quota-show
--------------------------+

--------------------------+

--------------------------+

It seems the the parameter security_group and security_group_rule doesn't have to have the same value.

I increased the value for security_group_rule to 20.
root@xifi-juno-ctrl:~# neutron quota-update --security_group_rule 20 --tenant-id 00000000000000000000000000003233
--------------------------+

--------------------------+

--------------------------+

Now I was able to add a rule to the security group via CLI.
tgu@potemkin:~$ nova secgroup-add-rule aed8b799-a662-4e38-a36e-0edc6a31b763 tcp 22 22 0.0.0.0/0
--------------------------------------------------

--------------------------------------------------

--------------------------------------------------

Next step was to launch again a Blueprint.
There again I reached some quota limit, which I don't understand.
I currently have neutron and nova quota configured to 10 security_groups and 20 security_group_rule. There were only 7 security rules available, while running the test.

tgu@potemkin:~$ nova secgroup-list
----------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------

I would propose that security rules for Blueprint instances will be deleted during termination of blueprint instances.

After I deleted the obsolete security rules the launch of the Blueprint instance seems to be successful.

Success: Blueprint Instance test-2 status.
Description: Create environment test-2
Status: RUNNING

I'm wondering why the Status shows still INSTALLING. Is this the expected behavior?
See attachment.

Thanks for pointing in the right direction.

Hi

We have detected a bug, where it does not show some errors to the users. Concretely, in your case, there was a problem to create teh security groups, since it was a connection timeout with the neutron in Berlin2.
Could you try it again and we can check it if it continues?
The bug will be solved in next release.
Regards,
Henar
Error performing post on the resource: http://193.175.132.6:8774/v2/00000000000000000000000000003233/os-security-groups with payload: {"security_group":
{"name": "sg_00000000000000000000000000003233_CBinstance-orion-1-003233", "description": "descripcion" }} java.net.SocketException: Connection reset

Hi
thanks. I can reproduce the same error. I will check what it is happening and I will tell you.
Regards,
Henar

I clone a template from existing Blueprint of the catalog.
After I try to launch the template on the Berlin2 node it get stuck in the deploying state.
If I click on the info icon I can see the following Error:

Success: Blueprint Instance CBinstance status.
Description: Create environment CBinstance
Status: ERROR
Error: The Environment CBinstance is Invalid

See also attachment. Is there something I'm doing wrong?

Hi

TEll exactly what you are doing? If you are cloning, you are deploying in Spain2 right? Which tempalte are you cloning? Are you modifying for inscluding Berlin information?
In today logs, I cannot see this error. When did you do the test?
Regards,
Henar