Dear Carmen and all,

Here is a message from Juanjo Hierro about the immediate actions planned in order to improving the state of FIWARE Lab. I thought you might want to see this in case you are not on Basecamp. My apologies if you are receiving it for the second time.

Best regards,

ilknur

-------- Forwarded Message --------
Subject:

FIWARE Lab Recovery Task Force

Date:

Thu, 16 Apr 2015 07:22:15 +0200

From:

Juanjo Hierro <juanjose.hierro@telefonica.com><juanjose.hierro@telefonica.com>

Dear all,

As many of you know, many users of the FIWARE Lab have been experiencing very serious problems since certainly a longer period of time than all of us would desire. The success of the FIWARE Lab in terms of number of users rapidly led to exhaustion of physical resources which in turn led to malfunctioning of the environment due to the over-subscription. This gives the poor and misleading impression that "FIWARE doesn't work" when the fact is that the FIWARE Lab was mostly not working properly due to the fact that resources got exhausted.

We had planned an expansion of the computing capacity of the Spain node which, combined with new user account management policies was going not only to help us fixing the situation but keeping it stable in the future. Unfortunately, the expansion of the computing capacity, initially planned to be in during January, was delayed much more than expected and it was only recently implementable.

We would like to sincerely apologise for all these problems.

Now that a first expansion of the computing capacity of the Spain node is implementable, and the new user account management policies have been defined and communicated, giving existing users the ability to get ready, we have setup a FIWARE Lab Recovery Task Force in order to accelerate the recovery of the situation.

Members of this Task Force meet daily in order to follow-up progress on defined actions as well as to anticipate and solve any issues that may occur in the implementation. We will keep you informed about progress and any difficulties we may face, as well as the actions to overcome them.

The FIWARE Lab Recovery Task Force has defined an Action Plan with precise steps we would like to share with you:

Step 1: reminder about new user account management policies enforcement (already done).

A first email asking existing FIWARE Lab users to accept the new user account management policies, also covered in the new FIWARE Lab use terms and conditions, was sent to all FIWARE Lab users several weeks ago.

These new user account management policies will introduce the distinction between Trial Users, who will be allowed to experiment with FIWARE technology during a limited period of two weeks and with limited number of resources assigned, and Community Users who will be given a more stable environment for testing (e.g. SMEs/startups under the FIWARE Accelerator programme). Trial Users can always apply to for upgrading their accounts and become Community Users (this will be granted to SMEs/startups under the FIWARE Accelerator Programme as well as FIWARE Use Case projects, also to third parties if it is understood that the application they aim at developing which will be showcased on the FIWARE Lab is considered a relevant reference example for the development of the FIWARE community). In order to apply for becoming Community User an application form has been provided which is accessible through the Help&Info page of the FIWARE Lab portal ("Request Community Account Upgrade" button).

A reminder was sent to current users of the FIWARE Lab on Monday April 13 and the day after, telling them that they had until April 15th 2015 19:00 CET to accept the new user account management policies. This message was disseminated through different means:

§ mailed to current users of the FIWARE Lab
§ shown in a screen at login of users of the FIWARE Lab
§ posted on Basecamp so that FIWARE accelerator projects get aware and can forward the remainder to SMEs/startups under their respective programmes
§ mailed to the FI-PPP SB and the old FI-PPP AB members so that coordinators/architects of the FIWARE Use Case projects can forward the reminder to developers in their projects

You can check the message that was sent to existing FIWARE Lab users where a brief description of the new types of Users here:

https://docs.google.com/a/fbk.eu/document/d/1FreMU2soDr6yx61z7-c7iftM9vmjj0sKbxYD9zi2nEs/edit?usp=sharing

Step 2: Increase of computing capacity in Spain-2 node

In parallel and starting on April 15th, activities targeted to expand the computing capacity of the recently launched Spain-2 node have been prioritised so that we reach a capacity of aprox 500 cores in the datacenters located in both Sevilla and Malaga. We expect this expansion to be completed by April 20th if not earlier.

Besides this expansion, additional capacity (close to 500 additional cores) will be activated in the Spain-2 node with servers located in a new datacenter in Las Palmas de Gran Canarias. Conversations are ongoing with Red.es, owner of the infrastructure associated to datacenters linked to the Spain-2 node, trying to accelerate this process so that these additional 500 cores get available before the end of next week.

Step 3: New Trial users only allowed to work on Spain-2 node. Limitation of number of concurrent trial users.

Changes will be implemented in the Cloud portal and the user account registration process/tool on April 16th (today) so that new users, who will be labeled as Trial Users, will only see the Spain-2 node.

In addition, Trial Users will be temporarily limited to 100 Trial users. We will fine tune this limit as soon as computing capacity is expanded as far as this doesn't put in danger the stability of the FIWARE Lab environment. Given said this, it is worth reminding that Trial Users can always apply for upgrading to Community Users.

Most likely, the fact that Trial users will only be allowed to work on the Spain node will remain for a relatively long period of time.

Users who try to create an account when the limit on Trial users has been reached, will receive a message indicating that this limit has been reached, inviting them to try again in two weeks. Accepted Trial Users as well as users who haven't been able to create a Trial User account will also be allowed to directly apply for becoming Community Users. SMEs/startups under the FIWARE Accelerator programme are encouraged to apply for becoming Community Users as soon as possible.

Step 4: Freeing resources on current nodes, giving users the ability to apply to become community users, asking users in some of the current FIWARE Lab nodes to migrate

Starting April 16th (today), organizations in charge of the current nodes will free resources on their nodes (virtual infrastructure and public IP addresses allocated to users who didn't accept the new user account management policies).

They should complete this process before April 23rd. FIWARE Lab node providers are aware of the current situation so they should accelerate this process as much as possible, trying to complete the process the sooner the better. Progress will be assessed on April 20th and April 23rd. It is expected that the status of the FIWARE Lab will improve drastically during this process and will become stable by April 23rd.

In parallel, we will encourage legacy users (those who created their account before April 16th and have accepted the new terms and conditions) to apply for becoming Community Users.

In order to improve the performance in several of the FIWARE Lab nodes and also support the new user account management policies, an upgrade of the underlying OpenStack release being used is requested. Because of that, those legacy users whose application is accepted and who were hosted on FIWARE Lab nodes with a version of OpenStack previous to the IceHouse release (e.g., Spain-1 and Prague, we will report what other nodes are in this situation) will be asked to migrate their environments to any of the other nodes. A recommendation about what node to migrate to will be given in response to their application to become Community Users. A period of two weeks will be given to them to accomplish this migration. As a result, no users will remain in these nodes by May 1st, enabling migration of the nodes to the latest OpenStack release.

Step 5: Deployment of new IdM/Keystone component and upgrading of Community Users. Migration of Spain-1 and Prague nodes to OpenStack Juno.

This step will start May 1st. Then, we will deploy the new version of the IdM/Keystone component which will incorporate a number of relevant improvements and will help management of user accounts.

Users whose application to become Community Users has been approved will become Community Users. Environments of those Trial users who have passed the 2 weeks trial period will be cleaned out. From then on, the new user account management policies will be in place, with the new IdM/keystone component supporting the ability to distinguish between kind of users and the nodes each user is able to get access to.

In parallel, FIWARE Lab nodes which were based on OpenStack releases previous to IceHouse will start their migration. Once they migrate, they will be made available as part of the FIWARE Lab federation. Physical resources allocated to the Spain-1 node will be assigned to the Spain-2 node, expanding the computing capacity of this node even further. The limit on number of Trial Users will then be adjusted.

Step 6: Assessment of nodes in the FIWARE Lab federation. Bringing FIWARE Lab nodes into quarantine when poor performance is not achieved.

After April 23th, performance of the several nodes in the FIWARE Lab federation will be weekly assessed. Those nodes who exhibit a bad performance (instability, low response to request for help by users) will be put into quarantine and their users (Community Users) will be asked to migrate to some of the more stable nodes.

We want to thank in advance all users of the FIWARE Lab for their patience during this process. We apologise for the temporary inconveniences this will cause. We hope this crash plan will help to overcome the current situation and will significantly improve the experience of FIWARE Lab users.

Please don't hesitate to share this message with those who may be relevant.

Should you have any question or require any assistance, please don't hesitate to ask to the FIWARE Lab Help Desk (fiware-lab-help@lists.fi-ware.org<fiware-lab-help@lists.fi-ware.org>)

Best regards,

Juanjo Hierro
FIWARE Coordinator and Chief Architect

Bu mesaj ve ekleri gönderilen kişiye özeldir ve gizli bilgiler içerebilir. Eğer mesajın gönderilmek istendiği kişi değilseniz lütfen kopyalamayınız, başkalarına göndermeyiniz ve göndericiyi bilgilendiriniz. Internet üzerinden gönderilen mesajların güvenli ve hatasız olduğunun garantisi olmadığından Atos grubu mesajın içeriğinden sorumlu tutulamaz. Göndericinin bilgisayarı anti-virüs sistemleri tarafından taranmaktadır, ancak yine de mesajın virüs içermediği garanti edilemez ve gönderici, meydana gelebilecek zararlardan sorumlu tutulamaz.

This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos group liability cannot be triggered for the message content. Although the sender endeavors to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.

_______________________________________________
Fiware-fractals-coaching mailing list
Fiware-fractals-coaching@lists.fi-ware.org
https://lists.fi-ware.org/listinfo/fiware-fractals-coaching