Application migration: best practices for success

Large enterprises usually have more than 1,000 systems running. Even smaller organisations may have hundreds of applications in their public cloud spaces or on their servers. In this world of IT systems, application migrations are common for the following reasons:

• At some point, software reaches its end of life and is not supported anymore by its vendor. For example, consider an application that uses a discontinued technology stack. The entire application needs to be replaced but the data must be stored and re-used. 
• New functionality or capabilities are required which the current software cannot deliver. For example, a system developed in-house that cannot offer a modern web user interface and must be replaced with modern off-the-shelf software, also involving data transfer from the old system to the new one.
• The number of individual IT systems is too high and requires consolidation. For example, an organisation wants to merge various source code repository servers from different departments into one unifying installation.

In addition to these three general motivations, there is a growing repatriation trend. The public cloud provides the optimal environment for most systems but not for all. For some systems, private cloud hosting can be more cost-efficient. Bringing back or “repatriating” an IT system from the public cloud to the private cloud can save companies significant costs.

However, the motivations listed above bring challenges for application migrations. Common issues companies face revolve around the technologies being old and complex, lack of IT expertise or deployments that are just unstructured and undocumented. Therefore, application migration projects are challenging and prone to failure.

How many application migration projects fail?

The effort of migrating software and data from one place to another is inherently complex and thus prone to errors. Prominent studies, such as the research from Gartner  in 2019, suggest that 50% of the data migration projects resulted in problems significantly affecting the business. The Cloud Security Alliance points out in their ERP and Cloud Adoption Report that 90% of the respondents have experienced a failing cloud migration project.

To make matters more complex, a migration project can be considered complete from a technical point of view but fail entirely from a business perspective. 

Was the migration of applications done but unsuccessful?

A migration project can be considered technically successful if all data is covered by counting records or the system is behaving entirely according to specification. However, the system may not be ready for business users for a number of reasons. 

Common issues include: 

• Sorting of results: a list is generated in alphabetical order, sometimes by order of record creation. Underlying data stores will affect sorting results out of the scope of the application. Result listings usually need to be specified, and users relying on an implicit result order will experience problems.
• Amount of data: users will use the system in unanticipated ways. For example, using an issue system for requirements engineering. Unanticipated uses lead to larger-than-expected data sets, lists or just texts. A large amount of data can result in problems in the user interface or can cause errors in data handling across interface boundaries.
• Data processing:  if a migration involves a new underlying system architecture, this will likely result in different runtime behaviour. “New” does not mean faster.  Virtualisation and containerisation, or new individual resource limits can lead to longer execution times, frustrated users or even sudden halts when hitting time-out limits.

Many more potential problem areas exist. And, of course, internal technical problems not visible to the user can be a source of concern as well. Examples include incompatible software versions, bugs in the software or unforeseen data encoding. These issues show that a solid project setup is required to ensure the overall success of a migration project.

How to set up a successful migration

Migration projects can be very different depending on the technologies involved. However, in all cases, project management must set up and steer the effort properly to ensure its success. Three key elements are essential: Planning, testing and participation. 

• Planning: a plan must be defined and transparently communicated to all stakeholders in advance. In the ideal case,  project management announces the overall schedule several months before actual efforts start. Since migration might involve uncertainties, the plan must allow for some corrections.
• Test migrations: in reality, a migration project performs several individual migrations. First, it performs several migrations covering specific and critical scopes from individual stakeholders. Then, multiple full migrations covering the entire (data) range are also required. Since the first complete migration is highly likely to fail, the project can plan for multiple runs in advance.
• Participation: project management invites all stakeholders from the beginning. A good migration project enables early access to test systems. Then, issues become transparent as early as possible and can be covered while time reserves are still available.

It is essential to distinguish between users and stakeholders in your project plan.  The group of stakeholders includes users but also several other roles: the system’s sponsors, IT managers of systems which require  integration, or regulatory functions in an organisation, such as the cybersecurity officer.

The implicit goal of the proposed setup is successful expectation management. Transparency allows all stakeholders to understand the result and correct or adapt the effort at the early stages. Late corrections to a migration that’s already taken place are usually painful to apply and cost more.

A blueprint for planning your application migration

The following diagram shows a planning blueprint for a migration project. The project manager needs to cover three main threads:

• Project management communicates the entire migration plan early on, in advance. Continued information in periodic meetings ensures transparency about the progress and expectation management. As with every project, it must formally come to a conclusion so everyone has the same understanding of the outcome.
• Engineering continuously implements the tooling and infrastructure setup for the migration. They perform the migration several times in a testing setup. Engineering anticipates that the first tests will reveal issues which require modifications. Subsequent iterations of tests and corrections are likely required until all problems are solved. At the same time,  engineering provides individual test installations to the relevant stakeholders.
• Stakeholders have test installations to understand the planned outcome. They can test the system with their use cases; maybe some of these are not covered by a specification so far. Project management interviews stakeholders at the defined end of the testing periods to capture all issues. Dedicated meetings implicitly ensure stakeholders spending time on the test system and provide a formal handover of the issues found.

The diagram outlines a simple schedule divided into three main threads: project management, engineering and stakeholder involvement. General meetings involving all stakeholders start very early. The approach runs in an iterative process that covers issues at various progress stages.

Most importantly, the project will continue after the transfer of the actual data or system has been completed. It’s likely that a few stakeholders will claim to be completely surprised that a migration happened. Despite all the preparation and coverage of issues before the transfer, project management must plan for a sufficient period to cover the problems resulting from late feedback. Defined meetings throughout the entire project duration ensure transparent and explicit communication.

Of course, every migration project is different and bears individual challenges. Some projects will have many stakeholders, some projects will have an excessive amount of data to cover, and some projects involve stakeholders from other countries. Some projects will deal with legacy technology, and some will struggle because people  with the required know-how are not available anymore. The blueprint above focuses on the non-technical, general challenges. It helps you to cover three key elements: transparent planning, iterative testing and active participation.

Define your success criteria for migration

The team performing the migration should define criteria for being successful. However, overall success will be determined if stakeholders  also conclude that their expectations are met. As a general rule of thumb, the users expect the system to perform equally or better than the previous system. Any extra effort resulting from the new system will be considered a burden and result in complaints or impact on the business.

Therefore, expectation management started at early stages increases the acceptance of the system after migration. The migration is only successful if all stakeholders confirm that the effort meets the expected goals. A final meeting summarising the actions involving all stakeholders ensures the successful conclusion of a migration project.

Further reading

To continue reading on this topic, consider the following links:

• A business guide to cloud migration
• Migrating to Ubuntu: Six facts for CentOS users
• Getting started with your open-source private cloud
• What is a private cloud?
• Lift, Shift or Rebuild – our whitepaper about cloud migration

Or, in case you would like to contact us – we would be happy to answer your questions:

Contact Canonical