Making the move to the cloud can be daunting but it is worth the effort for many businesses. Kevin Kline, Principal Programme Manager at SentryOne, talks through a five-step process to make sure the transition is as smooth as possible.
The cloud is no longer a new concept – it has gained so much popularity that Gartner predicts more than US$1.3 trillion in IT spending will be directly or indirectly affected by the shift to cloud by 2022. However, with data being the lifeblood of businesses, taking the step to move to the cloud can be daunting. No IT team should ever go into this process lightly – it requires detailed pre-planning and careful consideration to ensure that the transition is as smooth as possible. But that doesn’t mean it isn’t worth the effort, with businesses seeing multiple benefits including reduced costs and easier user access. To make a migration successful, there are five steps that businesses can work through to keep on top of their data and get their cloud data centre up to speed as quickly as possible.
1. Researching the right cloud for you
One of the challenges when choosing the best cloud for your business is to consider what kind of downtime you can afford while the migration process is going on. Depending on how long you can be offline, this might impact which provider and which features and services you opt for. But a more productive technique is to start replicating data to the cloud while still working off the on-premises system – that way you have the current system that you’re used to working efficiently while the cloud system is being set up in parallel.
When it comes to migrating to the cloud, the same theories apply as they do when migrating to any other platform. Essentially, you need to understand the capabilities of where you’re moving from, compared to where you’re moving to. For example, if you move from a SQL Server on-premises to a SQL database in the cloud, there may be a few aspects of your existing database that the cloud provider does not support. As cloud services continue to develop rapidly, the list of unsupported elements is reducing, but at first there may be some sticking points to be aware of. Researching the cloud provider you are considering and being fully informed about the potential challenges you might face during migration is an important step before beginning the transition.
2. The deep clean
When you’re making the step to move to the cloud – a brand new platform – you don’t want to take unnecessary data and applications from your current system with you as storage is one of the major costs of cloud computing. Cloud migration gives businesses the chance to start afresh, with a clean database that only contains exactly what you need. Therefore, you want to sort through all of the data on your current system and decide what should be kept and what you are happy to leave behind. Not only does this make your system easier to navigate on the cloud (as you no longer have to sift through unnecessary data sets), but it also makes the actual migration quicker, reduces the storage costs of the database, and makes the system faster once it’s up and running.
As part of this cleansing process, make sure that you review the data you do want to migrate to ensure that it will still be compliant once it is held in the cloud. Data that is stored on-premises will come under a specific set of compliance rules, but they may vary slightly on the cloud. Consider compliance regulations too such as GDPR – this requires you to know exactly where data is being stored, e.g. are the cloud servers located within the UK or Europe? Not all cloud providers will provide the same protection and it is therefore crucial to research exactly what your chosen cloud provider offers you to maintain legal compliance once your data is already in the cloud.
3. Document every detail
Closely linked to data cleansing is documentation and discovering the lineage of your data. Few teams enjoy documentation or do it well but it is more important during a cloud migration than any other time. Data lineage means looking at the history of your data – where does it originate from? It also means discovering where it has been since it was created. For example, data lineage might start with a Point-of-Sale retail system which saves data to flat files, which are then moved every night to an intermediate staging SQL Server. Next, data is cleaned, transformed and moved from the staging server using a Python programme to the production database. Finally, the data in the production database is moved and aggregated on a data warehouse server where end-users can run their own reports and analysis. In this system, our data lineage is four stages deep, five if you also count all of the end-user reports and analytics.
The reasoning behind this is to determine whether or not any data needs to be modified before it is migrated to the cloud – not all aspects of on-premises data will be useful or well-suited to a cloud environment. If there is any data that needs to be kept on-premises, you can document it fully so that once the migration is underway, you know exactly what data needs to move and what doesn’t.
4. Keeping it parallel
During the migration itself, there are two main aspects to consider – the ETL (extract, transform, load) process of data and the applications, both of which may need to be kept in parallel between the old system and the new one. For ETL, businesses can have the data replicated on to the new cloud system while users are still on the on-premises environment using the existing data.
For the applications, you have two options. In some cases, you may be able to simply switch over from the old, on-premise application and database and switch to the new, cloud application and database. But that is often impossible. So teams need to understand how their apps are currently configured so that they can be accurately rewritten for the cloud, replicating each layer exactly or in some cases, even writing new code to take advantage of cloud-centric features available only in the cloud. It’s vital for database administrators (DBAs) and the subject matter experts who use the application to then check the two systems against each other, as there is the potential for some on-premises applications to be slightly behind or deliver slightly different calculated values compared to the updated version that was rewritten for the cloud.
5.Test and repeat. Test and repeat
Once your cloud system is up and running, the most important job is to test the new system against the old one to check that you get the same results back from the same queries and transactions. IT teams should build reports that can evaluate this behaviour while the systems are both still running at the same time and show exactly what each query returns. Ensuring that they balance out completely is vital, as only when you reach this point can you start using the new cloud-based system and know that you can trust it. In addition, it is crucial to monitor both the on-premises system and the cloud system. A frequent problem with cloud migrations is that teams fail to monitor their cloud system once it is deployed and consequently find out that they have dramatic cost overruns only after the monthly bill arrives.
One important point to also bear in mind during testing is whether there are any month-end, quarter-end or year-end processes that may change your assumptions or cause problems if they are not tested before teams fully move over to the cloud. Many end of period reporting processes require additional CPU, IO and storage while those reports are being processed. Keeping these in mind at the testing stage helps teams to make sure that all of their processes will work effectively in the cloud regardless of whether they are not needed for several months or even until the following year.
Many organisations are reaping the benefits of moving their on-premises database systems into the cloud. However, migrations from your local data centre into the cloud are by no means simple or guaranteed to succeed. In order to best maximise the chances for a healthy and successful migration, remember the five steps: spend time researching and analysing exactly what you need; spend time doing a deep clean on your data; make sure to fully document the applications and databases you plan to move; run your new and old systems in parallel for a while and finally, test and monitor your cloud system around the clock. By following these five steps, you will get your application and database migrated quickly, efficiently and reliably.
