The data warehouse is an analytical database with data pipelines and business applications to satisfy the organisation’s needs. Warehouse migration is moving data, pipelines, business applications, security considerations, and permission management from on-premise infrastructure to the cloud. In this post, we’ll talk you through the benefits and costs of a data warehouse migration to the Google Cloud.
Google Cloud offers different services for storing data, processing data, and reporting. Every organisation is unique in how they are managing data. Therefore the migration process is unique and depends on many factors that you should take into consideration before starting a migration project. Before you start your preparation activities for the migration, it is important to understand the big picture. Let’s begin with why you would migrate your data warehouse in the first place.
What are the benefits of the data warehouse migration and what is the real business value?
Why migrate your data warehouse from on-premise to Google Cloud
Having your data warehouse in a cloud environment gives your organisation multiple benefits in comparison to on-prem. At Devoteam G Cloud, we’re convinced Google Cloud is the best public cloud to migrate your data warehouse to. Here are the benefits:
Unlock Scalable flexibility for your infrastructure
Make your infrastructure scalable, flexible, and cost-effective. In Google Cloud, the infrastructure can be dynamically changed depending on current needs – increase the number of CPU, memory, or data storage volume. And the same to the other side – compute and storage resources can be reduced so you will pay for what you currently use.
For example, infrastructure capacity can be tuned to successfully pass seasonal loads and then rolled back to regular settings. Treat your infrastructure as a service (IaaS) – dynamically apply changes for fast, secure infrastructure management.
Reduce downtime
Make your services reliable. For example 99.99% SLO of monthly uptime for BigQuery. Google Cloud offers backup and DR capabilities – SQL databases, persistent disks, and geo-redundant storage to implement robust backup strategies.
Increase security
Cloud provides increased security. It may seem that storing your data and software outside of the organisation’s perimeter is less secure, but human error is the prominent factor of data leaks and it does not matter if it is hosted on-prem or in the cloud.
With Google Cloud, you can implement a zero-trust approach and secure access to the cloud resources from untrusted networks without VPN. Having permissions management in a single place, you are increasing the visibility of who can do what.
Cost of Ownership comparison between on-prem and cloud
Google Cloud fees can be estimated in the Google Calculator. The cost of ownership for the server in the cloud may first appear to be higher than the cost for its on-premises equivalent. But it’s actually the other way around. Let’s have a look at the costs that are related to on-prem vs. cloud.
Service life of servers
Self-hosted servers have an average service life of 3-5 years. They should be physically repaired and replaced. For example, a good-quality average single server (64 GB RAM, multi-core, SSD) with no redundancy and backing-up, can run $2K – $8K, if you are going low-end.
Server maintenance
The server should be maintained during its service life – temperature and humidity control, server racks, and spare parts: all of that must be purchased, maintained, and regularly replaced according to the manufacturer’s schedule. The installation of the Air Control System for the server room for example would cost $1K – $2K per square meter, which is approximately $15K – $30K for a 3 by 5 meters server room.
IT staff costs
Regular software updates are required in order to close vulnerabilities and security breaches. So there is a need for dedicated IT staff. The average in-house base year salary of IT support staff is $40K (based on Glassdoor salaries). Add here more variables such as taxes, legally required benefits, sick leaves, ongoing training and certifications, and management expenses.
Software license management
Usually, organisations have more licenses than needed and it is not easy to track the requirements in licenses. A common server OS license would cost $1K, but the good practice is to provide redundancy by having a duplicate server, so let’s consider $2K. Then, the license for every user or client, who accesses the server. With the cost of $40 per user’s license, if you have 10 persons on the IT staff who are working with the server, it turns into $400. Adding here SQL server licenses would take, depending on the organization’s size, $2K and more. So the total is $4.4K without any additional software licenses like antivirus, backing up, etc
Power supply
The average low-mid server consumes approx 300W. Let’s calculate the year consumption, which is 300W * 24h * 365 days = 2.628 MWh. In addition to the server’s consumption, electricity is required for additional equipment such as air temperature and humidity control, power to racks and chassis, and UPS. Average Power Usage Effectiveness (PUE) in data centres is 1.5 In our case, it would be 2.628 * 1.5 = 3.942 MWh per server. Having electricity cost as $0.2 per kWh, it will be 3.942 * 0.2 * 1,000 = $0.8K per year. As a result, we see that the cost of electricity for the server for 3 years can exceed the cost of the server itself
The power supply must be reliable and redundant on critical servers so that it can continue to run even if the main power supply fails, preventing downtime. In February 2021, the snow storm Uri dealt significant power outages in Texas, USA, that lasted 5 days. Is the infrastructure of your company prepared to function without incoming power? The cost of UPS that can feed a regular server for 8 hours is $4K – $10K, and $12K – $30K for the full day accordingly.
Backup and disaster recovery
The backup and disaster recovery mechanisms should be set up. What if a fire starts in the server room and destroys all the equipment and data? Having a reliable data backup is minimally required, but if you want to prevent business downtime, then consider a fully functional disaster recovery site.
You are okay with it if you can answer the following questions: do all of the warehouse components have a recovery plan? Was this plan tested within the last 12 months? Is the disaster recovery site located at an appropriate distance from the production site? Does the recovery site have tested and working temperature and humidity control, uninterruptible power supply, internet service providers, and power generators?
Non-IT maintenance & service jobs
Keep in mind that non-IT maintenance and service jobs also have to be performed, like cleaning service, cables and wires maintenance, air conditioning maintenance, and server room renting. That all requires specialist staff.
Recap of the costs of a single server
These rough calculations were done for the single server and of course, they can vary significantly depending on many different circumstances. Summing it up, the upfront cost of owning the single server is $10K optimistically. The annual cost will vary depending on the number of servers and the server room equipment. According to power supply cost calculations, it will at least be $0.8K just for power for each server.
Big business, big numbers
Numbers don’t look painful for big businesses, but big business works with big data that requires much more than a single server. Now multiply the expenses for thousands of servers – more than $10M upfront cost, the same sum in 3 – 5 years for the infrastructure renewal, $1M annual electricity bill, and double it all in order to leverage the Disaster Recovery plan. And keep in mind IT staff expenses and non-IT maintenance expenses.
Startups and scale-ups
In the case of a startup, every additional thousand dollars can be killing. That’s why Google offers the startups Cloud Program. It is available for no equity funding startups or even those who pass Series-A. The program helps businesses to grow and develop quickly and confidently. The offer includes Google Cloud credits, access to online courses and hands-on labs, access to technical support, and Google Workspace.
Business’ adaptability to rapid changes
In Google Cloud, you pay a single fee and receive compute, storage, backup, and management services. With Google’s pay-as-you-go pricing structure, you only pay for what you use – no prepays, no termination charges.
Let’s consider opportunity costs. The adaptability of a business to rapid changes is the advantage it has over its competitors. For example, Ocado, the UK online supermarket, struggled to keep up with the surge in traffic brought on by the Covid-19 coronavirus pandemic in 2020. Its online services were unavailable for a few days. How much can the downtime of an online supermarket cost per hour?
In the cloud, you can dynamically tune your infrastructure and effectively go through the seasonal loads, and then rollback to regular settings. Here, money is saved twice. The first is when you are preventing downtime due to unexpected (or expected) highload. The second is when you are not paying for extra resources that are released after the peak load passes. Again, costs can be estimated with Google Cloud’s Pricing Calculator – computing power, storage, network traffic, AI/ML operations, and many others.
Sustainability
It is also important to think about environmental impact. Google data centres are more energy efficient than typical enterprise data centres (trailing twelve-month PUE of 1.10 across all large-scale data centres). In addition, Google set a goal to match energy consumption with carbon-free energy by 2030.
Summing it up, Google Cloud gives a flexible, scalable (up and down), reliable and secure solution for your migration to the cloud with easy-to-understand costs. Backups and Disaster Recovery services are available to ensure the data is safe. Having secure global access to resources is another advantage of switching to the cloud, which is crucial nowadays given the shift in work style brought on by the pandemic. A reliable Google Cloud premier partner with experience with this kind of projects, like Devoteam G Cloud, can help you to tackle your data warehouse migration more time- and cost-efficiently.