Seamless Data Access, Worry-Free Transformation! ZStack Database Migration and Replication Platform DMS in Practice

Introduction

In the previous issue, we initially introduced how to systematically perform migration and the quick usage of ZMigrate. However, users may also face massive, mission-critical, and heterogeneous database instance replication and migration in practice. Especially under the domestic IT innovation initiative, multiple domestic database vendors continuously deliver new solutions and products to clients. For customers, migrating directly from familiar systems like Oracle, PG, MySQL, and MSSQL to domestic databases such as Kingbase, DM, GaussDB, and OceanBase carries extremely high risks. To minimize risks, alleviate customer concerns, and serve as an effective supplement to whole-machine migration, we have launched a standalone database migration tool to help customers complete such heterogeneous migrations with minimal cost.

System portability has become a standard recognition among architects

As CTO/CIO/ architect, we often face the following scenarios:

• The company adjusts its strategy, and the IT infrastructure needs to be migrated from a foreign public cloud provider to a domestic public cloud provider.
• Considering security or cost factors, the company plans to migrate from a public cloud to a self-built private cloud platform.
• Public cloud vendors engage in price undercutting competition, prompting the company to migrate to a public cloud platform offering deeper discounts.
• Aging servers approaching end-of-life or data center relocation necessitates system migration to a new cloud platform.
• Existing business systems require migration from legacy cloud platforms to newIT innovation cloud platforms..

Especially in the current landscape where IT innovation vendors are proliferating like blooming flowers, system portability is destined to become a crucial element in designing future enterprise-level architectures, and is gradually emerging as standard knowledge among CTOs/architects. During migration processes, we typically categorize systems into the following two major classifications:

For stateless service migration, you can refer to the previous article Business Migration? Fear Not! ZStack ZMigrate Migration Software Best Practices.

For stateful cross-platform database migration, especially for distributed or strongly consistent databases relied upon by critical services, whole-machine migration solutions face significant challenges:

• Long system downtime: Often requiring 10-20 minutes or even longer, while customers demand near-zero downtime solutions.
• High migration failure risk: When dealing with distributed databases like TiDB, post-migration startup failures may occur.
• Cross-OS migration limitations: Original databases running on CentOS need to be migrated to Ubuntu or Kylin OS.

As the saying goes, “Experts excel in their specialized fields.” For complex stateful databases, ZStack provides a professional migration platform solution – DMS (Database Migration Service). DMS offers cross-platform, cross-version, cross-type full-incremental integrated real-time synchronization capabilities for dozens of mainstream databases.

Core Advantages of DMS

Advantage I: Business-Transparent Migration

When migrating databases, administrators select appropriate migration methods based on business requirements. The typical process for full-server migration is as follows:

1. Stop Database to halt the database (ensuring zero writes), which usually takes 3-5 minutes or longer. Stopping the database waits for ongoing transactions to complete or roll back, triggering a Full CheckPoint operation to flush dirty pages. This process becomes slower with higher database Insert Buffer and write workloads.
2. Initiate the final incremental filesystem replication, typically requiring 1-2 minutes.
3. Start the new virtual machine on the target side, taking 2-3 minutes.
4. After the new virtual machine starts, launch the database within it to load data files and index metadata. This step requires 3-5 minutes.

Higher database write pressure leads to longer downtime (often 10-20 minutes or more), during which services remain inaccessible.

For scenarios requiring high business continuity, a pre-built new database can be deployed with DMS to handle replication, traffic redirection, and migration while ensuring uninterrupted database access. DMS integrates full and incremental synchronization: after completing full data transfer, it uses Log Ship technology (similar to database primary-secondary replication) to parse source database logs (e.g., MySQL Binlog, Oracle Redo, PostgreSQL WAL), convert them into DDL/DML statements, and execute them on the target database. This achieves near-zero-latency data replication (mimicking primary-secondary replication effects).

As shown in the figure below:

This approach easily redirects read traffic to the new database. Pre-launch production grayscale testing ensures the new database meets QPS capacity and RT requirements for switching. Service cutover only requires stopping legacy applications and redirecting data sources to the new database address (JDBC/ODBC URL), enabling seamless and transparent migration.

Advantage II: Migration Across Heterogeneous Environments or Platforms

Similarly, DMS can handle migrations across various heterogeneous environments or databases, such as the following scenarios:

• A system previously using MySQL 5.0, where customers want seamless migration to MySQL 8.3 to resolve weak query performance issues.
• Databases deployed in CentOS systems, where customers require the target database kernel to be Ubuntu or Kylin V10.
• Databases deployed on Windows that customers want to migrate to Linux virtual machines.
• Systems running in Linux virtual machines that customers aim to migrate to target databases operating on k8s.
• Databases running on cloud vendors’ RDS (e.g., Huawei/Aliyun) where DMS can smoothly migrate the source RDS to ZStack RDS or cloud platforms even without OS permissions, retaining only database access rights.

Beyond these scenarios, DMS addresses challenges in database cluster migrations—such as Oracle RAC, distributed GaussDB/OceanBase, SQL Server distributed clusters, or Raft-based three-node MySQL clusters. Traditional whole-machine migration approaches may fail to start target database clusters due to node sequence issues. In contrast, DMS leverages logical database replication to perfectly resolve node consistency problems in such cluster migration scenarios.

Core Advantage III: Lightweight and Non-Intrusive Migration Process

Compared with other database migration solutions, DMS requires smaller data volumes and achieves higher efficiency during migration. It eliminates the need to install any Agent, minimizes system intrusion, and avoids additional investigations into OS version compatibility or migration tool issues.

Taking Oracle migration as an example: Consider a 400GB Oracle database typically composed of – Data tablespace 50GB, Index tablespace 150GB, Online Redo 10GB, Archive Log 140GB, Temp tablespace 20GB, and Undo tablespace 30GB. DMS only needs to migrate the 50GB of data in the data tablespace to successfully launch the new database, making it particularly suitable for scenarios requiring rapid business testing and validation.

Core Advantage IV: Flexible Data Combination  

DMS can specify a particular database or table within the system for split, merge, or migration operations, offering finer migration granularity. For example:  

• The source is a high-configuration host (64Core + 256GB RAM) running multiple MySQL databases. The customer may require phased migration from non-core to core systems for gradual validation.  
• Alternatively, the source may involve 20 low-configuration virtual machines hosting 20 instances. The customer might aim to consolidate the workloads of these 20 database instances into 4 high-configuration virtual machines to reduce node count and resource overhead. During this process, operations like sharding and merging multiple databases into one large database can even be implemented.

Appendix: Three Steps for DMS Database Migration  

We take the migration of a MySQL instance deployed on VMware to ZStack Cloud as an example.  

1. Set Up Data Source Connection  

Configure the source MySQL address (on VMware) and target MySQL (on ZStack).  

2. Create Data Migration Task

Select the migration destination:  

Choose the data source to migrate.  

Among 4 databases under the source, select the trade database.  

Enable CDC real-time replication.  

3. 3. Start Migration Task  

Select tables for real-time migration and initiate replication.  

Monitor migration progress, parallelism, and duration for large tables.  

Check the timestamp of incremental replication. When the incremental Stream replication rate reaches 0, it indicates real-time consistency between source and target databases.