At a certain point, they’re almost synonyms: robustness and functionality. People often intertwine the two, claiming that complexity leads to redundancy and redundancy leads to utility. I’m not one to argue with that. A Saturn V rocket built from bamboo sticks would be a sight to see, and a factory-built from plywood would be anything but resilient. But there’s a time when simplicity trumps complexity. When efficiency is derived not from nuance, but from candor.
We battled with this duality of sophistication and clarity when working on the migration of a pretty large Oracle infrastructure to the cloud. The database was huge, with myriad ETL layers ingesting & transforming various types of datasets from various sources. Given the nature of the product we supported, everyone started to think BIG and design the most resilient architecture possible. There seemed to be no way that could fail. Sure, it’s a valid approach, but maybe not the right one this time. As such, my responsibilities consisted of constant reminders: “keep it simple!”
It was a quote riddled with nuisance that I used in every possible circumstance. Through some extensive architectural discussions with the team, particularly involving the use of AWS’s DMS (Data Migration Service) for managing the ongoing production ETL job, we arrived at a conclusion. Maybe the most complex and promising solution wasn’t the right solution. DMS was proof of that. Keep it simple. Were prodigious delivery times and wasteful overhead maintenance worth an overwhelming architecture?
AWS’s DMS is a managed service that enables customers to migrate on-prem databases to AWS-managed databases. DMS aids data migration from rather large Oracle or Microsoft SQL servers to AWS’s RDS (Relational Database Services), Amazon Redshift, or Amazon S3 (Simple Storage Service). As an owner of data ingestions, I came out with a radical—I use that ironically—proposal of using DMS for ongoing data ingestion, essentially continuing with data ingestions on on-prem using the existing ETL jobs, and implementing a DMS for replicating that incremental data to cloud.
This concept was somewhat ludicrous to the team, as it was tantalizingly simple. Maybe even too easy. Either way, we got the team engaged in a few quick PoCs to gauge DMS’s viability, and sure enough, it checked out. The approach gained traction with management and helped dramatically minimize the migration timeframe, providing rapid data transfers to customers on the cloud.
You don’t need reinforced concrete and rebar to build a pencil—I assure you, it’d be rather difficult to write with—and you don’t need a lofty architecture for data migration or IT in general. The core of digital transformations lie within the prospect of delighting customers, and the simple way is often the best way to do so. So let’s stop making concrete pencils—and make working ones first.
