Choosing a data platform: start with requirements, not features

Why feature comparisons mislead

Platform evaluations often begin with a spreadsheet: vendors across the top, features down the side, a tick in almost every cell. The exercise feels rigorous and tells you very little, because the major platforms can all store large volumes of data, run transformations, serve reports, and support machine learning. Each also releases new capabilities faster than any comparison can keep up with.

The differences that affect you are not on the feature list. They lie in how a platform fits the work you need to do, the people who will run it, and the systems you already have. A platform that is an excellent choice for one organization can be an expensive mismatch for another with different skills and workloads.

Starting with requirements changes the question from "which platform is best?" to "which platform fits us best?". The second question has an answer.

Define the workload first

Describe what the platform must do as concrete workloads, not as capabilities. For each one, capture the following.

• Sources. Which systems supply the data, in what formats and volumes, and how often do they change?
• Freshness. Does the business need this data monthly, daily, hourly, or within seconds? Be honest: real-time delivery is costly, and many decisions are made on yesterday's figures.
• Usage. Who uses the output, and how? Scheduled reports, self-service analysis, data science, and AI applications place very different demands on a platform.
• Constraints. Where must the data reside? Which regulations apply? What has to integrate with existing identity and security controls?
• Growth. What is likely to change in the next two to three years, as distinct from what might conceivably happen one day?

Then separate the essential from the hypothetical. A requirement that supports a committed business need is essential. A capability that might be useful someday is not, and designing for it adds cost and complexity now. A platform can be extended later. An over-built one is much harder to simplify.

Compare the full operating model

Storage and compute are the easiest parts of a platform to compare and the least likely to determine success. What matters over the life of the platform is what it takes to run.

Two of these deserve particular attention. The first is skills: a technically superior platform that your team cannot operate confidently will underperform a simpler one they know well. The second is cost: consumption-based pricing is flexible, but it produces unwelcome surprises unless spend is visible by workload and someone owns it.

Account for what you already have

Few organizations choose a platform from a blank page. There is an existing cloud commitment, an established reporting tool, an identity system, and a team with particular experience. These are not constraints to apologize for. They are among the strongest inputs to the decision.

An organization already invested in the Microsoft ecosystem, with Power BI in daily use, will find some options a more natural extension than others. One with a strong engineering team and data spread across several clouds may weigh things differently. Neither is wrong. The aim is a platform that builds on your position instead of working against it.

This is also where independent advice earns its place. A recommendation should follow from your requirements, and it should be clear about the trade-offs of the option it favors.

Validate before committing

Requirements narrow the field. A proof of concept settles what remains. Its purpose is to test the assumptions that carry the most risk, not to confirm that the platform works in general.

• Choose a representative workload: real data, realistic volumes, and at least one difficult source.
• Agree the decision criteria before starting, so the result is judged against what you said mattered.
• Measure what you will live with: performance, cost for the workload, and how long your own team takes to build and troubleshoot.
• Include a failure. Break a pipeline deliberately and see how easily the cause is found.
• Record the trade-offs, including the reasons for not choosing the alternatives.

A short, focused test costs far less than discovering a mismatch after migration. It also produces a written rationale, which is valuable when the decision is questioned later or when the team changes.

Decide, then migrate in stages

A platform decision is the start of the work. The safest route to value is to migrate in stages: begin with one business area, deliver something people use, and apply what you learn to the next. Each stage should leave the organization better off even if the following one is delayed.

Set up the operating disciplines with the first stage, not after the last. Version control, automated testing, monitoring, cost visibility, and governance are far easier to establish with one workload than to retrofit across many.