AI Data Infrastructure Assessment: Is Your Data Ready?

An AI data infrastructure assessment is a structured check of whether your data can support the AI system you want to build. It answers five questions: does the right data exist, can you legally use it for this purpose, can a model reach it in the form and at the speed the system needs, is it accurate and consistent enough to learn from, and can anyone say where it came from. It takes days rather than months, and it is the cheapest way to discover that a promising AI project is not buildable yet.

The finding that surprises teams is rarely that data is missing. It is that the data exists in seven systems, in four formats, with three incompatible definitions of "customer", and nobody owns the reconciliation. The model was never the hard part. Getting a clean, labelled, legally usable, reliably refreshed table in front of it is the hard part, and it is where the majority of an AI project's real effort lands.

What is an AI data infrastructure assessment?

It is a diagnostic pass over the data layer beneath a proposed AI use case: sources, pipelines, storage, quality, ownership, and legal basis. The output is not a platform recommendation. It is a verdict per use case (ready, ready after specific work, or not viable on current data) plus a costed list of what the gaps would take to close.

It is narrower than a full AI readiness audit, which also weighs feasibility, ROI, regulatory exposure, and whether the organisation can absorb the system. Data readiness is one dimension of that audit, but it is the dimension that most often decides the answer, so it is worth understanding on its own. If you are choosing between the two, the broader audit is the right instrument when you have several candidate use cases and need them ranked; a focused data assessment is right when the use case is already chosen and the open question is whether the data can carry it.

Two things it is not. It is not a data quality project: it measures quality, it does not fix it. And it is not a modernisation programme. Plenty of AI use cases run fine on a decade-old database with a nightly export, and telling a client they need a lakehouse before they can classify support tickets is selling infrastructure, not solving a problem.

Why does data readiness decide whether an AI project ships?

Because data problems are discovered late and priced early. The estimate for an AI build is usually written against the data everyone believes exists. The work is then done against the data that actually exists, and the gap between those two things is absorbed by the schedule.

The pattern is consistent enough to predict. A pilot runs on a hand-cleaned extract that someone pulled once, by hand, into a spreadsheet. The results are good. The project is approved on those results. Then production needs the same data automatically, daily, from the source system, and it turns out the extract involved four manual joins, one judgement call about duplicate records, and a column that a colleague in operations maintains privately. The pilot did not prove the system works. It proved the system works on data that no pipeline currently produces.

This is why data gaps outrank almost every other risk. A feasibility problem can be answered with a literature review. An adoption problem can be worked on with the people affected. A data gap has to be closed by building something, and that build is frequently larger than the AI project it was supposed to support. Several of the most common AI integration mistakes are data problems wearing an engineering costume.

What does "AI ready data" actually mean?

Readiness is not a single score. It is six independent properties, and a use case can fail on any one of them while looking healthy on the other five.

1. Access

Can a system read the data programmatically, on a schedule, without a person in the loop? This sounds trivial and frequently is not. Data locked inside a vendor SaaS product with no export API, a PLC on a factory floor that speaks a proprietary protocol, a reporting tool that renders numbers but will not release them, or a database that operations will not let anything else query during business hours: each of these is an access failure, and each turns a two-week integration into a negotiation.

Check for the boring blockers too. Who owns the credentials. Whether a read replica exists or every query hits production. Whether the export that "runs nightly" has failed silently for a month because nobody watches it.

2. Coverage and representativeness

Does the data include the situations the system will face, in roughly the proportions it will face them? A defect classifier trained on the defects that were photographed will miss the ones nobody bothered to photograph. A demand model trained on 2024 and 2025 has to answer whether those years were normal. A fraud model trained on caught fraud learns to detect the fraud your existing controls already catch.

The specific thing to look for is systematic absence: a class of cases missing not at random but because of how the data was collected. Random gaps degrade a model gently. Systematic gaps produce a model that is confidently wrong in exactly the situations that matter.

3. Quality and consistency

Quality is measurable, and measuring it takes hours rather than weeks. Completeness by column, duplicate rate, timestamp sanity, value ranges outside the physically possible, and the share of records that fail a business rule everyone assumed was enforced.

Consistency is the harder half. It is the question of whether the same field means the same thing across systems and across time. A status code that was repurposed in 2023. A "region" that finance and sales define differently. A unit of measurement that changed when a supplier changed. Models absorb these inconsistencies as signal and produce behaviour nobody can explain later.

4. Labels and ground truth

Supervised learning needs examples of the right answer, and most organisations have fewer than they think. The question is not whether outcomes are recorded somewhere but whether they are recorded in a way that links back to the input, with the timing right.

Two failure modes recur. The first is label leakage: a field that is only populated after the outcome is known, which makes the model look excellent in testing and useless in production, because at prediction time that field is empty. The second is disagreement: two experts label the same case differently, which caps model accuracy at the level of human agreement no matter how good the architecture is. If you have never measured inter-rater agreement on the labels, you do not yet know what accuracy target is achievable.

Where labels do not exist, the assessment should price creating them, because that cost is real and it is usually the largest single line item in the project.

5. Lineage and governance

Lineage is the ability to answer where a number came from, who transformed it, and under what legal basis you hold it. It matters commercially, because a model nobody can trace is a model nobody will trust with a decision that carries consequences. It matters legally, because both GDPR and the EU AI Act ask questions that lineage answers and nothing else does.

Practical checks: does a named owner exist per source, is personal data identified and its lawful basis documented, are retention periods defined and actually applied, and is there a record of consent or contractual basis for using this data to train a model, which is often a different purpose from the one it was collected for. That last point catches people. Data lawfully collected to deliver a service is not automatically lawful to use as training data, and the overlap between GDPR and the EU AI Act is where that question gets decided.

6. The serving path

Training data and production data are different problems. A model trained on a warehouse table has to run, at inference time, on data arriving through a live path with its own latency, availability, and failure modes. Assess that path explicitly: how fresh does the input need to be for the prediction to be useful, what happens when a source is late or down, and does the same transformation logic apply in both places.

When training and serving apply different transformations, you get training and serving skew, a class of bug that produces a model that scored well offline and quietly underperforms in production for months before anyone works out why.

How do you assess data quality quickly?

By probing rather than surveying. A full data catalogue takes months and answers questions you did not ask. A targeted probe takes days and answers the one that matters: can this specific use case be built on this specific data.

The sequence that works:

• Start from the decision, not the data. Write down the prediction the system must make, at what moment, and what information a human would have at that moment. That defines the feature set and immediately eliminates any field that will not exist at prediction time.
• Pull a real sample, end to end. Not a curated extract. Twelve months of production data through the actual access path you would use, so the extraction friction shows up in the assessment rather than in month three.
• Profile it mechanically. Null rates, cardinality, duplicates, distributions, time series continuity. Look specifically for step changes in a distribution, which usually mark a system migration or a definition change rather than a real world event.
• Reconcile against a known truth. Take a figure the business already trusts, monthly output, revenue, defect counts, and rebuild it from the raw data. If you cannot reproduce a number the organisation reports every month, you have found the gap before the model did.
• Test the labels. Have two domain experts independently label the same fifty cases and measure how often they agree. This one exercise sets the realistic accuracy ceiling for the entire project.
• Try a weak baseline. Fit something deliberately simple, a logistic regression or a gradient boosted tree on obvious features. If a weak model finds no signal at all, more sophisticated architecture rarely rescues it. If a weak model performs suspiciously well, look for leakage.

That last pair is the highest value hour in the whole assessment. A weak baseline is both a floor and an alarm.

Do you need a data warehouse before you start with AI?

Usually not. The requirement is a reliable, repeatable path from source to model, and a scheduled export into object storage satisfies that for a large share of first projects. Warehouses and lakehouses earn their keep when many use cases share the same data, when governance has to be centralised, or when analysts need self-service access. Those are good reasons. "We need it before we can do AI" is not one.

The sequencing argument runs the other way, in fact. Building a platform before a validated use case means designing for requirements you are guessing at. Building the first use case on a modest pipeline teaches you what the platform actually needs to do, and the pipeline you write for it is rarely wasted. The exception is when the assessment finds the same data gap blocking three or four candidate use cases at once. At that point the shared foundation is the cheaper path, and it should be built deliberately rather than as a side effect.

Beware the reverse failure too. A team that spends eighteen months on a platform and has shipped no AI has not de-risked anything. It has converted an AI project into an infrastructure project, and infrastructure projects are judged on delivery dates that data platforms are historically bad at meeting.

What does the EU AI Act require of your data?

For high-risk systems, the EU AI Act sets explicit obligations on the data layer. Training, validation, and testing data sets must be relevant, sufficiently representative, and to the best extent possible free of errors and complete for the intended purpose. Providers must apply data governance practices covering collection processes, data origin, preparation steps, formulation of assumptions, assessment of availability and suitability, and examination for bias that could affect health, safety, or fundamental rights.

Read as engineering requirements rather than legal text, those obligations translate into things a data assessment can check now: documented provenance per source, a written record of preparation and transformation steps, a bias examination against the groups the system will affect, and evidence that the data reflects the setting the system will be deployed in. Article 12 logging obligations add a further requirement, that the system records what it did, which is a pipeline design decision and a painful thing to retrofit.

Whether your system is high-risk depends on its intended purpose, and that is worth settling early because it changes the data architecture. Classifying your AI system under the risk framework covers how that determination is made, and retention requirements for AI system documentation covers how long the resulting records have to survive. In regulated settings such as healthcare, MDR and IVDR obligations sit on top of all of this and raise the evidence bar further. This is readiness work, not legal advice, but doing it before the architecture freezes is the difference between designing for compliance and rebuilding for it.

What are the most common data infrastructure gaps?

Across assessments, the same handful recur:

• No ground truth. Outcomes are known informally but never recorded against the case that produced them. Common in maintenance, where the repair is logged but the failure mode is not.
• Manual steps hidden inside the pipeline. A spreadsheet, a rename, a person deciding which duplicate to keep. Invisible in the pilot, blocking in production.
• No historical depth. The system that holds the data overwrites rather than appends, so there is a current state and no history to learn from. Discovering this in week one is a good week. Discovering it in month four is not.
• Definition drift. A field whose meaning changed without a migration, leaving two eras of data in one column.
• Undocumented personal data. Free text fields containing names, notes, or identifiers that nobody classified, which surface during the compliance review and stall the launch.
• No monitoring. Pipelines that fail silently. If nothing alerts when yesterday's load did not arrive, the model is quietly serving predictions on stale input.
• Sensor and edge data that never leaves the machine. Frequent in manufacturing, where the signal exists but the connectivity to collect it does not, and the collection project is the actual first project. Predictive maintenance lives or dies on exactly this.

A practical data readiness checklist

Score each item for one specific use case. Anything answered "no" or "unknown" is a work item with a cost attached, not a detail to resolve later.

• The prediction, its timing, and the information available at that moment are written down.
• Every required field is reachable through an automated path with a named owner.
• At least one full business cycle of history exists, with known gaps documented.
• Ground truth exists, links to the input record, and was available only after the prediction point.
• Inter-rater agreement on labels has been measured, or the labels are objective.
• A profiling report exists: nulls, duplicates, ranges, distribution shifts over time.
• A business figure has been reproduced from raw data and reconciles.
• Personal data is identified, with a lawful basis for this specific purpose and a defined retention period.
• The production serving path is described, including freshness requirements and failure behaviour.
• Training and serving share one transformation implementation.
• Pipeline failures raise an alert that a named person receives.
• A weak baseline has been fitted and its result reviewed for both signal and leakage.

Twelve items, most of them answerable in a day or two each. The value is not in the score. It is that every "unknown" becomes visible before it becomes a schedule overrun.

What do you do when the data is not ready?

Not ready is a finding, not a verdict. Three responses are usually available, and the assessment should say which applies.

Narrow the use case. Frequently the data supports a smaller version of the same idea. Instead of predicting failures across the whole plant, predict them on the two machine types with instrumented history. A narrow system that ships beats a broad one that waits.

Run a collection phase first. If the gap is labels or history, the first project becomes recording the data properly, with the AI build scheduled once enough has accumulated. This is an unglamorous recommendation that saves entire budgets, and the timeline should be stated honestly: if you need a year of seasonal data, you need a year.

Change the approach. Some gaps are architectural rather than absolute. Weak supervision, transfer learning, synthetic augmentation, or a rules based system with human review can bridge a sparse label situation. Each has costs and each is a legitimate answer, provided it is chosen deliberately rather than to avoid delivering bad news.

What none of the three is: proceeding anyway and hoping the data improves. That is the path described in how to avoid AI project failure, and it ends the same way each time.

The honest summary

Data readiness is the constraint that determines what AI you can build, and it is knowable in advance for a fraction of what it costs to discover mid-build. The assessment is deliberately unglamorous: pull the real data through the real access path, profile it, reconcile it against a number the business trusts, measure whether your experts agree with each other, fit something simple, and write down what is missing.

What comes out is a use case that is buildable now, one that is buildable after specific and priced work, or a clear statement that the data cannot carry the idea yet. All three outcomes are useful. Only one of them is expensive to learn later.

If you are still deciding whether AI belongs on the roadmap at all, the earlier signals of AI readiness are the cheaper first read, and realistic machine learning ROI sets expectations for what a working system returns. Once the data question is settled, the integration work that follows spends its budget on genuine uncertainty rather than on rediscovering that the numbers never reconciled.

Frequently asked questions

Do we need a data warehouse before starting with AI?

Usually not for a first project. A warehouse helps analytics at scale, but a single well-understood source is often enough to prove a use case. Building a warehouse first delays the AI work by quarters and frequently optimises for reporting questions rather than the ones a model needs answered.

How long does a data infrastructure assessment take?

Typically two to four weeks, depending on how many systems are in scope and how quickly access is granted. Access approvals are usually the long pole rather than the analysis, which is why identifying the data owner in week one matters more than any technical step.

What if our data is spread across systems that do not talk to each other?

That is the normal starting condition, not a disqualifier. The assessment establishes which joins are actually required for the use case rather than assuming everything must be unified. Most first projects need two or three sources connected, not a company-wide integration programme.

Can we assess readiness without giving an external partner access to production data?

Yes. Schemas, row counts, field descriptions, and a small anonymised or synthetic sample answer most readiness questions. Full access matters when you move from assessment to building, and by then the scope is narrow enough to justify a proper data agreement.