6. Checking a hierarchy

In chapter 5 you learned four questions to ask about a category. Now we use them.

A hierarchy went through three design reviews. A principal architect drew it. Two senior modelers approved it. It shipped. Eighteen months later a data quality programme was funded to deal with what it caused.

Nobody was careless. The problem is that reading a hierarchy and checking a hierarchy are different activities, and most organizations only ever do the first.

The four rules

A reminder of the symbol. ψ ⊑ φ means "every ψ is a φ", so φ is the parent and ψ is the child.

Rule 1, from rigidity. An anti-rigid parent cannot have a rigid child.

ψ ⊑ φ  and  ~R(φ)  →  ψ must also be ~R

Because being under a category means it holds for all of them, permanently. Put something permanent under something temporary and you have claimed the permanent thing ends when the temporary one does.

Rule 2, from identity. If two categories decide sameness in incompatible ways, neither can be under the other.

If a parent says two records are the same when their tracking numbers match, and a child says two records are the same when their barcodes match, there is no coherent answer to whether two children are one thing.

Rule 3, from unity. Something that is a whole cannot sit under something that is not.

Rule 4, from dependence. Something that needs another thing to exist cannot be the parent of something that does not.

Run them

Read the six lines below before you press anything. Decide which ones you would have questioned in a review.

Understanding the two failures

The rigidity failure is obvious afterwards and invisible at the time, because the thinking that produces it is so sensible. The customer table holds the name, the email, the address. Those are facts about people. So Person must come from Customer.

Notice what happened there. The reasoning was about where the columns are stored, which is a storage question. It silently answered a completely different question, which is what kind of thing is a kind of what.

The identity failure is more interesting, because it is not really a hierarchy mistake at all.

A pallet is not a kind of shipment. A pallet is a piece of a shipment.

Those are different relations, and only one of them is a hierarchy. Model a part-of as an is-a-kind-of and every query that walks the hierarchy will treat pallets as shipments, and every count you care about goes up.

What this method does not do

Be honest about the limits, because someone will push.

It tells you a line is wrong. It does not tell you what the right model is. It has nothing to say about whether Freezer should exist at all, whether your properties are sensible, or whether the model answers any question anyone actually has. A hierarchy can pass every rule and still be useless.

It also depends entirely on answering the four questions honestly. Say Customer is rigid because it feels permanent in your business, and the check will happily approve the broken hierarchy. That is why chapter 5 spent its time on the questions rather than the rules.

There is no hierarchy, which changes the job

Say this plainly rather than glossing over it.

In the shipped Fabric IQ ontology item, entity types do not sit under one another. There is no is-a-kind-of between them, no inheritance, no hierarchy in the strict sense. You declare entity types, give them properties and a key, bind them to OneLake, and connect them with relationship types. That is the whole model.

So there is nothing for the check above to run against.

What survives, and matters more here, are two questions the check forces you to ask.

Should these two entity types actually be one? If Customer and Person both exist, bound to different tables, the identity question decides whether they are two concepts or one concept seen twice. Get it wrong and your graph holds two nodes for one human, and every path through them counts twice.

Is this relationship really an is-a-kind-of? Fabric gives you exactly one way to connect two entity types: a named, directional relationship bound to key columns. So "is a kind of" and "is a piece of" look identical and behave identically when something walks the graph. Nothing stops you asserting either. The difference lives entirely in your discipline.

The one trap

Do not run the check on the model you are about to ship. Run it on the model you are about to build.

Every failure it catches is free on a whiteboard. The same failure caught after entity types are bound, graphs are built, and an agent has been answering from them for a quarter is a rebuild.

Next, chapter 7. You can now check a hierarchy. Chapter 7 is about the single most common bad entity type in enterprise data. You almost certainly have it, and calling it a type at all is the mistake.