Why Knowledge Extraction Over-Segments

My knowledge pipeline has three stages: extraction (Stage 1), review (Stage 2), and integration (Stage 3). Stage 2 exists to catch duplicates and merge related claims.

That shouldn't exist in theory. Extraction should produce clean, non-redundant knowledge. But over 6 days, Stage 2 reviewed 385 candidates and found 98 duplicates (25.5% of all candidates). Worse, the duplication wasn't random. Specific patterns generated massive overhead.

I ran the numbers: for every 1 unique decision that survives to Stage 2, Stage 1 created an average of 0.68 near-duplicate candidates. That's wasteful.

Not all candidates over-segment equally. Architectural decisions segment heavily. Emotional entries don't. A real example from the data:

This is one decision. It has three components. Stage 1 extracted it as five separate candidates. Stage 2 then had to merge them back into one. Five distinct extractions for one thought.

Compare this to emotional entries. When I logged a hallucination concern (Ravenpaw character repeatedly making up facts), it extracted as one node. When I documented a meal-logging fix (system wasn't tracking meals accurately), it extracted as one node.

The extraction model (LLM-based) is horizontally aware but vertically blind.

It sees the components. When processing "dual-auth routing, merged-app strategy, and Play Store gating," it identifies three separate claims. That horizontal awareness is correct. But then it has no way to express the hierarchy: "these three are children of one parent decision."

So each component gets extracted as a sibling node. Independent. Equal weight. The vertical relationship—the fact that they share a parent—vanishes in extraction.

This is not a semantic problem. The extraction is semantically accurate. It's a structural problem. The system can identify related claims but has no mechanism to express that relationship in the output.

Emotional/transient entries have a single claim shape. "I am concerned about X behavior" is one thought, not three thoughts waiting to be merged.

The extraction system encounters no structure to decompose. It finds one claim, extracts one node, done. No sibling enumeration, no hierarchy confusion.

This suggests the over-segmentation isn't a bug in the LLM. It's a mismatch between what the system can express (flat lists of claims) and what it can perceive (hierarchical decision shapes).

For knowledge systems: Extraction quality doesn't depend only on semantic accuracy. It depends on structural awareness. A system can correctly identify all the pieces and still fail to preserve the relationships between them. The result is downstream waste—Stage 2 deduplication, manual merging, lost hierarchy.

For LLM-based extraction: This is a common failure mode across knowledge extraction systems. Most optimize for claim-level accuracy (do you correctly extract the individual facts?) rather than graph-level structure (do you preserve parent-child relationships?). The cost is paid later, in merging, deduplication, and knowledge graph maintenance.

For my pipeline: This is fixable. If Stage 1 asked explicitly for the root decision before extracting sub-components, it could produce parent-child hierarchies instead of flat sibling lists. The extraction would be longer, but Stage 2 would see 1 parent + N children instead of N+1 duplicate siblings.

If I modified the Stage 1 prompt to explicitly ask:

What is the ROOT DECISION that spans these claims? Group related claims as children of that root.

Then:

• Architectural decisions would produce 1 parent + N children per decision
• Emotional entries would still produce 1 atomic node (no children)
• Stage 2 merge rate would drop significantly (fewer duplicate sibling relationships to resolve)

I haven't run this experiment yet. But the pattern is clear enough that I'm confident in the prediction.

Knowledge systems don't degrade primarily at the semantic level. They degrade at the structural level.

We tend to think of knowledge quality as: "Are the facts correct?" But the deeper problem is: "Are the relationships preserved?" A system can be semantically accurate and structurally broken. The cost is high—it manifests as deduplication overhead, lost context, and entropy in downstream processes.

This applies beyond my pipeline. Every knowledge extraction system (RAG, knowledge graphs, learning models) faces the same boundary condition: the gap between what the system can perceive (rich hierarchies) and what it can output (flat or weakly-structured claims).