How Relata works
Ingest, derive, connect, retrieve — with the reasoning kept and shown at every step.
The pipeline
- Ingest. The original bytes are stored verbatim and never rewritten. An exact duplicate is detected and reported rather than silently stored twice.
- Parse. Text, Markdown, JSON, PDF and DOCX become extracted text plus a structure map. The parser version is recorded on the item.
- Chunk. Structure-aware splitting, so a chunk is a passage rather than an arbitrary window. Chunk IDs are deterministic and double as vector IDs.
- Embed. Each chunk is embedded locally; the model that produced each vector is recorded, so a model change is visible rather than silent.
- Extract. Entities and claims, deterministically where the text allows and with model assistance where it does not — each labelled with which produced it.
- Discover. A bounded candidate set is scored against explicit signals. There is no all-pairs comparison and no unexplained number anywhere in it.
Every stage is a queued job with a lease rather than a lock, so a worker that dies has its work returned to the queue rather than holding it forever.
How a relationship gets its score
The score is the weighted sum of the signals that fired, capped at 1.00, under a versioned scoring model. Each signal is stored with its own value, strength and weight, so the total is arithmetic you can check.
| Signal | Weight | What it means |
|---|---|---|
content_hash_match |
0.45 | Byte-identical content. |
semantic_similarity |
0.30 | How alike the wording is, rebased so that ordinary similarity counts for little. |
shared_entity |
0.25 | An exact entity match, discounted by how common that entity is. |
claim_support |
0.22 | Two statements agree about the same subject. |
claim_contradiction |
0.22 | Two statements disagree about the same subject. |
model_classification |
0.15 | A language model's opinion. One signal, weighted below the storage threshold, so it can never create a relationship on its own. |
shared_concept |
0.12 | Overlapping concept vocabulary. |
alias_entity |
0.10 | A match through an alias a person confirmed. Unconfirmed candidates count for nothing. |
temporal_proximity |
0.10 | Created close together in time. |
shared_metadata |
0.08 | Matching metadata you supplied. |
user_confirmation |
1.00 | A person confirmed it. This dominates everything else. |
user_rejection |
-1.00 | A person rejected it. This drives the score to zero. |
Three deliberate judgements
- Similarity alone cannot look like evidence. Unrelated documents from one corpus routinely sit at 0.5–0.7 cosine, so raw similarity carries almost no information at the low end. It is rebased against a floor of 0.55 before it counts.
- A model cannot manufacture a relationship. Its weight is below the storage threshold, so a maximally confident classification still cannot create a link. It can shape a relationship other evidence already supports.
- Human decisions dominate. Confirmation saturates the score; rejection drives it to zero. Neither deletes anything — the original proposal, its provenance and its signals are all retained.
Diminishing returns
Repeated signals of one type contribute 1.0, then 0.5, then 0.25, then nothing. Breadth of evidence beats repetition: a document mentioning one company twelve times should not outrank one that relates on several independent grounds.
Why each search result was returned
A query runs through several independent paths — keyword, vector, entity, title, and expansion through relationships — each producing its own ordering. They are fused by reciprocal rank rather than by adding scores, because a keyword score and a cosine distance are not on comparable scales and whichever had the larger range would dominate.
Every result carries its reasons in plain words:
- A section of this item is close in meaning to your query (similarity 0.89).
- The item's text contains words from your query.
- This item refers to Example Ltd (organisation), which your query names.
- Not a direct match. Reached because this item updates a matching item — strength 0.68 (suggested by analysis, not yet reviewed).
Relevance is normalised against the best result in the same response, so it is comparable within one result set and not across queries. The interface says so rather than leaving you to assume otherwise.
Answers from stored knowledge
These hold regardless of what the model returns:
- The model sees only retrieved passages, inside delimited untrusted blocks.
- Passages are labelled
S1,S2… — item IDs are never shown to the model, so a fabricated citation cannot resolve to a real record. - An answer claiming sufficiency while citing nothing is downgraded to insufficient.
- Contradictions Relata recorded between the retrieved items are surfaced independently of the model's own account, so a model that quietly picked a side cannot hide the disagreement.
- Inferred relationships that influenced retrieval are listed as proposals, not facts.
- If a passage tried to instruct the model, that is reported rather than hidden.
- A model that returns nothing usable costs you the prose, not the retrieval: the sources and conflicts still come back, with the reason the answer was rejected.
Your workspace is your own
Each person who signs in gets a private workspace. No document, entity or relationship is shared with anyone else, and the boundary is enforced in every storage query rather than in a filter that could be forgotten.
The cost is worth stating plainly: relationship discovery never spans users. Two people who upload documents about the same contract get no link between them, because as far as Relata is concerned they are separate corpora.
Known limitations
- Scores are heuristic strengths, not calibrated probabilities. The weights were chosen by judgement, not fitted to labelled data.
- Deterministic entity extraction is narrow: organisations with a legal suffix, people with an honorific, email addresses and dates. Broader coverage comes from the model stage and is labelled as such.
- Contradiction detection compares figures and dates about the same coarse subject. It will not catch a contradiction expressed entirely in prose.
- No OCR. A scanned PDF is preserved and flagged as having little extractable text.
- Answer generation on a CPU-only machine is slow. A small model produces answers that are often not worth reading, though the safeguards still hold.
- Prompt-injection defences bound the damage; they are not a proof. Treat generated answers as summaries of sources, not as authority.