Skip to content

03 — Concept Ontology

Jupyter notebook

This page shows the code of 03_concept_ontology.ipynb without outputs: the notebook needs a running service to execute and is not run at build time.

View on GitHub · Open in nbviewer · Open in Colab · Raw .ipynb

Kernel: Python 3 · Environment: fractal_rag package (pip install -e fractal_rag) with a running Weaviate instance · Companion page: fractal-rag/

The Concept side-collection is the lightweight ontology layer that the ingest pass uses to tag every artifact. Each Concept carries:

  • a canonical name (e.g. Hausdorff Dimension)
  • aliases for synonym matching
  • a one-paragraph definition
  • domain tags (e.g. dimension-theory, classics)
  • an optional hausdorff_dim_range for classical fractals

Every FractalArtifact has a concepts cross-reference to whichever concepts it mentions. This notebook walks the ontology and shows how to follow those cross-refs.

from fractal_rag.client import client_session
from fractal_rag.config import get_settings
from weaviate.classes.query import Filter

settings = get_settings(refresh=True)

List concepts by domain

with client_session() as client:
    coll = client.collections.get(settings.concept_collection)

    # All concepts in the 'dimension-theory' domain.
    response = coll.query.fetch_objects(
        filters=Filter.by_property('domain').contains_any(['dimension-theory']),
        limit=50,
    )

print(f'Found {len(response.objects)} dimension-theory concepts:')
for o in response.objects:
    p = o.properties
    rng = p.get('hausdorff_dim_range') or []
    rng_s = f"  [D_H={rng[0]}]" if len(rng) == 1 else (f"  [D_H in {rng}]" if rng else "")
    print(f'  - {p["name"]}{rng_s}')

Semantic search across definitions

If you don't know the canonical name, search_concepts on definition_vec works as a free-text lookup.

QUERY = 'measure of long-range dependence in a time series'

with client_session() as client:
    coll = client.collections.get(settings.concept_collection)
    response = coll.query.hybrid(
        query=QUERY,
        limit=5,
        target_vector='definition_vec',
    )

for o in response.objects:
    p = o.properties
    print(f'- {p["name"]}: {p["definition"][:120]}...')

Find all artifacts referencing a concept

The cross-reference is FractalArtifact.concepts -> Concept[]. Filter using Filter.by_ref('concepts').by_id().equal(<concept_uuid>).

CONCEPT_NAME = 'Mandelbrot Set'

with client_session() as client:
    concepts = client.collections.get(settings.concept_collection)
    hits = concepts.query.fetch_objects(
        filters=Filter.by_property('name').equal(CONCEPT_NAME),
        limit=1,
    )
    if not hits.objects:
        raise SystemExit(f'concept {CONCEPT_NAME!r} not found')
    cuid = str(hits.objects[0].uuid)
    print(f'Concept UUID: {cuid}')

    art = client.collections.get(settings.artifact_collection)
    refs = art.query.fetch_objects(
        filters=Filter.by_ref('concepts').by_id().equal(cuid),
        limit=10,
    )

for o in refs.objects:
    p = o.properties
    print(f"  [{p.get('source_type'):>10}] {p.get('title')[:70]}")

Sanity-check the tagging quality

A quick eyeball check: list the 10 most-tagged concepts (i.e. those referenced by the most artifacts). Concepts that no artifact ever cites are candidates for removal from seed.yaml; concepts that only ever appear in 1-2 artifacts may be too narrow.

from collections import Counter

with client_session() as client:
    concepts = client.collections.get(settings.concept_collection)
    all_concepts = concepts.query.fetch_objects(limit=200).objects

    art = client.collections.get(settings.artifact_collection)
    counts = Counter()
    for cobj in all_concepts:
        n = art.aggregate.over_all(
            total_count=True,
            filters=Filter.by_ref('concepts').by_id().equal(str(cobj.uuid)),
        ).total_count
        counts[cobj.properties['name']] = n

print('Top 10 most-tagged concepts:')
for name, n in counts.most_common(10):
    print(f'  {n:>6}  {name}')
print()
print('Untagged concepts:')
for name, n in counts.items():
    if n == 0:
        print(f'  - {name}')

Where to go from here

  • The concepts/seed.yaml file is the source of truth for this ontology. Add new entries (with aliases) and rerun fractal-rag ingest --source all to retag every artifact.
  • The fractal-explorer subagent uses these tools (list_concepts, get_concept, artifacts_by_concept, search_concepts) under the hood for any concept-focused question — see Using the agent.