Hosted VM (K3s + GPU)¶

This page covers the production-grade deployment: a single VM (CyVerse Jetstream-2, an on-prem GPU node, or any host with NVIDIA drivers) running K3s, with Weaviate 1.35 and the snowflake-arctic-embed-l-v2.0 text2vec-transformers sidecar pinned to a GPU.

This is the topology the k3s-deployment/*.yaml manifests in this repo target. Once it's up you ingest the full ~50K-article corpus and either run Claude Code on the same host, or expose the MCP server to a remote workstation over SSH.

Reference hardware¶

The repo was originally built and tested on:

• AMD EPYC 7713 (256 vCPU), 503 GB RAM
• 3× NVIDIA A100 80 GB
• 5.5+ TB free on /opt
• Ubuntu 22.04 / 24.04

You can run on far less. Minimum: 1 modern NVIDIA GPU with ≥16 GB VRAM, 8 vCPU, 32 GB RAM, 100 GB free disk.

Prerequisites¶

Quick host checks:

nvidia-smi                                 # GPUs visible to the OS
sudo systemctl status k3s                  # active (running)
kubectl get nodes -o wide                  # node Ready, GPU label present

1. Apply the stack¶

The manifests are a kustomize base, k3s-deployment/base/ (namespace, NVIDIA device plugin with its time-slicing config, Weaviate, the embedding sidecar), plus host overlays under k3s-deployment/overlays/. On an x86 GPU node the base is all you need; K3s itself provides the nvidia RuntimeClass and the local-path StorageClass the PVC requests.

kubectl label node <node> nvidia.com/gpu.present=true   # the device plugin's nodeSelector
kubectl apply -k k3s-deployment/base
kubectl -n kube-system rollout status daemonset/nvidia-device-plugin-daemonset

Verify GPUs are now schedulable:

kubectl describe node | grep -A2 "nvidia.com/gpu"
#  Allocatable:
#   nvidia.com/gpu:  3

2. Wait for Weaviate + the embedding sidecar¶

# First start pulls ~6 GB of images and the sidecar loads the model before it
# reports Ready; budget 5-10 minutes.
kubectl -n ai-workloads rollout status deployment/t2v-transformers --timeout=15m
kubectl -n ai-workloads rollout status deployment/weaviate --timeout=15m
kubectl -n ai-workloads get pods -o wide

Smoke-test from the host:

curl -s localhost:30080/v1/.well-known/ready              # 200 OK
curl -s localhost:30080/v1/meta | jq '.modules | keys'    # ["text2vec-transformers"]

And from inside the cluster, confirm the sidecar is also healthy:

kubectl -n ai-workloads exec deploy/weaviate -- \
  wget -qO- http://t2v-transformers:8080/.well-known/ready
# {"status":"ready"}

3. Install the fractal-rag package on the host¶

cd /opt/tswetnam/github/fractal-notebooks/fractal_rag
pip install -e ".[dev,parsers]"
fractal-rag concepts validate            # OK: 32 concepts validated

The default config.py already targets the NodePort 30080/30051 exposed by weaviate-service.yaml, and repo_root defaults to /opt/tswetnam/github/fractal-notebooks, so a checkout at that path needs no environment overrides. If the repository lives anywhere else, set FRACTAL_RAG_REPO_ROOT before ingest --source docs or --source notebooks, for example in a gitignored .env at the repo root, which is read when you run fractal-rag from there. Those two sources index nothing, without an error, when the path does not exist.

4. Bootstrap the schema¶

fractal-rag schema bootstrap
# { "created": ["Concept", "FractalArtifact"], "already_present": [] }

5. Pull the corpus from iRODS¶

fractal-rag pull
# Mirrors /iplant/home/tswetnam/projects/fractal-notebooks/ -> /opt/tswetnam/fractal-notebooks/
# (override with FRACTAL_RAG_IRODS_ROOT / FRACTAL_RAG_CORPUS_ROOT, e.g. when /opt is not writable)
# ~30 GB. Resumable; gocmd skips files whose checksums already match.

6. Run the full ingest¶

fractal-rag ingest --source all 2>&1 | tee ~/fractal-rag-ingest.log

Measured on the DGX Spark (2026-10-25): the docs finish in about a minute, the notebooks in about five, and the 50,000 JSONL records take about two hours at 14 chunks per second with two sidecar replicas (every chunk is three vectors). The ingest inserts one chunk at a time, so the GPU is not the bottleneck; the second replica, which uses the second GPU time slice, is the only cheap lever:

kubectl -n ai-workloads scale deploy/t2v-transformers --replicas=2

The manifests declare one replica, so any later kubectl apply -k scales the sidecar back to one; repeat the command before the next long ingest.

Watch progress with:

kubectl -n ai-workloads logs -f deploy/t2v-transformers
fractal-rag status                       # repeat to see counts climbing

When it finishes, sanity-check that all four source types landed. The json object count should equal the inserted_chunks of the json report; if it is smaller, the index was built by the pre-2026-10-25 parser that keyed records on the DOI (see Troubleshooting).

fractal-rag status
# { "FractalArtifact": 104061, "Concept": 32 }   # 2026-10-25: 512 docs + 3,537 notebook + 100,012 json chunks

# Inspect by source_type:
python -c "
from fractal_rag.client import client_session
from fractal_rag.config import get_settings
from weaviate.classes.query import Filter

s = get_settings()
with client_session(s) as c:
    art = c.collections.get(s.artifact_collection)
    for st in ('json_corpus', 'notebook', 'markdown', 'pdf'):
        n = art.aggregate.over_all(
            total_count=True,
            filters=Filter.by_property('source_type').equal(st),
        ).total_count
        print(f'{st}: {n}')
"

7. Wire Claude Code¶

Option A — Run Claude Code on the same VM¶

The repo's .mcp.json references fractal-rag serve on PATH. Just open Claude Code in the repo and run /mcp to confirm fractal-rag is connected, then invoke the subagent (see Using the fractal-explorer agent).

Option B — Run Claude Code on a workstation, MCP on the VM¶

Use SSH to wrap the remote fractal-rag serve so its stdio shows up locally:

// On your workstation, drop this into the project's .mcp.json:
{
  "mcpServers": {
    "fractal-rag": {
      "command": "ssh",
      "args": [
        "-T",
        "-o", "ServerAliveInterval=30",
        "user@your-gpu-vm.example.org",
        "cd /opt/tswetnam/github/fractal-notebooks/fractal_rag && fractal-rag serve"
      ]
    }
  }
}

The MCP protocol runs over stdin/stdout, so SSH (which forwards exactly those) needs no extra setup. Keys-only auth is recommended; password prompts will break the connection.

8. Walk the verification checklist¶

The eight-prompt checklist lives in the repository's verification spec. Target ≥⅞ prompts producing well-cited answers via the fractal-explorer agent. Iteration tips (parser tweaks, swapping the embedding model) are at the bottom of that doc.

Operations playbook¶

Logs¶

kubectl -n ai-workloads logs -f deploy/weaviate
kubectl -n ai-workloads logs -f deploy/t2v-transformers

Restart after a node reboot¶

sudo systemctl restart k3s                # if K3s wasn't enabled-on-boot
kubectl -n ai-workloads rollout restart deploy/weaviate
kubectl -n ai-workloads rollout restart deploy/t2v-transformers

The manifest pins Weaviate's Raft node name (CLUSTER_HOSTNAME=node1), so a re-created pod rejoins its own single-node cluster; a pod created before that setting existed recovers through RAFT_ENABLE_ONE_NODE_RECOVERY (see Troubleshooting).

Upgrade Weaviate¶

Change the image tag in k3s-deployment/base/weaviate-deployment.yaml and apply that file alone: the deployment uses the Recreate strategy, so the old pod releases the volume before the new one mounts it, and a full kubectl apply -k would also reset the sidecar to one replica.

kubectl apply -f k3s-deployment/base/weaviate-deployment.yaml
kubectl -n ai-workloads rollout status deploy/weaviate
curl -s localhost:30080/v1/meta | python -c "import json, sys; print(json.load(sys.stdin)['version'])"

Stay at 1.35.3 or newer: 1.35.0 zeroes every keyword score after a bulk delete (see Troubleshooting). The 2026-10-25 move from 1.35.0 to 1.35.23 kept the 104,061 objects and repaired the scores on restart.

Re-ingest after a schema change¶

fractal-rag schema bootstrap --drop      # destructive: nukes both collections
fractal-rag schema bootstrap
fractal-rag ingest --source all

Swap the embedding model¶

Edit k3s-deployment/base/t2v-transformers-deployment.yaml to change the image tag (e.g. to transformers-inference:sentence-transformers-all-mpnet-base-v2 or nomic-embed-text-v2-moe), or the images: entry of your overlay, then:

kubectl apply -k k3s-deployment/base          # or your overlay
kubectl -n ai-workloads rollout status deploy/t2v-transformers
fractal-rag schema bootstrap --drop      # vectors are model-specific
fractal-rag ingest --source all

DGX Spark (arm64) notes¶

The published transformers-inference images are multi-arch, but on arm64 they ship a CPU-only PyTorch, so the sidecar must be built locally on a CUDA 13 base (docker/t2v-transformers-cuda-arm64/build.sh, arctic-embed-l-v2.0 baked in), imported into K3s's containerd and selected through the dgx-spark overlay:

bash docker/t2v-transformers-cuda-arm64/build.sh
docker save fractal-rag/t2v-transformers:arctic-embed-l-v2.0-cu130-arm64 | sudo k3s ctr -n k8s.io images import -
kubectl apply -k k3s-deployment/overlays/dgx-spark

The device plugin must be v0.17.4 or newer on the GB10 (the base ships v0.20.1), nvidia-smi reports memory as N/A on this unified-memory platform, and a pod's memory limit also caps its GPU allocations. On a GB10 the sidecar embeds about 16 long or 50 short texts per second; a full ingest ran at about 8 chunks per second with one replica and 14 with two. See k3s-deployment/README.md for the full recipe.

Where to go next¶

• Run the eight-prompt agent checklist → Using the fractal-explorer agent
• Pod stuck Pending? GPU not visible? → Troubleshooting
• Move to a lighter local setup for iteration → Local setup