Date: 2026-02-22 Lines of enquiry: 7 Status: Complete

1. Federated/Multi-Source Knowledge Systems in Developer Tools¶

Dendron Multi-Vault¶

Dendron is the closest existing system to what picobot needs. Its multi-vault architecture allows multiple knowledge repositories to coexist in a single workspace with cross-references between them.

Key design decisions: - Each vault is a separate git repo (or a directory within a monorepo) - Vaults are declared in dendron.yml with a fsPath and optional remote URL - Cross-vault links use an explicit prefix: [[dendron://vault-name/note.path]] - When a plain link like [[recipes.pasta]] exists in multiple vaults, Dendron prompts the user to disambiguate - Remote vaults are auto-cloned when a workspace is pulled down - Sync is per-vault: each vault pushes/pulls independently

Relevance to picobot: Dendron proves that multi-repo KB federation works in practice. The explicit cross-vault link syntax is important -- it means the system never silently resolves ambiguity. However, Dendron's approach is workspace-centric (VS Code extension), not CLI-agent-centric.

Obsidian Multi-Vault¶

Obsidian takes a different approach: one vault at a time, no cross-vault linking.

Tradeoffs identified by the community: - Single vault maximizes knowledge incubation through unexpected cross-links - Multiple vaults provide separation of concerns (work/personal/shared) - No native cross-vault links means splitting reduces serendipitous discovery - Plugin ecosystem is per-vault, requiring duplicate configuration

Relevance to picobot: The Obsidian community's experience validates that splitting knowledge into separate stores has a real cost in discoverability. A federated system must provide unified search across all remotes to avoid this penalty.

Relay (Obsidian Plugin)¶

Relay uses CRDTs for real-time collaboration within shared vaults. This is the collaborative editing approach -- different from federation, which is about composing multiple independent sources.

BMAD Federated Knowledge¶

The most directly relevant prior art. BMAD-METHOD's federated knowledge extension (bmad-federated-knowledge) implements a priority-based multi-source knowledge system.

Architecture: - Config in YAML: each federated repo has repo, branch, local_cache, sync_policy, and priority (numeric 0-999) - Local knowledge gets priority 999 (always highest unless overridden) - FederatedDependencyResolver aggregates content from multiple sources - KnowledgeMerger supports three strategies: priority (higher wins), merge (combine), manual (human decides) - Per-knowledge-type strategy configuration (templates use priority, workflows use merge, data uses manual) - CLI: bmad-fed init, bmad-fed add, bmad-fed sync, bmad-fed build-context - Output: unified context.md mapping all knowledge sources

Limitations: BMAD flattens repos to XML for AI consumption rather than preserving the original file structure. It is designed for agent context building, not for a live queryable knowledge base.

2. Git Strategies for Multi-Remote Knowledge Bases¶

Option A: Git Submodules¶

Each remote KB is a submodule mounted at a path like knowledge/remotes/team/.

Pros: - Clean separation -- each remote is its own repo with independent history - Standard git tooling, well-understood by developers - Explicit version pinning (submodule points to a specific commit)

Cons: - Two-step workflow: must commit in submodule, then update reference in parent - Cannot make atomic commits across parent and submodule - Submodule state can drift if not updated regularly - Adds complexity for users unfamiliar with submodules

Option B: Git Subtree¶

Remote KB content is merged directly into the main repo's tree.

Pros: - Content is immediately available after clone (no submodule init) - Single repo to manage from the user's perspective - Standard git operations work without special commands

Cons: - History of subtree merges pollutes the main repo's log - Contributing changes back upstream requires git subtree push, which is non-obvious - Harder to reason about which content came from which remote

Option C: Multiple Remotes on One Repo¶

A single KB repo with multiple remotes (origin, team, community) on different branches or namespaces.

Pros: - Simplest git model -- one repo, multiple fetch sources - Can selectively pull from specific remotes

Cons: - Branch management becomes complex with many remotes - No natural way to overlay/shadow content from different sources - Merge conflicts between remotes require manual resolution

Option D: Separate Repos with Virtual Overlay (Recommended)¶

Each remote is a separate git repo cloned to a cache directory. The KB system presents a unified virtual view by resolving reads through the priority chain: local -> project -> org -> community.

Pros: - Each remote is fully independent -- no coupling between repos - Priority resolution happens at the application layer, not the git layer - Remotes can be added/removed without affecting local repo - Sync policy can differ per remote (some hourly, some daily, some manual) - Sparse checkout or partial clone can reduce bandwidth for large community repos

Cons: - Application must implement the overlay/resolution logic - Search must span multiple directory trees - Storage cost of multiple full clones (mitigated by partial/shallow clones)

Recommendation: Option D (separate repos with virtual overlay) is the best fit for picobot. It matches the BMAD approach, avoids git-layer complexity, and keeps each remote fully independent. The overlay logic belongs in KnowledgeBase, not in git.

Sparse Checkout for Large Remotes¶

Git sparse checkout (cone mode, introduced in Git 2.25) allows checking out only specific directories from a large repo. Combined with partial clone (--filter=blob:none), this enables efficient access to community KBs where only certain topics are relevant.

git clone --depth 1 --filter=blob:none --sparse <community-kb-url>
git sparse-checkout set domains/ projects/

3. Encryption: age vs git-crypt vs SOPS¶

age (FiloSottile/age)¶

A modern, simple encryption tool with small explicit keys and no configuration options.

Key properties: - X25519 keys (native) or Ed25519/RSA SSH keys as recipients - Multiple recipients per file: -r recipient1 -r recipient2 - Each file encrypted with a random symmetric key; recipient public keys encrypt that symmetric key in the header - Keys are small and textual -- designed for generating dedicated keys per task - No agents, no config files, no key rings -- pure UNIX composability - Encryption is non-deterministic (same plaintext + same recipients = different ciphertext each time)

git-crypt¶

Transparent file encryption in git using GPG.

Key properties: - Uses git's clean/smudge filters for transparent encrypt/decrypt - AES-256-CTR with synthetic IV derived from SHA-1 HMAC - Requires GPG agent and key management - Encrypted files are not compressible (every small change stores entire file) - Mature but complex setup

git-agecrypt¶

Age-based alternative to git-crypt using git's smudge/clean/textconv filters.

Key properties: - Transparent: files are plaintext in working tree, encrypted in git - Configured via git-agecrypt.toml at repo root + .gitattributes for file patterns - Solves non-deterministic encryption by caching blake3 hashes of plaintext under .git/git-agecrypt/ - Supports Age keys, ed25519 SSH keys, and Age plugin stubs (YubiKey) - Per-file recipient configuration possible - Limitation: starts fresh process per file during git operations (slow with many files) - Limitation: entire file loaded into memory during encrypt/decrypt

agebox¶

Age-based encryption tool designed for gitops workflows.

Key properties: - Not transparent (explicit encrypt/decrypt commands, adds .agebox extension) - Tracks encrypted files in .ageboxreg.yml registry - Supports bulk reencrypt when recipients change - Dry-run mode for CI validation - Simpler than SOPS, more explicit than git-agecrypt

SOPS (Mozilla/getsops)¶

Structured encryption for YAML, JSON, ENV, and INI files.

Key properties: - Encrypts values while keeping keys in cleartext - Git diffs show which key changed (not just "binary file differs") - Supports age, AWS KMS, GCP KMS, Azure Key Vault, PGP - Configuration via .sops.yaml with path_regex and encrypted_regex - Git integration via clean/smudge filters for auto-encrypt on commit - Handles merge conflicts much better than whole-file encryption

Recommendation for Picobot¶

Primary: age with topic-level recipient lists (agebox-style workflow)

Rationale: 1. Picobot's KB uses markdown files with YAML frontmatter. SOPS's key-level encryption is designed for config files, not prose documents. Whole-file encryption with age is the right granularity. 2. git-agecrypt's transparent approach is elegant but has performance issues with many files and adds complexity. 3. agebox's explicit encrypt/decrypt model fits picobot's kb sync workflow -- encrypt before push, decrypt after pull. 4. Per-topic recipient configuration maps naturally to sharing.yaml: each topic can list its age recipients. 5. SSH key support means no new key infrastructure -- users' existing ed25519 keys work.

Proposed encryption flow:

picobot kb sync
  1. Read sharing.yaml for topic encryption config
  2. For encrypted topics: age -r <recipients> each file before commit
  3. Push to remote (files encrypted at rest)
  4. On pull: age -d -i <identity> each encrypted file
  5. Plaintext in working tree, encrypted in git

4. Priority/Overlay Systems¶

Nix Overlays¶

Nix uses a composable overlay system where each overlay is a function (final: prev: { ... }) that can modify or add packages.

Key design properties: - Overlays are applied in order: later overlays override earlier ones - prev refers to the package set before this overlay; final refers to the fully resolved set (for self-referential definitions) - Priority system via lib.mkDefault (priority 1000), lib.mkForce (priority 50), lib.mkBefore/lib.mkAfter (500/1500) - Lower numeric priority = higher precedence (opposite of BMAD) - Lists are merged, not replaced (module system knows the type) - Composable: overlays from different sources (flakes, channels, local) are combined

Relevance: The Nix pattern of "function that transforms a previous state" is powerful but complex. For picobot, we need a simpler model since KB entries are documents, not function definitions.

CSS Cascade¶

CSS resolves conflicts through a multi-level priority system.

Priority levels (highest to lowest): 1. Origin and importance (!important from user agent < author < user) 2. Cascade layers (@layer declarations, last declared layer wins) 3. Specificity (ID > class > element) 4. Source order (last declaration wins)

Relevance: The cascade's insight is that priority can be resolved at multiple levels. For picobot: remote priority (local > project > org > community) is like origin; topic-level overrides are like specificity; timestamp is like source order.

Kubernetes Kustomize Overlays¶

Kustomize uses a base + overlay model where overlays patch base configurations.

Key design properties: - Base contains common configuration; overlays contain environment-specific patches - Strategic merge patches: merge maps, replace scalars, use merge keys for lists - JSON patches for precise operations on specific paths - ConfigMap generator with behavior: merge for combining values - Each overlay is a directory with its own kustomization.yaml

Relevance: Kustomize's base + overlay model maps well to KB federation: community KB is the base, org KB overlays it, project KB overlays that, local KB overlays everything.

BMAD Priority (Revisited)¶

• Numeric priority 0-999, higher wins
• Local always 999 unless overridden
• Per-knowledge-type merge strategy (priority/merge/manual)
• Hierarchical: local > project > organization > community

Recommended Priority Model for Picobot¶

Use a layered resolution with explicit numeric priorities inspired by BMAD and Kustomize:

# ~/.picobot/config.json (remotes section)
remotes:
  local:
    priority: 100  # highest - always wins
    path: ~/.picobot/workspace/knowledge
  project:
    priority: 80
    repo: git@github.com:team/project-kb.git
    sync: on_demand
  org:
    priority: 60
    repo: git@github.com:company/org-kb.git
    sync: daily
  community:
    priority: 40
    repo: git@github.com:picobot-community/kb.git
    sync: weekly

Resolution algorithm: 1. On kb search/read, iterate remotes in priority order (highest first) 2. For read: return first match (shadowing) 3. For search: return all matches, annotated with source, sorted by priority 4. For store: always write to local 5. Topic-level overrides in sharing.yaml can pin specific topics to specific remotes

5. Conflict Resolution Strategies¶

Last-Write-Wins (Current Picobot)¶

Picobot already implements LWW using the updated timestamp in YAML frontmatter. During merge conflicts, the version with the newer updated field wins.

Pros: Simple, deterministic, no user intervention required. Cons: Loses the older version's changes entirely. No content merging.

Priority-Based Shadowing (Recommended Primary Strategy)¶

In a federated system, conflicts between remotes are resolved by priority. If local/domains/python.md and community/domains/python.md both exist, the local version shadows the community version.

Key design: This is not a merge -- the lower-priority version is invisible unless explicitly requested. This avoids the complexity of content merging while preserving the ability to override community content locally.

Implementation:

def read(self, path: str) -> str:
    for remote in sorted(self.remotes, key=lambda r: r.priority, reverse=True):
        target = remote.base / path
        if target.exists():
            return target.read_text()
    return "Error: not found"

Content Deduplication¶

Picobot already has duplicate detection (title >60% overlap or content >70% overlap). In a federated system, this should run cross-remote to warn when storing content that duplicates an entry in a higher-priority remote.

Three-Way Merge for Same-Remote Conflicts¶

When pulling from a remote and the same file was modified both locally and remotely (within the same remote), use git's standard three-way merge. If that fails, fall back to LWW on the updated timestamp.

Manual Resolution Escape Hatch¶

For cases where automatic resolution is unacceptable (e.g., team policies, critical domain knowledge), allow entries to be tagged with conflict: manual in frontmatter. When a conflict is detected on such entries, pause sync and notify the user.

CRDT Considerations¶

CRDTs (Conflict-free Replicated Data Types) provide mathematically guaranteed convergence for distributed edits. However, they are designed for real-time collaborative editing (character-by-character), not for document-level federation. CRDT adoption for picobot's use case would be overengineering -- the sync cadence is minutes/hours/days, not milliseconds.

Verdict: CRDTs are not appropriate here. Priority-based shadowing with LWW fallback is the right level of complexity.

6. Sharing and Access Control Patterns¶

Current Picobot Model¶

sharing.yaml defines per-topic access levels (private or shared). Private topics are excluded from git sync via .gitignore. This is a binary model: either a topic syncs or it does not.

Encryption-Based Access Control (Proposed)¶

Instead of excluding private topics from sync entirely, encrypt them so they can be synced but only decrypted by authorized recipients.

Design:

# sharing.yaml (extended)
topics:
  self:
    access: encrypted
    recipients:
      - age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p  # personal key
  user:
    access: encrypted
    recipients:
      - age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
  projects:
    access: shared
  domains:
    access: shared
  journal:
    access: encrypted
    recipients:
      - age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
      - ssh-ed25519 AAAA... teammate@company  # teammate can also read journal

Benefits over gitignore-based privacy: - Encrypted topics can sync to remote (backup, multi-device access) - Team members with the right keys can access shared encrypted topics - Different recipients per topic enables fine-grained access control - Revoking access = reencrypt without that recipient's key

Git Branch-Per-Audience¶

An alternative model where different branches contain different subsets of content:

• main branch: shared topics only
• personal branch: all topics (including encrypted)
• team branch: shared + team-encrypted topics

Verdict: This adds significant git complexity. Encryption-based access control is simpler and more flexible.

Hierarchical Sharing with Remote-Level Defaults¶

# Per-remote sharing defaults
remotes:
  personal:
    default_access: encrypted
    recipients: [personal-key]
  team:
    default_access: shared
    encrypted_topics: [hr, finance]
    recipients: [team-key-1, team-key-2]
  community:
    default_access: shared
    read_only: true  # cannot store to community remote

7. Counter-Perspective: Is Federation Worth the Complexity?¶

Arguments Against Federation¶

Complexity cost: - Every layer of indirection (priority resolution, cross-remote search, encryption, sync policies) is a potential failure point - Users must understand which remote a piece of knowledge came from and why - Debugging "why is my entry being shadowed?" requires understanding the priority system - Encryption key management is a real operational burden

Simpler alternatives:

1. Single shared repo with branch protection. One repo, everyone pushes to it, CODEOWNERS restricts who can modify which topics. Simplest possible model.

2. Copy-on-read. Instead of federation, simply copy interesting entries from other KBs into your local KB. No sync, no priority, no conflicts. Just manual curation.

3. Bookmarks/links to external KBs. Instead of importing external knowledge, store links to it. The agent follows links when needed. Zero sync overhead.

4. Import command. picobot kb import <url> <topic> one-shot imports from a remote KB. No ongoing sync relationship.

When Federation Becomes Necessary¶

Federation solves specific problems that simpler alternatives cannot:

1. Multi-device personal KB. You need your private knowledge on your laptop and your desktop. A single remote with encryption handles this.

2. Team knowledge that updates independently. The team maintains domain knowledge that evolves over time. You want to see updates without manual re-importing. This requires sync.

3. Community templates and patterns. A community maintains best-practice templates. You want to inherit them but override locally. This requires priority-based overlay.

4. Organizational compliance. The org mandates certain knowledge entries (security policies, coding standards). These must be present in every agent's KB and cannot be deleted locally. This requires read-only remotes with high priority.

Phased Adoption Recommendation¶

Do not build full federation immediately. Instead:

Phase 1: Single remote sync (already exists) - picobot kb sync with one remote, sharing.yaml, LWW conflict resolution - This is what picobot has today

Phase 2: Encryption for private topics - Replace .gitignore-based privacy with age encryption - Encrypted topics sync to remote, only decryptable by authorized recipients - No federation yet -- just one remote with encrypted topics

Phase 3: Multiple remotes with priority - Add remotes config with priority levels - Implement virtual overlay in KnowledgeBase.read() and KnowledgeBase.search() - Each remote syncs independently - Read resolution: highest priority first

Phase 4: Team and community features - Read-only remotes for community/org knowledge - Topic-level recipient lists for team encryption - Cross-remote duplicate detection - Sparse checkout for large community repos

This phased approach means the complexity is adopted only as the use case demands it.

Synthesis: Recommended Architecture for Picobot¶

Config Model¶

// ~/.picobot/config.json
{
  "knowledge": {
    "remotes": {
      "local": {
        "priority": 100,
        "path": "~/.picobot/workspace/knowledge"
      },
      "team": {
        "priority": 80,
        "repo": "git@github.com:company/team-kb.git",
        "branch": "main",
        "sync": "daily",
        "sparse_topics": ["domains", "projects"],
        "read_only": false
      },
      "community": {
        "priority": 40,
        "repo": "git@github.com:picobot-community/kb.git",
        "branch": "main",
        "sync": "weekly",
        "read_only": true
      }
    },
    "encryption": {
      "enabled": true,
      "tool": "age",
      "identity": "~/.picobot/age-identity.txt"
    }
  }
}

Extended sharing.yaml¶

topics:
  self:
    access: encrypted
    recipients:
      - age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
  user:
    access: encrypted
    recipients:
      - age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
  projects:
    access: shared
  domains:
    access: shared
  journal:
    access: encrypted
    recipients:
      - age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

Resolution Algorithm¶

read(path):
  for remote in remotes sorted by priority (descending):
    if remote.has(path):
      if encrypted: decrypt with identity
      return content
  return not_found

search(query):
  results = []
  seen_paths = set()
  for remote in remotes sorted by priority (descending):
    for match in remote.search(query):
      if match.path not in seen_paths:
        match.source = remote.name
        results.append(match)
        seen_paths.add(match.path)
  return results

store(topic, title, content):
  always store to local remote
  check cross-remote duplicates before storing

sync():
  for remote in remotes:
    if remote.sync_due():
      if remote.encrypted_topics:
        encrypt_before_push(remote)
      git_pull_merge(remote)
      if remote.encrypted_topics:
        decrypt_after_pull(remote)
      git_push(remote)

Key Implementation Classes¶

FederatedKnowledgeBase(KnowledgeBase):
    remotes: list[KBRemote]
    encryption: EncryptionConfig

    read() -> priority-based resolution across remotes
    search() -> unified search with source annotation
    store() -> always local, cross-remote dedup check
    sync() -> per-remote git sync with encryption

KBRemote:
    name: str
    priority: int
    path: Path  # local cache
    repo: str | None  # git URL
    sync_policy: str
    read_only: bool
    sparse_topics: list[str] | None

EncryptionConfig:
    tool: str  # "age"
    identity: Path

    encrypt_topic(topic_dir, recipients) -> None
    decrypt_topic(topic_dir) -> None

Migration Path from Current KnowledgeBase¶

The existing KnowledgeBase class becomes the implementation for a single remote. FederatedKnowledgeBase wraps multiple KnowledgeBase instances and implements priority-based resolution. The current sync() method moves into KBRemote. This preserves backward compatibility: a config with only the local remote behaves identically to today's system.