Transparent AI Framework — Protocol for Open AI System

A public, versioned, transparent, replayable semantic commons — governed openly and designed to resist capture.

Version: 0.1.0 Status: Draft Author: Technology Shield Last Updated: 2026-03-25

1. Preamble

The large AI systems being built today are funded by billions of dollars, trained on opaque data, governed by invisible rules, and designed to concentrate power.

This protocol is a countermeasure.

It does not claim to produce unbiased truth. Instead, it creates a system where:

• All inputs are inspectable
• All transforms are declared
• All rankings are attributable
• All disputes are preserved
• Alternative interpretations can coexist

You do not eliminate bias by claiming neutrality. You reduce bias by making every assumption legible, forkable, challengeable, and reversible.

That is the heart of this protocol.

2. Protocol Layers

The Transparent Semantic Commons operates across five layers:

┌──────────────────────────────────────────────────────┐
│                 5. RETRIEVAL LAYER                     │
│  Search, ranking, neighbourhood lookup,               │
│  contradiction surfacing, competing views              │
├──────────────────────────────────────────────────────┤
│                 4. GOVERNANCE LAYER                    │
│  Reputation, dispute resolution, weighting rules,     │
│  moderation, anti-poisoning, forkability              │
├──────────────────────────────────────────────────────┤
│                 3. PROOF LAYER                         │
│  Signatures, hashes, timestamps, lineage,             │
│  challenge records, verification                      │
├──────────────────────────────────────────────────────┤
│                 2. TRANSFORMATION LAYER                │
│  Chunking, normalisation, tagging, translation,       │
│  embedding, summarisation                             │
├──────────────────────────────────────────────────────┤
│                 1. SOURCE LAYER                        │
│  Raw public artefacts: text, images, audio,           │
│  datasets, claims, arguments                          │
└──────────────────────────────────────────────────────┘

Each layer has clear responsibilities, and objects at each layer are defined by the Open Embeddings Schema (separate document).

3. Layer 1 — Source

Purpose

Accept and preserve raw public artefacts with verified provenance.

Rules

Storage

Artefacts are stored off-chain in durable decentralised storage (IPFS, Arweave, Filecoin, or federated node operators). Content hashes and signatures are anchored on-chain for tamper-evidence.

4. Layer 2 — Transformation

Purpose

Record every operation applied to a source artefact, with full transparency about who did it, what software they used, what model they ran, and what parameters they chose.

Rules

Transform Types

5. Layer 3 — Proof

Purpose

Provide cryptographic evidence that the provenance chain is intact, that objects have not been tampered with, and that challenges and disputes are preserved.

Mechanisms

Verification Protocol

For any object retrieved from the commons:

1. Hash verification — Does the object's content match its declared hash?
2. Signature verification — Is the signature valid for the declared signer?
3. Identity resolution — Does the signer's DID resolve to a valid public key?
4. Lineage verification — Does the provenance chain back to the source artefact verify?
5. Challenge check — Are there open challenges against this object or any object in its chain?
6. Governance check — Are there governance decisions (tombstones, supersessions) affecting this object?

Result: verified | unverifiable | disputed | tombstoned | superseded

6. Layer 4 — Governance

Purpose

Govern the commons in a way that resists capture by any single party, organisation, or alliance.

6.1 Core Governance Rules

These rules are constitutional — they define the minimum governance that all participants accept:

6.2 Roles

Anti-capture constraint: No single role may control all of: - Data admission - Ranking - Dispute resolution

These three powers must be held by different parties.

6.3 Challenge Resolution Process

Challenge opened
    │
    ▼
Evidence gathering period (configurable, default 14 days)
    │
    ▼
Review by Challenge Reviewers (minimum 3 independent reviewers)
    │
    ├── Upheld → Governance action (tombstone, supersede, flag)
    ├── Dismissed → Challenge marked as dismissed with rationale
    └── Escalated → Broader community vote
         │
         ▼
    Appeal window (configurable, default 30 days)
         │
         ├── No appeal → Decision final
         └── Appeal → Re-review with expanded panel

6.4 Forkability

This is the ultimate anti-capture safety valve.

If a community believes governance has been captured, they must be able to fork:

The fork inherits the full history. The new community can then modify governance going forward while preserving the shared past.

7. Layer 5 — Retrieval

Purpose

Provide transparent, policy-driven retrieval that makes bias visible rather than invisible.

7.1 Retrieval Policy as a Public Object

Retrieval policies are themselves signed, versioned objects:

{
  "type": "RetrievalPolicy",
  "policy_id": "cid:bafy...",
  "candidate_sources": ["public", "science", "english"],
  "similarity_metric": "cosine",
  "recency_weight": 0.1,
  "reputation_weight": 0.2,
  "challenge_penalty": 0.5,
  "diversity_requirement": {
    "minority_view_floor": 0.15,
    "contradiction_surfacing": true,
    "max_single_source_share": 0.3
  },
  "jurisdictional_filters": [],
  "version": "1.0.0",
  "published_by": "did:key:z6Mk...",
  "signature": "ed25519:..."
}

Different communities can publish different retrieval policies while sharing the same underlying commons. This is how pluralism works in practice.

7.2 Bias-Resistant Retrieval Pattern

For any query, the retrieval response includes:

{
  "query": "...",
  "policy_used": "cid:bafy...",
  "model_spaces_used": ["bge-m3", "e5-large-v2"],
  "results": {
    "supporting": [
      {
        "embedding_id": "cid:bafy...",
        "similarity": 0.94,
        "provenance_status": "verified",
        "challenge_status": "none"
      }
    ],
    "contradicting": [
      {
        "embedding_id": "cid:bafy...",
        "similarity": 0.87,
        "provenance_status": "verified",
        "challenge_status": "none"
      }
    ],
    "disputed": [
      {
        "embedding_id": "cid:bafy...",
        "challenge_id": "cid:bafy...",
        "challenge_status": "open"
      }
    ]
  },
  "policy_explanation": "Results include top 10 supporting and top 5 contradicting. Minority view floor of 15% applied. Disputed objects surfaced separately."
}

Instead of returning one answer, the system returns:

1. Closest matches (supporting evidence)
2. Strongest counter-matches (contradicting evidence)
3. Unresolved disputes (challenged objects)
4. Policy explanation (which rules shaped the response)
5. Provenance status (verification state of each result)

This is fundamentally healthier than a single invisible ranking.

7.3 Index Operator Rules

Anyone can run an index. Index operators:

• Must declare which retrieval policies they support
• Must not alter underlying object data
• Must serve provenance chains on request
• Must surface challenges and disputes
• Must declare their funding and governance
• Can be forked (anyone can run a competing index)

8. Anti-Capture Mechanisms

8.1 Structural

8.2 Economic

8.3 Social

9. Bootstrapping Plan

Phase 1 — Foundation

Phase 2 — Community

Phase 3 — Federation

Phase 4 — Maturity

10. Community and Communication

Recommended Stack

Why This Stack

• Matrix is an open network for decentralised communication, governed by a foundation committed to open standards. It fits a serious protocol-building community.
• ActivityPub is a W3C Recommendation for decentralised social networking. It provides reach without platform dependence.
• AT Protocol is an open framework for public conversation with account portability. It is good for discovery and attracting technical contributors.
• Git provides permanence, versioning, and collaboration for specifications.
• FOSDEM and similar events connect the open-source and decentralised-web communities in person.

11. Charter

Transparent Semantic Commons Charter v0.1

1. No hidden transforms.
2. No silent edits.
3. No opaque ranking.
4. No single embedding monopoly.
5. Every claim must be linkable to evidence.
6. Every object must be challengeable.
7. Every governance action must be public.
8. Every community must be able to fork.
9. Consumers should contribute back.
10. The commons exists for collective flourishing, not extraction.

12. The Hard Truths

The enemy is not just secrecy. It is also:

• Convenience — closed systems are easier to use
• Apathy — most people will not participate in governance
• Scale asymmetry — billion-dollar systems have more resources
• Moderation burden — open systems attract abuse
• Slow institutional capture — governance can be co-opted gradually

This protocol wins only if it is:

1. Simpler to inspect than closed systems
2. Easier to fork than captured systems
3. More useful in practice than being merely morally appealing

A working prototype matters more than a perfect philosophy.

13. Naming

Candidate names for the broader initiative:

The recommended name is Transparent Semantic Commons — it is precise, unglamorous, and resistant to marketing capture.

14. Relationship to Technology Shield

Technology Shield contributes this protocol and framework as part of its commitment to cybersecurity that serves people, not just organisations.

The Transparent AI Framework connects to Technology Shield's other collateral:

15. What to Build First

One repo. One schema. One Matrix room. One public manifesto. One explorer UI. One sample corpus. One challenge flow.

Start there. Everything else follows.