Tool Artifact Registry — Design

1. Problem

MaastrichtU-IDS runs a growing set of tools — shacl-manager, sulo-schema-builder, rdf_tx, obda-lazy-cache-demo — across several deployments (the ids3 and idsg2 clusters, partner sites, laptops). Each consumes and produces data artifacts: RDF graphs, SHACL shapes, validation reports, OWL ontologies, mapping files, masked replicas.

Today there is no shared record of:

• what tools exist, who is responsible for them, under what licence;
• what a tool can consume and produce;
• what a given deployment actually did produce, and when;
• where those artifacts live and how to get at them;
• how an artifact in one institution's estate relates to one in another's.

Nothing outlives the person who ran the job. This document specifies a service that fixes that, and that any third party can deploy for their own estate and cross-link with ours.

1.1 Requirements

Verbatim from the stakeholder:

1. Store in a graph (or NoSQL/SQL DB) — choose and justify.
2. Tool information (name, link, repo, licence, responsible party, …).
3. Artifacts each tool generates and consumes.
4. Endpoint to advertise artifacts a tool generates (with access ways, FAIR data).
5. Endpoint to advertise artifacts a tool consumed (or, tool sent to be consumed).
6. Deployable by many people: anyone can run their own artifact registry (containerised, minimal ops, sane defaults, Helm/compose).
7. Deployments must be referenceable by each other: a tool or artifact in one registry can point at one in another registry (stable global identifiers, federation/cross-links, discovery of peer registries).

1.2 Non-goals for v1

• Not an artifact store. The registry never holds artifact bytes (§3, D1).
• Not a workflow engine. It records what ran; it does not schedule or execute.
• Not multi-tenant. One registry serves one estate; multiple estates federate (§9).
• Not an access broker. It describes how to request access; it does not grant it.

2. Prior art

Three mature ecosystems exist. None covers the intersection this project needs. Building is justified — reinventing their formats is not.

2.1 Software / tool registries

2.2 Data lineage platforms

2.3 Catalogue federation

CKAN + DCAT-AP harvesting, OAI-PMH, and Signposting solve federated catalogue discovery and machine-actionable FAIR navigation. None of them model tools, deployments, or runs.

2.4 Conclusion

The gap is the intersection: an RDF-native, self-hostable, federatable registry in which tool identity, declared capability, deployment instances, and instance-level produce/consume lineage live in one graph under dereferenceable global IRIs. No existing system ships that.

What we adopt rather than reinvent: biotoolsSchema, DCAT 3, PROV-O, EDAM, SPDX, FAIR Signposting, and OpenLineage (via a translating adapter, §7.6).

2.5 Why not just deploy RSD?

RSD is the closest existing system to requirement 6 and is institutionally adjacent to us. It is rejected because requirements 3, 4, 5 and 7 — the core of this project — are entirely absent from its data model, and retrofitting an artifact/run/instance graph onto a PostgREST-over-Postgres schema designed for software cards is a larger job than building on a triplestore that already speaks our vocabularies.

We keep the door open: GET /api/v1/software/{id}/export/biotools emits a biotoolsSchema-conformant record, so our tool descriptions can populate a bio.tools or RSD instance without a runtime dependency in either direction (§7.2).

3. Decisions

4. Data model

4.1 Layers

Software        shacl-manager                       abstract; repo, licence, party
  └─ Release      v2.1  (image sha256:ab12…)        a versioned, runnable plan
       └─ Instance  shacl.ids.unimaas.nl            a deployment; agent that acts
            └─ Run    01J9F…  2026-08-30T14:02Z     one execution
                 ├─ used       → Artifact           (consume advertisement)
                 └─ generated  → Artifact           (produce advertisement)
                                    └─ Distribution  access descriptors

4.2 Entities

4.3 Relations

@prefix tar:  <https://w3id.org/tar/ns#> .

<Release>   dct:isVersionOf        <Software> .
<Software>  tar:hasCapability      <Capability> .
<Capability> tar:produces          <ArtifactType> ;
             tar:consumes          <ArtifactType> .

<Instance>  a                      prov:SoftwareAgent ;
            tar:runsRelease        <Release> ;
            dct:publisher          <Agent> ;
            dcat:endpointURL       <https://shacl.ids.unimaas.nl> .

<Run>       a                      prov:Activity ;
            prov:qualifiedAssociation [ prov:agent   <Instance> ;
                                        prov:hadPlan <Release> ] ;
            prov:used              <Artifact_in> .          # consumed
<Artifact_out> prov:wasGeneratedBy <Run> .                  # produced

<Artifact>  dcat:distribution      <Distribution> ;
            dct:conformsTo         <ArtifactType> ;
            prov:wasDerivedFrom    <https://peer.example.org/artifact/01J7…> .

The prov:qualifiedAssociation form is deliberate: it binds who acted (the Instance) and what plan they followed (the Release) in one reified node, which is exactly PROV's intended use and lets a run be attributed even when the Release is unknown.

Federation touches the model in exactly one place: any object position may hold a foreign IRI. There is no "remote artifact" type. A cross-registry lineage edge is an ordinary triple.

4.4 Identifiers

{TAR_BASE_IRI}/software/{uuidv7}
{TAR_BASE_IRI}/release/{uuidv7}
{TAR_BASE_IRI}/instance/{uuidv7}
{TAR_BASE_IRI}/artifact/{uuidv7}
{TAR_BASE_IRI}/artifact-series/{uuidv7}     # version concept IRI (D10)
{TAR_BASE_IRI}/run/{uuidv7}
{TAR_BASE_IRI}/type/{uuidv7}                # local ArtifactTypes only; EDAM keeps its own IRIs

Every IRI dereferences with content negotiation: text/turtle, application/ld+json, application/json (a flattened developer-facing shape), text/html (the UI page).

4.5 Worked example

<https://reg.ids.unimaas.nl/software/01J8A…>
    a schema:SoftwareApplication ;
    schema:name "shacl-manager" ;
    schema:codeRepository <https://github.com/MaastrichtU-IDS/shacl-manager> ;
    dct:license <https://spdx.org/licenses/Apache-2.0> ;
    dct:publisher <https://ror.org/02jz4aj89> ;
    tar:hasCapability [
        tar:consumes <http://edamontology.org/data_2600> ,
                     <https://reg.ids.unimaas.nl/type/01J8B…> ;   # SHACL shapes graph
        tar:produces <http://edamontology.org/data_2048> ] .

<https://reg.ids.unimaas.nl/instance/01J8C…>
    a prov:SoftwareAgent, dcat:DataService ;
    rdfs:label "shacl.ids.unimaas.nl" ;
    tar:runsRelease <https://reg.ids.unimaas.nl/release/01J8D…> ;
    dcat:endpointURL <https://shacl.ids.unimaas.nl> .

<https://reg.ids.unimaas.nl/run/01J9F…>
    a prov:Activity ;
    prov:startedAtTime "2026-08-30T14:02:11Z"^^xsd:dateTime ;
    tar:status "success" ;
    prov:qualifiedAssociation [ prov:agent   <https://reg.ids.unimaas.nl/instance/01J8C…> ;
                                prov:hadPlan <https://reg.ids.unimaas.nl/release/01J8D…> ] ;
    prov:used <https://reg.mumc.nl/artifact/01J7Z…> .            # foreign input

<https://reg.ids.unimaas.nl/artifact/01J9G…>
    a dcat:Dataset ;
    dct:title "Validation report — patients.ttl vs fhir-shapes v3" ;
    dct:conformsTo <http://edamontology.org/data_2048> ;
    dct:license <https://spdx.org/licenses/CC-BY-4.0> ;
    prov:wasGeneratedBy <https://reg.ids.unimaas.nl/run/01J9F…> ;
    dcat:distribution [
        a dcat:Distribution ;
        dcat:accessURL <https://shacl.ids.unimaas.nl/reports/9f2a> ;
        dcat:mediaType "text/turtle" ;
        dcat:byteSize 2118342 ;
        spdx:checksum [ spdx:algorithm spdx:checksumAlgorithm_sha256 ;
                        spdx:checksumValue "9f2a…" ] ;
        tar:accessProtocol "https" ;
        tar:authMethod     "apikey" ;
        tar:availability   "restricted" ] .

5. Store choice

5.1 Decision

Embedded Oxigraph for the RDF domain graph, plus SQLite (via sqlx) for operational state. Both embed in the single Rust binary. The default deployment is one container and one volume.

5.2 Justification against the requirements

• Req. 1 (graph): the domain genuinely is a graph — lineage traversal, cross-registry cross-links, typed relations. Modelling it as RDF also is the interoperability story; a relational schema would need a serialisation layer to reach the same place.
• Req. 6 (minimal ops): no external database process. No JVM (Fuseki), no index build step (QLever), no separate Postgres. docker run -v tar-data:/data … is a complete install.
• Req. 7 (federation): SPARQL SERVICE gives federated query to power users at no implementation cost, and peer graphs isolate foreign data cleanly (§5.4).

5.3 Costs, honestly

5.4 Named graphs

<urn:tar:local>        authoritative — triples this registry minted
<urn:tar:peer:{id}>    cached foreign stubs, read-only
<urn:tar:shapes>       SHACL shapes used for write validation
<urn:tar:vocab>        preloaded EDAM / SPDX / DCAT terms

Provenance of every triple is recoverable by construction; foreign data can never be mistaken for local authority; evicting a peer is DROP GRAPH <urn:tar:peer:{id}>.

6. FAIR metadata and access descriptors

6.1 dcat:Distribution fields

Vocabulary audit (2026-08-30): every tar: term above was audited against the standard vocabularies — see 2026-08-30-vocabulary-audit.md. tar:availability is now always written beside dct:accessRights with the EU access-right authority table (the DCAT-AP reading); several other tar: terms were replaced outright (tar:atInstance → prov:wasAssociatedWith, tar:externalKey → dct:identifier, …).

6.2 metadata-only is the common case, not an edge case

For IDS's health-data work, most artifacts must be findable and described but not retrievable. tar:availability = metadata-only means: the registry advertises that the artifact exists, its type, its shape, its provenance chain, its responsible party, and tar:accessRequestURL — and carries no downloadURL at all. FAIR is not open, and the model says so structurally rather than by convention.

6.3 Signposting

Every artifact and software GET emits Signposting Link headers:

Link: <https://reg.ids.unimaas.nl/artifact/01J9G…>          ; rel="cite-as"
Link: <https://reg.ids.unimaas.nl/artifact/01J9G….ttl>      ; rel="describedby"; type="text/turtle"
Link: <https://reg.ids.unimaas.nl/artifact/01J9G….jsonld>   ; rel="describedby"; type="application/ld+json"
Link: <http://edamontology.org/data_2048>                    ; rel="type"
Link: <https://shacl.ids.unimaas.nl/reports/9f2a>            ; rel="item"; type="text/turtle"
Link: <https://spdx.org/licenses/CC-BY-4.0>                  ; rel="license"
Link: <https://orcid.org/0000-…>                             ; rel="author"
Link: <https://reg.ids.unimaas.nl/api/v1/registry>           ; rel="collection"

metadata-only artifacts omit rel="item" and add rel="describedby" only — a client can tell the difference between "no bytes here" and "bytes behind auth" without parsing the body.

6.4 FAIR principle coverage

7. API surface

Base path /api/v1. Every GET on a resource honours Accept for text/turtle, application/ld+json, application/json, text/html, and emits Signposting headers.

7.1 Identity and discovery

GET  /.well-known/tar-registry     registry self-description (JSON-LD)
GET  /api/v1/registry              dcat:Catalog record
GET  /healthz  /readyz  /metrics   liveness, readiness, Prometheus

7.2 Software and releases

POST   /api/v1/software                        register
GET    /api/v1/software                        list; q, license, publisher, edam_topic, keyword, kind
GET    /api/v1/software/{id}
PATCH  /api/v1/software/{id}
DELETE /api/v1/software/{id}                   soft delete (tombstone; IRI keeps resolving)
POST   /api/v1/software/{id}/releases
GET    /api/v1/software/{id}/releases
GET    /api/v1/software/{id}/export/biotools   biotoolsSchema JSON (§2.5)

7.3 Capabilities

PUT  /api/v1/software/{id}/capability          declare produces[] / consumes[]
PUT  /api/v1/instances/{id}/capability         narrow the inherited declaration
GET  /api/v1/capabilities?produces={typeIRI}&consumes={typeIRI}

GET /api/v1/capabilities is the matchmaking endpoint: "what can consume what shacl-manager emits?" It answers before any run exists, which is why D6 keeps it separate from lineage.

7.4 Instances

POST   /api/v1/instances                       register a deployment
GET    /api/v1/instances                       list; software, operator, status, release, registry
GET    /api/v1/instances/{id}
PATCH  /api/v1/instances/{id}
POST   /api/v1/instances/{id}/tokens           mint a scoped API token
GET    /api/v1/instances/{id}/runs
GET    /api/v1/instances/{id}/artifacts

7.5 Artifacts, runs, advertisement

POST /api/v1/artifacts                         register an artifact + distributions
GET  /api/v1/artifacts/{id}
GET  /api/v1/artifacts/{id}/lineage?depth=&direction=up|down|both
GET  /api/v1/runs/{id}

POST /api/v1/advertise/produced                requirement 4
POST /api/v1/advertise/consumed                requirement 5

Both advertisement endpoints are idempotent on (run_key, artifact_iri, role) — a retried CI step does not duplicate lineage. Both accept foreign IRIs in artifact position; that is how cross-registry lineage forms with no coordination.

POST /api/v1/advertise/produced:

{
  "run": {
    "external_key": "gh-actions/12345/attempt-1",
    "started_at": "2026-08-30T14:02:11Z",
    "ended_at":   "2026-08-30T14:02:49Z",
    "status": "success",
    "release": "https://reg.ids.unimaas.nl/release/01J8D…"
  },
  "artifacts": [{
    "title": "Validation report — patients.ttl vs fhir-shapes v3",
    "conforms_to": "http://edamontology.org/data_2048",
    "license": "https://spdx.org/licenses/CC-BY-4.0",
    "keywords": ["shacl", "validation", "fhir"],
    "was_derived_from": ["https://reg.mumc.nl/artifact/01J7Z…"],
    "distributions": [{
      "access_url":   "https://shacl.ids.unimaas.nl/reports/9f2a",
      "download_url": "https://shacl.ids.unimaas.nl/reports/9f2a.ttl",
      "media_type": "text/turtle",
      "byte_size": 2118342,
      "checksum": { "algorithm": "sha256", "value": "9f2a…" },
      "conforms_to": "https://reg.ids.unimaas.nl/shapes/validation-report",
      "access_protocol": "https",
      "auth_method": "apikey",
      "availability": "restricted",
      "access_request_url": "https://ids.unimaas.nl/data-access"
    }]
  }]
}

Response 201:

{
  "run": "https://reg.ids.unimaas.nl/run/01J9F…",
  "artifacts": ["https://reg.ids.unimaas.nl/artifact/01J9G…"],
  "created": true
}

POST /api/v1/advertise/consumed takes the same run block and an artifacts array whose entries are either a bare reference or a full inline registration:

{
  "run": { "external_key": "gh-actions/12345/attempt-1" },
  "artifacts": [
    { "iri": "https://reg.mumc.nl/artifact/01J7Z…" },
    { "title": "local input graph", "conforms_to": "http://edamontology.org/data_2600",
      "distributions": [{ "download_url": "s3://ids-bucket/in.ttl", "access_protocol": "s3" }] }
  ]
}

An unknown foreign IRI is stored verbatim and queued for background resolution. The advertisement never blocks on the network (§9.4).

The Instance is derived from the presenting token, never from the payload — see §8.3.

7.6 OpenLineage adapter

POST /api/v1/openlineage                       accepts an OpenLineage RunEvent

Rationale for the adapter-not-canonical shape (D7). OL coverage of our required fields:

Mapping performed on ingest:

7.7 Query

GET  /api/v1/search?q=&type=&federated=false   cross-entity, faceted
GET  /api/v1/graph?iri=&depth=                 subgraph for UI rendering
POST /sparql                                   read-only SPARQL 1.1 (Oxigraph)

POST /sparql is a first-class surface, not a bonus: it gives analysts and peer registries a standard federated query language without us designing one.

7.8 Federation

GET    /api/v1/peers
POST   /api/v1/peers                           admin: add by base URL, validated via well-known
DELETE /api/v1/peers/{id}                      DROP GRAPH <urn:tar:peer:{id}>
POST   /api/v1/peers/announce                  inbound mutual-discovery announcement
GET    /api/v1/peers/suggested                 peers-of-peers, for admin review
GET    /api/v1/resolve?iri=                    resolve a foreign IRI, cache a stub, return it

7.9 Errors

RFC 9457 application/problem+json throughout. SHACL write-validation failures return 422 with the validation report embedded as text/turtle in a report member — the same report format shacl-manager emits, so tooling is shared.

8. Auth model

8.1 Principals

8.2 Scopes

advertise:produce, advertise:consume, register:software, register:instance, read:private, admin:*. Tokens are minted per Instance, stored as Argon2id hashes in SQLite, shown once, revocable, optionally expiring.

Bootstrap: TAR_ROOT_TOKEN on first boot creates the initial admin; the registry refuses to start with a default or empty value.

8.3 Core authorisation rule

An Instance may only advertise runs in which it is itself the agent.

The Instance is taken from the presenting token and never from the request body. No principal can forge another deployment's lineage. Any job.namespace or instance field in a payload is retained as a label for debugging and ignored for authorisation.

Software records are editable by their creator or by a curator; ownership transfer is an admin action. Every write records prov:wasAttributedTo in the graph and an append-only row in the SQLite audit log (who, when, what, from where).

8.4 Federation trust

Peer data is always a read-only stub in <urn:tar:peer:{id}> and is never merged into <urn:tar:local>. A peer cannot create, modify, or delete local records. Inbound /api/v1/peers/announce only produces a suggestion for admin review — never an auto-added peer.

9. Federation model

1. Self-description. /.well-known/tar-registry returns JSON-LD: base IRI, title, operator, software version, public-read flag, SPARQL URL, capabilities, and the peer list.
2. Adding a peer. An admin POSTs a base URL. The registry fetches the well-known document, validates it, checks the advertised base IRI matches, and stores the peer. Optionally it POSTs /api/v1/peers/announce back for mutual discovery.
3. Cross-linking needs no resolution. An unknown foreign IRI in any object position is stored verbatim. Advertisement latency is never coupled to peer availability.
4. Lazy resolution. A background worker dereferences unknown foreign IRIs with Accept: text/turtle, writes a minimal stub (type, title, publisher, home registry) into the peer graph, and caches with a TTL (TAR_PEER_RESOLVE_TTL, default 24 h). Failures back off exponentially and are visible in the peer admin UI.
5. Peers of peers. The resolver reads a peer's advertised peer list and surfaces them at /api/v1/peers/suggested. They are never auto-added — the trust boundary stays manual (D9).
6. Federated search. ?federated=true fans out to peers' /api/v1/search with TAR_FEDERATED_SEARCH_TIMEOUT (default 3 s), returning partial results with a partial: true flag and a per-peer status list. Power users use SPARQL SERVICE.
7. Conflict rule. The registry that minted an IRI is authoritative for it. Stubs never overwrite local triples. If two registries describe the same Software, the Software IRIs differ and are linked by owl:sameAs / schema:sameAs — set deliberately by a curator, never inferred.

10. Deployment

10.1 Artifact

One statically linked Rust binary in a distroless image (~30 MB). Volumes: /data (Oxigraph) and /data/ops.db (SQLite).

10.2 Compose

services:
  registry:
    image: ghcr.io/maastrichtu-ids/tool-artifact-registry:0.1.0
    environment:
      TAR_BASE_IRI: https://reg.example.org
      TAR_ROOT_TOKEN: ${TAR_ROOT_TOKEN:?set me}
    ports: ["8080:8080"]
    volumes: ["tar-data:/data"]
    healthcheck:
      test: ["CMD", "/tar", "healthcheck"]
volumes: { tar-data: }

One service. That is the whole minimal install (req. 6).

10.3 Helm

Chart at deploy/helm/tool-artifact-registry: Deployment (1 replica — Oxigraph is single-writer, strategy: Recreate), RWO PVC, Service, Ingress, ConfigMap, Secret, ServiceMonitor. Probes on /healthz and /readyz.

10.4 ids3

Kustomize overlay at services/ids3/projects/tool-artifact-registry/ following the existing _base project pattern; an ArgoCD Application; Vault + VSO for TAR_ROOT_TOKEN and the OIDC client secret; egress through egress-proxy for peer resolution and forge polling; Harbor for the image; Traefik for ingress and TLS.

10.5 Configuration

Everything but TAR_BASE_IRI and TAR_ROOT_TOKEN has a working default (req. 6, sane defaults).

10.6 Operations

• Backup: GET /admin/dump streams N-Quads (all graphs, or ?graph=); SQLite via .backup.
• Restore: tar restore --nquads dump.nq --ops ops.db.
• Upgrade: graph migrations are additive SPARQL Updates shipped per release and applied idempotently on boot; SQLite via sqlx migrate.
• Scaling: read-heavy growth is answered by the GraphStore trait and a remote-SPARQL backend (§5.3), not by replicas.

10.7 Seed data

tar seed --from ids-examples registers the sibling repos as Software with declared capabilities, so a fresh install is demonstrable immediately:

11. Frontend

v1 covers browse/search, registration and editing, and peer administration. Layout is two-column with a sticky right rail; Instances is a top-level tab alongside Software, Artifacts, Runs and Peers. Lineage graph visualisation is deferred to v2 — v1 renders the same data as tables.

Full screen inventory, routes, component list, API contracts per screen, and states: docs/design-handoff.md.

12. Open questions

13. Out of scope for v1

Artifact byte storage; workflow execution; access granting/brokering; multi-tenancy; lineage graph visualisation; harvest-based federation; DOI minting; signed advertisements; horizontal write scaling.