Protocol

Overview

This protocol is a fundamental protocol that corresponds to ODS-RAM L4 Metadata Exchange and facilitates the exchange of metadata relating to the location and semantics of data, enabling data discoverability and semantic interpretation in distributed environments.

RDF (Resource Description Framework) is adopted as the information model for metadata, and JSON-LD and Turtle are adopted as data model formats. By publishing this metadata, providers of data and services enable easier discovery, interpretation, and utilization of data and services. Furthermore, by using higher-level protocols such as Discovery and Search on top of this foundational protocol, even if data and services are distributed across a network, the entire Open Dataspaces can function transparently as a single system. This metadata is primarily intended to associate real-world entities such as mobility and supply chains with access to data and services in the cloud; however, the former is not mandatory.

concept 1

Through the mandatory metadata concerning services and access, the external specifications of services are grounded in a globally resolvable manner, and the necessary information for access, including destination information, becomes explicit.

concept 2

These metadata components are constructed on the side of general web application servers and clients. A metadata server statically or dynamically retrieves metadata from an application server.

A metadata client provides metadata to an application client. As a design approach, while conventional web application systems are database-oriented and required to satisfy system requirements such as transactions, real-time processing, and high throughput, metadata-related systems shall prioritize the discovery of necessary data from a wide variety of data across broad-scale environments, and as a trade-off for this, consistency of data and similar properties shall initially be sacrificed and added as needed. It is assumed that use cases requiring strict consistency management of metadata are limited, since diverse real-world data and existing application server data are transformed and replicated into a unified metadata format.

concept 3

Abstract Normative Specification

Concepts and Roles

Role	Description
Business Domain	A specific business domain. Common services such as Discovery Service may be constructed per business domain.
Provider	Provides data or services related to the business domain and may manage multiple resources.
Consumer	Uses data or services provided by a Provider.
Resource	Information managed on Open Dataspaces that corresponds to physical or logical entities in the real world. Examples include physical entities such as drones, logical entities such as routes, and inter-company commercial relationships. Resources are searchable targets of the Discovery Service within Open Dataspaces.
Metadata Server	Publishes metadata as RDF. Accepts external retrieval requests at a Metadata Endpoint. Retrieves static data from a Metadata Store and dynamic data from an Application Server as needed.
Metadata Store	Stores metadata as RDF and mainly handles static data.
Metadata Client	Retrieves metadata, for example through a JavaScript library in a browser.
Metadata Endpoint	RDF retrieval endpoint in a Metadata Server. Includes a SPARQL endpoint.
Application Server	Business application server of a Provider. Includes general application servers.
Application Client	Client that uses business applications, such as dedicated software or a browser.

Scope

• RDF metadata publication

• RDF Patch synchronization

• SPARQL retrieval

Normative Requirements

• Metadata SHALL be represented in RDF.

  • Rationale: To unify semantic representation and ensure interoperability.

• Metadata SHALL include globally resolvable semantic definitions of interfaces related to data exchange or service provision, for example SAMM.

  • Rationale: Ambiguous interface semantics may lead to unintended usage.

• Metadata SHALL include destination information of interfaces held by data or service providers.

  • Rationale: To enable transparent access to distributed services within a unified Open Dataspace under appropriate access control.

• Metadata SHOULD be transmitted and received in JSON-LD format.

  • Rationale: High compatibility with web-based implementations and extensive track record of adoption.

• If a provider is associated with a physical or logical entity in the real world, this SHOULD be included as metadata.

  • Rationale: Enables association of real-world entities with network services.

• If such entities are related to entities of other providers, these reference relationships SHOULD be included as metadata.

  • Rationale: Following reference relationships enables discovery of other providers and enhances overall interoperability.

• A mechanism for managing metadata differences MAY be provided, such as RDF Patch. Eventual consistency is assumed. Lineage, timestamps, and version control SHOULD be used to manage differences when necessary.

  • Rationale: While strong consistency is not required, metadata freshness may be important.

Non-functional / Cross-layer Requirements

• It SHALL be capable of integrating with Identity & Trust（L3）.

• If access control is required for metadata itself, it SHALL integrate with Identity & Trust（L3） and authorization management SHALL be applied.

• The RDF model SHALL conform to the ODS semantics policy.

Metadata

Metadata is provided from systems that offer data or services to a Metadata Server and is then published. It becomes referable from other systems by referencing metadata of other service providers or by being published to a Discovery Service.

Metadata consists of service metadata related to semantic definitions and destination information of data and services, and resource metadata corresponding to real-world entities. Service metadata consists of descriptions of the service specification itself and the deployed state (instance) of that specification in a given environment. The description of the service specification itself may be stored in another Metadata Server and expressed by referencing it. Specifications unique to a particular service may be stored in the corresponding Metadata Server. Service metadata may be referenced from resource metadata. Service specifications are described as Aspect Models using SAMM or similar approaches.

Overall Structure of Service Metadata

Service metadata consists of two parts:

{
  "@context": { ... },
  "@graph": [ ... ]
}

Item	Type	Required	Description
@context	object	Yes	Namespace definitions
@graph	array	Yes	Metadata contents related to data/service providers

Namespace Definition @context

@context defines the mapping between prefixes used in metadata and their corresponding namespaces.

The following namespaces are used:

• @prefix http: <http://www.w3.org/2011/http#> .

• @prefix dcterms: <http://purl.org/dc/terms/> .

• @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

• @prefix ods: <https://github.com/ODS-DFS-L4/ods/> .

• Namespace defining the Aspect Model, for example: @prefix uasl: <urn:samm:com.foo.uasl:0.5.3#> .

• Namespace used by the domain application API definition file, for example: @prefix api: <https://uasl.foo.com/0.5.3/api#> .

• Namespace for the domain application (explicit company namespace), for example: @prefix inst: <http://www.example.com/uasl/instance#> .

  • The default prefix is not used; the prefix inst is explicitly used.

Domain application instances are defined as type ods:DomainApp.

• dcterms:conformsTo is used to explicitly link to the Aspect Model and API definition.

• Properties defined in the Aspect Model for the domain application are assigned actual values.

• ods:hasBaseEndpoint indicates the base endpoint of the API.

  • The API base endpoint is defined as class ods:BaseEndpoint (a subclass of ods:DomainAppEndpoint).

• ods:hasSparqlEndpoint indicates the SPARQL endpoint.

  • The SPARQL endpoint is defined as class ods:SparqlEndpoint (a subclass of ods:DomainAppEndpoint).

• ods:hasEndpoint indicates endpoints for each API.

  • Each API endpoint is defined as class ods:DomainAppEndpoint (a subclass of ods:DomainAppEndpoint).

• ods:accessURL specifies the URL of each endpoint.

Key	Type	Description
dcterms	string (URI)	Dublin Core Terms
rdf	string (URI)	RDF namespace
api	string (URI)	API identification namespace
inst	string (URI)	Instance identification namespace
ods	string (URI)	ODS (Open Dataspace) related
service	string (URI)	Service definition

Metadata Related to Data/Service Providers @graph

@graph consists of multiple objects:

• DomainApp: Represents the entire domain application.

• DomainAppEndpoint: Represents URI information of endpoints defined by SAMM.

• BaseEndpoint: Represents the common URI part of REST API endpoints.

• SparqlEndpoint: Represents endpoint information for SPARQL.

DomainApp

DomainApp represents the entire domain application.

Structure

{
  "@id": "inst:MyServiceProd",
  "@type": "ods:DomainApp",
  "dcterms:conformsTo": [ ... ],
  "ods:hasBaseEndpoint": { ... },
  "ods:hasEndpoint": [ ... ],
  "ods:hasSparqlEndpoint": { ... },
  "service:serviceProviderName": { ... }
}

DomainAppEndpoint

DomainAppEndpoint represents URI information of endpoints defined by SAMM.

Common Structure

{
  "@id": "inst:xxxEndpoint",
  "@type": "ods:DomainAppEndpoint",
  "service:xxxRequestUri": {
    "@type": "http://www.w3.org/2001/XMLSchema#anyURI",
    "@value": "https://example.com/api/v1/path"
  }
}

BaseEndpoint

BaseEndpoint represents the common URI part of REST API endpoints.

{
  "@id": "inst:myServiceBaseEndpoint",
  "@type": "ods:BaseEndpoint",
  "ods:accessURL": {
    "@type": "http://www.w3.org/2001/XMLSchema#anyURI",
    "@value": "https://example.com/api/v1"
  }
}

SparqlEndpoint

SparqlEndpoint represents endpoint information for SPARQL.

{
  "@id": "inst:myServiceSparqlEndpoint",
  "@type": "ods:SparqlEndpoint",
  "ods:accessURL": {
    "@type": "http://www.w3.org/2001/XMLSchema#anyURI",
    "@value": "https://example.com/sparql"
  }
}

Resource Metadata

Resource metadata represents domain data generated and managed by applications, expressed in RDF and given meaning using a shared vocabulary. Since it is domain-dependent, concrete examples are provided in binding.md.

Common Specification

(1) Basic Structure

• Data is expressed in RDF using JSON-LD.

• Each file follows the basic structure below:

{
  "@context": { ... },
  "@graph": [ ... ]
}

Element	Description
@context	Namespace and vocabulary definitions
@graph	Array of RDF resources

ID Representation

• Each entity is uniquely identified by @id.

• HTTP URI format is adopted.

Example:

http://localhost:8080/resource/aircraft/{UUID}

Data Types

• Literal values are explicitly expressed using @value and @type.

• Typical types:

  • xsd:string

  • xsd:integer

  • xsd:double

  • xsd:boolean

  • xsd:dateTime

Message Types

Type	Sender ⇒ Receiver	Description
sparqlQuery	Metadata Client ⇒ Metadata Endpoint	RDF retrieval
patchNotify	Metadata Endpoint ⇒ Metadata Client	Differential notification

Protocol Flow

Error Handling

This section describes the error handling used in this protocol.

Error Type	Description	Handling
RDF format error	Invalid JSON-LD	Correct input
Endpoint unreachable	Network issue	Retry
Patch inconsistency	Differential order issue	Resynchronize

References

Michael Franklin, Alon Halevy, David Maier, "From Databases to Dataspaces: A New Abstraction for Information Management", SIGMOD Record, Vol. 34, N