Checking a resource
Transreption ended on a rule: the format is a property of the route, not
of the resource. Follow it one step further. If a route reaches Turtle, the resource has a
Turtle face, whatever it was written in, and anything with a Turtle face can be
checked: does it parse, and does what it says fit the shapes it is supposed to fit. This
chapter calls that a CHECK. The RDF Toolbox
(ikigai-toolbox) is the page that does it in a
browser, and this chapter builds what the page reads and how it reads it.
The code is crates/check-resource. Unlike the other chapters of this part, it does not chain
onto Part I’s space: its host holds two players of tic-tac-toe, a transreptor, the shapes a player
must fit, and two published spaces beside them. Nothing in it starts a server, so there is nothing
to stop afterwards.
/// The chapter's host: its own doors, then the toolbox's names, then the shapes check.
pub fn space() -> Fallback {
let ours = EndpointSpace::new()
.bind(Exact::new(PLAYER_X), player("player-x", X_TRIPLES))
.bind(Exact::new(PLAYER_O), player("player-o", O_TRIPLES))
.bind(Exact::new(SHAPES), shapes())
.bind(Exact::new(TO_TURTLE), to_turtle());
Fallback::new(vec![
Arc::new(ours) as Arc<dyn Space>,
Arc::new(ikigai_toolbox::space()) as Arc<dyn Space>,
Arc::new(ikigai_shacl::space()) as Arc<dyn Space>,
])
}A resource with no Turtle in it
Each player answers N-Triples, one triple per line, and nothing else. Reading one requires a
scope, declared the way Multi-verb endpoints declared the counter’s write: the
kernel enforces the requires, so the endpoint’s code holds no check of its own.
/// The two players, as N-Triples. O carries one property the shapes do not name.
const X_TRIPLES: &str = r#"<urn:iki:tutorial:check:player:x> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://example.org/ttt#Player> .
<urn:iki:tutorial:check:player:x> <http://example.org/ttt#name> "Ada" .
<urn:iki:tutorial:check:player:x> <http://example.org/ttt#mark> "X" .
"#;
const O_TRIPLES: &str = r#"<urn:iki:tutorial:check:player:o> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://example.org/ttt#Player> .
<urn:iki:tutorial:check:player:o> <http://example.org/ttt#name> "Grace" .
<urn:iki:tutorial:check:player:o> <http://example.org/ttt#mark> "O" .
<urn:iki:tutorial:check:player:o> <http://example.org/ttt#wins> "3" .
"#;
/// A player: a graph in N-Triples, readable only by a caller holding [`READ`]. Nothing in
/// it knows Turtle exists.
fn player(id: &'static str, triples: &'static str) -> FnEndpoint {
FnEndpoint::new(id, move |_: &Invocation<'_>| {
Ok(Representation::new(ReprType::new(N_TRIPLES), triples.as_bytes().to_vec()).cacheable())
})
.with_description(
Description::new(id)
.title("A player")
.summary("One player of tic-tac-toe, as an RDF graph in N-Triples.")
.verb(Verb::Source)
.verb(Verb::Meta)
.output(N_TRIPLES)
.requires(READ),
)
}The Turtle face comes from a transreptor, declared lossless because every triple survives the trip. It is the kind of endpoint Transreption wrote, in the direction that reaches the pivot:
/// A transreptor from N-Triples to Turtle: the same triples, written with prefixes.
/// Lossless (every triple survives), so it is declared with no `.lossy()`.
pub fn to_turtle() -> FnEndpoint {
FnEndpoint::new("to-turtle", |inv: &Invocation<'_>| {
let content = inv.inline_str("content")?;
let invalid = |detail: String| Error::InvalidArgument {
name: "content".into(),
detail,
};
let mut writer = oxttl::TurtleSerializer::new()
.with_prefix("ttt", "http://example.org/ttt#")
.map_err(|e| Error::Endpoint(e.to_string()))?
.for_writer(Vec::new());
for triple in oxttl::NTriplesParser::new().for_slice(content.as_bytes()) {
let triple = triple.map_err(|e| invalid(e.to_string()))?;
writer
.serialize_triple(&triple)
.map_err(|e| Error::Endpoint(e.to_string()))?;
}
let turtle = writer
.finish()
.map_err(|e| Error::Endpoint(e.to_string()))?;
Ok(Representation::new(ReprType::new(TURTLE), turtle).cacheable())
})
.with_description(
Description::new("to-turtle")
.title("N-Triples to Turtle")
.summary("The same triples, written as Turtle.")
.verb(Verb::Source)
.verb(Verb::Meta)
.input(ArgSpec::new("content").summary("the N-Triples document"))
.input(
ArgSpec::new("as")
.summary("the target type; only text/turtle")
.optional(),
)
.output(TURTLE)
.transreptor([N_TRIPLES], [TURTLE]),
)
}The check reads as you
The toolbox reads a resource through one name, urn:toolbox:input:{iri}: the resource sourced
with as=text/turtle, and when it answers in another RDF syntax, carried to Turtle by whatever
transreptor plan the kernel finds. The name declares no capability of its own. It issues that
read under the caller’s capability, so a check never shows a reader more than they could
read directly, and the toolbox is not a way around a scope.
/// A resource's Turtle face, as the toolbox page reads it: `urn:toolbox:input:{iri}`,
/// issued under the caller's own capability.
pub async fn turtle_face(kernel: &Kernel, iri: &str, caller: &Capability) -> Result<String> {
let input = Iri::parse(format!("urn:toolbox:input:{iri}"))
.map_err(|e| Error::Endpoint(format!("`{iri}` makes no input name: {e}")))?;
let face = kernel
.issue(Request::new(Verb::Source, input), caller)
.await?;
String::from_utf8(face.bytes).map_err(|e| Error::Endpoint(format!("not UTF-8: {e}")))
}#![allow(unused)]
fn main() {
extern crate check_resource;
extern crate ikigai_core;
extern crate futures;
use futures::executor::block_on;
use ikigai_core::{Capability, Error};
use check_resource::{kernel, turtle_face, PLAYER_X, READ};
let kernel = kernel();
// A reader holding the scope gets the player's Turtle face: N-Triples in, Turtle out,
// and nothing in the player knows Turtle exists.
let reader = Capability::scoped([READ]);
let face = block_on(turtle_face(&kernel, PLAYER_X, &reader)).unwrap();
assert_eq!(
face,
"@prefix ttt: <http://example.org/ttt#> .\n\
<urn:iki:tutorial:check:player:x> a ttt:Player ;\n\
\tttt:name \"Ada\" ;\n\
\tttt:mark \"X\" .\n"
);
// A caller without it is refused, and the refusal names the player, not the toolbox.
let stranger = Capability::scoped(["urn:cap:tutorial:counter:write"]);
let refused = block_on(turtle_face(&kernel, PLAYER_X, &stranger)).unwrap_err();
assert!(matches!(refused, Error::Denied(_)));
assert_eq!(
refused.to_string(),
"denied: capability does not grant `urn:cap:tutorial:players:read` \
(declared by `urn:iki:tutorial:check:player:x`)"
);
}This page runs the same two reads. Its kernel holds the chapter’s host beside the spaces the
book’s other chapters run in, and cap narrows the page’s session the way it narrows a shell’s,
until cap reset gives the authority back. Run it, then run it again: the reader’s face comes
from the cache the second time, because the cache keys on the capability as well as the name.
narrowed — capability: urn:cap:tutorial:players:read @prefix ttt: <http://example.org/ttt#> . <urn:iki:tutorial:check:player:x> a ttt:Player ; ttt:name "Ada" ; ttt:mark "X" . [computed] reset to identity — capability: root (full authority) narrowed — capability: urn:cap:tutorial:counter:write error: denied: capability does not grant `urn:cap:tutorial:players:read` (declared by `urn:iki:tutorial:check:player:x`) reset to identity — capability: root (full authority)
Shapes, in a paragraph
SHACL describes what a graph should look like, in a graph of its
own. A node shape picks the nodes it applies to (here, sh:targetClass: every
ttt:Player), and its property shapes constrain what hangs off each one: how many values a
predicate may have, of which datatype, from which list. A shape is closed when it also says
sh:closed true: then a node may carry only the predicates its property shapes name, and those
listed in sh:ignoredProperties, and any other triple is a violation. An open shape lets a
stray property through unremarked; a closed one turns it into a finding. That is why
ikigai-layers requires sh:closed true on every node shape of a layer: an entry carries exactly
what its layer declared, and a property nobody declared is refused rather than kept. The player
shapes are closed the same way:
/// One closed node shape: a player has exactly one name and exactly one mark, X or O,
/// and nothing else but its type.
pub const PLAYER_SHAPES: &str = r#"@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ttt: <http://example.org/ttt#> .
ttt:PlayerShape a sh:NodeShape ;
sh:targetClass ttt:Player ;
sh:closed true ;
sh:ignoredProperties ( rdf:type ) ;
sh:property [ sh:path ttt:name ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ;
sh:property [ sh:path ttt:mark ; sh:in ( "X" "O" ) ; sh:minCount 1 ; sh:maxCount 1 ] .
"#;Checking data against shapes
The toolbox page takes a second input for this (shapes=), read exactly like the first, and
checks one against the other. The page does the checking in your browser tab, with
turtle-lsp compiled to WebAssembly, and no Rust
listing can run a browser tab. So here the shapes check runs on the host instead, through
urn:shacl:validate from ikigai-shacl. The inputs
are the same and are read the same way, as Turtle faces under one caller’s capability, but the
engine is a different one:
/// CHECK `iri` against the shapes at `shapes`: both read as Turtle faces under `caller`'s
/// capability, the way the check page reads its two inputs, then `urn:shacl:validate`.
pub async fn check(
kernel: &Kernel,
iri: &str,
shapes: &str,
caller: &Capability,
) -> Result<Report> {
let data = turtle_face(kernel, iri, caller).await?;
let shapes = turtle_face(kernel, shapes, caller).await?;
let validate = Request::new(
Verb::Source,
Iri::parse("urn:shacl:validate").expect("an IRI"),
)
.with_arg("data", ArgRef::Inline(data.into_bytes()))
.with_arg("shapes", ArgRef::Inline(shapes.into_bytes()))
.with_arg("as", ArgRef::Inline(b"application/json".to_vec()));
let report = kernel.issue(validate, caller).await?;
serde_json::from_slice(&report.bytes).map_err(|e| Error::Endpoint(format!("report: {e}")))
}#![allow(unused)]
fn main() {
extern crate check_resource;
extern crate ikigai_core;
extern crate futures;
use futures::executor::block_on;
use ikigai_core::Capability;
use check_resource::{check, kernel, PLAYER_O, PLAYER_X, READ, SHAPES};
let kernel = kernel();
let reader = Capability::scoped([READ]);
// X has a name, a mark and a type, and nothing else: it conforms.
let x = block_on(check(&kernel, PLAYER_X, SHAPES, &reader)).unwrap();
assert!(x.conforms);
// O carries `ttt:wins`, which the closed shape does not name: one violation.
let o = block_on(check(&kernel, PLAYER_O, SHAPES, &reader)).unwrap();
assert!(!o.conforms);
let found = &o.outcome().violations[..];
assert_eq!(found.len(), 1);
assert_eq!(found[0].focus_node, PLAYER_O);
assert_eq!(found[0].path.as_deref(), Some("http://example.org/ttt#wins"));
assert_eq!(found[0].component, "http://www.w3.org/ns/shacl#ClosedConstraintComponent");
}The same check runs in this page, written the way a shell writes it: the Turtle face piped
into urn:shacl:validate, where it fills data, the one required input left unnamed. X
conforms. O’s report is the graph the paragraph below describes, and its one result names the
property: sh:resultPath ttt:wins, from sh:ClosedConstraintComponent. The toolbox’s own
engine checks the same two players further down, in The page itself.
{
"conforms": true,
"violations": []
}
[2 computed]
@prefix sh: <http://www.w3.org/ns/shacl#> .
<urn:ikigai:shacl:report:b3:1258eb4246e33211b9d99fd84328c8723a8aae4c2627e1c1973fca3041184544:report> a sh:ValidationReport ;
sh:conforms false ;
sh:result <urn:ikigai:shacl:report:b3:1258eb4246e33211b9d99fd84328c8723a8aae4c2627e1c1973fca3041184544:result.0> .
<urn:ikigai:shacl:report:b3:1258eb4246e33211b9d99fd84328c8723a8aae4c2627e1c1973fca3041184544:result.0> a sh:ValidationResult ;
sh:focusNode <urn:iki:tutorial:check:player:o> ;
sh:resultMessage "Closed constraint not satisfied" ;
sh:resultPath <http://example.org/ttt#wins> ;
sh:resultSeverity sh:Violation ;
sh:sourceConstraintComponent sh:ClosedConstraintComponent ;
sh:sourceShape <http://example.org/ttt#PlayerShape> ;
sh:value "3" .
[2 computed]
The report is data, like every other answer in this book: as=application/json gives the
structure above, and the default, text/turtle, is the same findings as a SHACL validation
report graph you can query.
ikigai-shacl is linked here with default-features = false, which builds its subset
validator: closed shapes and the common constraints, with no network stack. A term outside the
subset is refused by name rather than skipped. The default build links
rudof and covers all of SHACL Core.
The page itself
On a host that serves the toolbox, the check is one link. urn:toolbox:check:{iri} is the page
checking that resource, shapes= adds the shapes, and a host routes /toolbox/check/{iri} to it,
so the check of player O against the player shapes is:
/toolbox/check/urn%3Aiki%3Atutorial%3Acheck%3Aplayer%3Ao?shapes=urn%3Aiki%3Atutorial%3Acheck%3Ashapes#shacl
The page reads both inputs through urn:toolbox:input:{iri}, as your own browser session, so
what you saw above is what it sees. What the page needs that this crate does not have is the
build: the page, its scripts and the 8.3 MB engine, which ikigai-toolbox’s build.sh
writes to a directory a host binds at urn:iki:toolbox:dist:{path}. This crate binds no build,
so asking it for the page fails on the first file the page is composed from:
#![allow(unused)]
fn main() {
extern crate check_resource;
extern crate ikigai_core;
extern crate futures;
use futures::executor::block_on;
use ikigai_core::{Capability, Error, Iri, Request, Verb};
let page = Iri::parse("urn:toolbox:check:urn:iki:tutorial:check:player:o").unwrap();
let err = block_on(check_resource::kernel().issue(Request::new(Verb::Source, page), &Capability::root()))
.unwrap_err();
assert!(matches!(err, Error::Unresolved(ref name) if name.as_str() == "urn:iki:toolbox:dist:page.template.html"));
}This book does bind one. It publishes a build of ikigai-toolbox 0.1.0 beside its pages, in
toolbox/, and the page’s kernel binds urn:iki:toolbox:dist:{path} to it: a door that asks the
browser for toolbox/{path} when, and only when, a name needs one of its files.
/// Where the toolbox reads its build, `urn:iki:toolbox:dist:{path}` (`ikigai_toolbox::DIST`):
/// in the page, the published copy of `ikigai-toolbox`'s `build.sh` output, which pages.yml
/// puts in `toolbox/` beside the book's root and which the browser fetches only when a name
/// asks for one of its files. So `urn:toolbox:check:{iri}` composes in the page, from the
/// build's own template, and nothing is fetched until a reader opens a check.
///
/// The kernel's futures must be `Send` and a browser fetch is not, so the fetch runs as a task
/// of the page's own (`spawn_local`) and hands its answer back over a channel, which is.
#[cfg(target_family = "wasm")]
fn dist_space() -> Arc<dyn Space> {
use ikigai_core::{AsyncFnEndpoint, Description, EndpointSpace, InvokeFuture, UriTemplate};
let fetched = AsyncFnEndpoint::new(
"book-toolbox-dist",
|inv: &ikigai_core::Invocation<'_>| -> InvokeFuture<'_> {
let path = inv.bindings.get("path").unwrap_or_default().to_string();
let (answer, answered) = futures::channel::oneshot::channel();
wasm_bindgen_futures::spawn_local(async move {
let _ = answer.send(fetch_published(&path).await);
});
Box::pin(async move {
answered
.await
.unwrap_or_else(|_| Err(Error::Endpoint("the fetch was dropped".into())))
})
},
)
.with_description(
Description::new("book-toolbox-dist")
.title("The toolbox build this book publishes")
.summary(
"One file of the RDF Toolbox build published beside this book (`toolbox/`), \
fetched by the browser when it is asked for.",
)
.verb(Verb::Source)
.verb(Verb::Meta)
.output("application/octet-stream"),
);
Arc::new(EndpointSpace::new().bind(
UriTemplate::parse("urn:iki:toolbox:dist:{path}").expect("a constant template"),
fetched,
))
}
/// Built natively there is no published build to fetch, so nothing is bound there: a check
/// page fails on its template exactly as the chapter's own host's does.
#[cfg(not(target_family = "wasm"))]
fn dist_space() -> Arc<dyn Space> {
Arc::new(ikigai_core::EndpointSpace::new())
}
/// The bytes of `toolbox/{path}` beside the book's root, as the browser fetches them. A name
/// that is not one plain file name is not found without a request: the toolbox asks only for
/// the files on its allowlist, and nothing else under the site is the dist's to serve.
#[cfg(target_family = "wasm")]
async fn fetch_published(path: &str) -> Result<Representation, Error> {
use js_sys::{Function, Promise, Reflect, Uint8Array};
use wasm_bindgen_futures::JsFuture;
#[wasm_bindgen]
extern "C" {
#[wasm_bindgen(js_name = fetch)]
fn fetch_url(url: &str) -> Promise;
}
let plain = !path.is_empty()
&& !path.starts_with('.')
&& path
.chars()
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '-' | '_'));
if !plain {
return Err(Error::NotFound(format!(
"urn:iki:toolbox:dist:{path}: not a file of the published build"
)));
}
let failed = |e: JsValue| Error::Endpoint(format!("toolbox/{path}: {e:?}"));
// Where the build is published, as `js/toolbox.js` says (`window.ikigaiToolboxBase`, the
// absolute path of `toolbox/` beside the book's root). mdbook's `path_to_root` would do,
// but it is a `const` in the page's head, which is not a property of the global object.
let base = Reflect::get(&js_sys::global(), &JsValue::from_str("ikigaiToolboxBase"))
.ok()
.and_then(|v| v.as_string())
.ok_or_else(|| {
Error::NotFound(format!(
"urn:iki:toolbox:dist:{path}: this page does not say where the toolbox build is \
published (js/toolbox.js sets window.ikigaiToolboxBase)"
))
})?;
let url = format!("{base}{path}");
let response = JsFuture::from(fetch_url(&url)).await.map_err(failed)?;
let field = |name: &str| Reflect::get(&response, &JsValue::from_str(name));
if !field("ok").map_err(failed)?.as_bool().unwrap_or(false) {
let status = field("status").map_err(failed)?.as_f64().unwrap_or(0.0);
return Err(Error::NotFound(format!("{url}: HTTP {status}")));
}
let read: Function = field("arrayBuffer")
.map_err(failed)?
.dyn_into()
.map_err(failed)?;
let pending: Promise = read
.call0(&response)
.map_err(failed)?
.dyn_into()
.map_err(failed)?;
let buffer = JsFuture::from(pending).await.map_err(failed)?;
Ok(Representation::new(
ikigai_core::ReprType::new("application/octet-stream"),
Uint8Array::new(&buffer).to_vec(),
))
}So the page is a host too. Pressing a “Check … here” button below asks this page’s kernel for
urn:toolbox:check:{iri} with shapes=, which composes the toolbox page from the build’s own
template. The page then does what a server’s routing would do: it points the composed page’s
/toolbox/… files at the published build, and answers its two inputs, /toolbox/input/{iri},
with urn:toolbox:input:{iri} from the same kernel. The toolbox runs in a panel that opens in
place, and its engine checks the two Turtle faces in your tab. Nothing of the toolbox is loaded
until you press one: the first press fetches its scripts and the 8.3 MB engine, and Close (or
Escape) puts focus back on the button.
Check player O against the player shapes in the RDF Toolbox, in a new tab, on copies of the two Turtle faces published with this page.
Check player X against the player shapes in the RDF Toolbox, in a new tab, on copies of the two Turtle faces published with this page.
The SHACL tab of O’s check shows the same finding as the cell above, worded by turtle-lsp. In the page, you hold the root of your own kernel, so the panel reads both players. Without the kernel (it did not load, or scripts are off), each button is a link instead, and the toolbox opens in a new tab on copies of the Turtle faces published beside this chapter. A test holds those copies equal to what the kernel answers a reader who holds the scope, but a copy is not a read: nothing checks your capability on a file anyone can download.
gonk, the host The gonk Book is about, serves the page at /toolbox when its configuration
names a build (gonk.toolbox.dist).
A CHECK is not urn:kernel:validate
The kernel has a name that sounds like this chapter and is not: urn:kernel:validate. It checks
a request against the contract of the action it would invoke, before anything runs: required
inputs present, values from the declared lists, no unknown arguments, the capability sufficient.
A CHECK reads a resource that already exists and asks whether what it says fits a set of
shapes. One asks “may this call run as written?”, the other “is this data what it should be?”.
Both answer in SHACL’s report vocabulary, which is where the confusion starts: the focus of a
urn:kernel:validate result is an action’s contract, and the focus of a CHECK result is a node
in your data. The machine client, later in the book, puts
urn:kernel:validate to work as the last step before an agent’s call runs. Here it is
pre-flighting a call to urn:shacl:validate that forgot its data and asked for a face the
endpoint does not have:
#![allow(unused)]
fn main() {
extern crate check_resource;
extern crate ikigai_core;
extern crate futures;
use futures::executor::block_on;
use ikigai_core::{ArgRef, Capability, Iri, Request, Verb};
let preflight = Request::new(Verb::Source, Iri::parse("urn:kernel:validate").unwrap())
.with_arg("endpoint", ArgRef::Inline(b"urn:shacl:validate".to_vec()))
.with_arg("verb", ArgRef::Inline(b"source".to_vec()))
.with_arg("args", ArgRef::Inline(b"shapes=urn:iki:tutorial:check:shapes&as=text/csv".to_vec()));
let report = block_on(check_resource::kernel().issue(preflight, &Capability::root())).unwrap();
let report = String::from_utf8(report.bytes).unwrap();
assert!(report.contains("sh:conforms false"));
assert!(report.contains("required input `data` is missing"));
assert!(report.contains(
"`text/csv` is not an accepted value of `as` (one of: text/turtle, application/json)"
));
}Nothing was validated against shapes there, and no player was read: the request never ran.
What not to assume
- Three places run a check here, and they are not the same. The listings run under
mdbook test, on the host side. The cells run in your tab, in the page’s kernel, withikigai-shacl’s subset. The panel runs the toolbox page and its engine in your tab, served by the page as its host. - Two engines, one question. The page’s SHACL engine is turtle-lsp’s; the listings’ is
ikigai-shacl’s subset. Both check SHACL Core’s closed shapes, but they are different code, and their messages differ even where their findings agree. - A check sees what you may read. A reader who cannot read a resource cannot check it, and the refusal is the resource’s, as above. That is the design, not a gap.
A caller holding some other scope checks nothing, and the refusal comes from the player, before any shapes are read:
narrowed — capability: urn:cap:tutorial:counter:write error: denied: capability does not grant `urn:cap:tutorial:players:read` (declared by `urn:iki:tutorial:check:player:o`) reset to identity — capability: root (full authority)