Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Views from CSV: one model, two sources

Views from JSON built a table of the planets over a JSON document, and its view template named records:{source} instead of the document. This chapter supplies a second source, a CSV table of the same planets, and shows the view that does not care which one it is given.

The data, as it is

The CSV table is an atom of its own, with a header row and one row per planet:

name,kind,year
Mercury,terrestrial,88
Venus,terrestrial,225
Earth,terrestrial,365
Mars,terrestrial,687
Jupiter,gas giant,4333
Saturn,gas giant,10759
Uranus,ice giant,30687
Neptune,ice giant,60190
[computed]

One model

The view’s model is the one the JSON chapter used: an array of records, each a set of named fields. A JSON document of objects already is one. A CSV table becomes one by a transreption that changes the form and not the data: one record per row, keyed by the header. That is a conversion between two representations of the same thing, which is exactly what ikigai’s transreptors are for. The one needed takes a CSV table and gives a JSON array of objects keyed by its header row.

ikigai-json has exactly that: urn:json:from-csv, registered as the kernel’s text/csv → application/json transreptor, which reads the table with the csv crate (RFC 4180, so a quoted field keeps its commas). You rarely name it. urn:json:query runs a jq program over a JSON resource, and given a resource that is not JSON it plans the kernel’s transreption to JSON first, so the identity program . over the CSV atom is the table as records:

[{"name":"Mercury","kind":"terrestrial","year":"88"},{"name":"Venus","kind":"terrestrial","year":"225"},{"name":"Earth","kind":"terrestrial","year":"365"},{"name":"Mars","kind":"terrestrial","year":"687"},{"name":"Jupiter","kind":"gas giant","year":"4333"},{"name":"Saturn","kind":"gas giant","year":"10759"},{"name":"Uranus","kind":"ice giant","year":"30687"},{"name":"Neptune","kind":"ice giant","year":"60190"}]
[computed]

Every value is a string, because CSV has no types: the years are "88", where the JSON document had 88. The row template splices each as text either way, so the rows come out the same.

The kernel does not turn one representation into another just because a reader asks for it with as= (ledger #2153), which is why the conversion is spelled as a query rather than as source …:planets.csv as=application/json.

Two sources, side by side

Each source is one template, and the two are the only place the view’s sources differ. The CSV source’s records are that query over the table:

$r{urn:json:query?src=urn:iki:tutorial:inspect:data:planets.csv&q=.}
[computed]

and the JSON source’s are the document itself:

$r{urn:iki:tutorial:inspect:data:planets.json}
[computed]

The view over the CSV table is the same template:planets, filled with source=csv:

<section id="view" class="inspect">
<h1>Planets</h1>
<p>One view, read from <code>urn:iki:tutorial:inspect:records:csv</code>. The view does not know which source it was given: the same two templates show the JSON document and the CSV table.</p>
<table class="inspect-table">
<caption>The eight planets, from the csv source</caption>
<thead><tr><th scope="col">Planet</th><th scope="col">Kind</th><th scope="col">Orbit (Earth days)</th></tr></thead>
<tbody>
<tr><th scope="row">Mercury</th><td>terrestrial</td><td>88</td></tr>
<tr><th scope="row">Venus</th><td>terrestrial</td><td>225</td></tr>
<tr><th scope="row">Earth</th><td>terrestrial</td><td>365</td></tr>
<tr><th scope="row">Mars</th><td>terrestrial</td><td>687</td></tr>
<tr><th scope="row">Jupiter</th><td>gas giant</td><td>4333</td></tr>
<tr><th scope="row">Saturn</th><td>gas giant</td><td>10759</td></tr>
<tr><th scope="row">Uranus</th><td>ice giant</td><td>30687</td></tr>
<tr><th scope="row">Neptune</th><td>ice giant</td><td>60190</td></tr>
</tbody>
</table>
</section>
[computed]

and the view over the JSON document, with source=json:

<section id="view" class="inspect">
<h1>Planets</h1>
<p>One view, read from <code>urn:iki:tutorial:inspect:records:json</code>. The view does not know which source it was given: the same two templates show the JSON document and the CSV table.</p>
<table class="inspect-table">
<caption>The eight planets, from the json source</caption>
<thead><tr><th scope="col">Planet</th><th scope="col">Kind</th><th scope="col">Orbit (Earth days)</th></tr></thead>
<tbody>
<tr><th scope="row">Mercury</th><td>terrestrial</td><td>88</td></tr>
<tr><th scope="row">Venus</th><td>terrestrial</td><td>225</td></tr>
<tr><th scope="row">Earth</th><td>terrestrial</td><td>365</td></tr>
<tr><th scope="row">Mars</th><td>terrestrial</td><td>687</td></tr>
<tr><th scope="row">Jupiter</th><td>gas giant</td><td>4333</td></tr>
<tr><th scope="row">Saturn</th><td>gas giant</td><td>10759</td></tr>
<tr><th scope="row">Uranus</th><td>ice giant</td><td>30687</td></tr>
<tr><th scope="row">Neptune</th><td>ice giant</td><td>60190</td></tr>
</tbody>
</table>
</section>
[computed]

The two tables are the same, row for row. Only the sentence above each one names its source, because the template prints its source argument there. The inspector’s navigation has both pages, Planets (JSON) and Planets (CSV), so a browser shows the same thing.

Two sources, two threads

The views share their templates, but each hangs from its own source’s golden thread. Write the JSON document, as the last chapter did:

written
[uncacheable]

The CSV view was read from nothing that write touched, so it is still cached:

<section id="view" class="inspect">
<h1>Planets</h1>
<p>One view, read from <code>urn:iki:tutorial:inspect:records:csv</code>. The view does not know which source it was given: the same two templates show the JSON document and the CSV table.</p>
<table class="inspect-table">
<caption>The eight planets, from the csv source</caption>
<thead><tr><th scope="col">Planet</th><th scope="col">Kind</th><th scope="col">Orbit (Earth days)</th></tr></thead>
<tbody>
<tr><th scope="row">Mercury</th><td>terrestrial</td><td>88</td></tr>
<tr><th scope="row">Venus</th><td>terrestrial</td><td>225</td></tr>
<tr><th scope="row">Earth</th><td>terrestrial</td><td>365</td></tr>
<tr><th scope="row">Mars</th><td>terrestrial</td><td>687</td></tr>
<tr><th scope="row">Jupiter</th><td>gas giant</td><td>4333</td></tr>
<tr><th scope="row">Saturn</th><td>gas giant</td><td>10759</td></tr>
<tr><th scope="row">Uranus</th><td>ice giant</td><td>30687</td></tr>
<tr><th scope="row">Neptune</th><td>ice giant</td><td>60190</td></tr>
</tbody>
</table>
</section>
[cached]

and the JSON view is recomputed from the new document:

<section id="view" class="inspect">
<h1>Planets</h1>
<p>One view, read from <code>urn:iki:tutorial:inspect:records:json</code>. The view does not know which source it was given: the same two templates show the JSON document and the CSV table.</p>
<table class="inspect-table">
<caption>The eight planets, from the json source</caption>
<thead><tr><th scope="col">Planet</th><th scope="col">Kind</th><th scope="col">Orbit (Earth days)</th></tr></thead>
<tbody>
<tr><th scope="row">Ceres</th><td>dwarf planet</td><td>1682</td></tr>
</tbody>
</table>
</section>
[computed]

A write to the CSV table works the same way the other way round; the inspector’s tests write one with a quoted comma and a field that holds markup, and check that the comma stays in its field, the markup arrives as text and the JSON view does not move.

No stopgaps left

While these two chapters were being written, the iteration and the CSV conversion were each a labeled stopgap in the inspector, shaped like the resources this chapter needed and named stopgap: so nobody mistook them for the design. Both gaps were filled where they belong, with the same arguments: urn:iki:fn:each in ikigai-fn 0.6.0 and urn:json:query with its CSV transreptor in ikigai-json 0.1.0. Replacing them changed two template lines and no output. The only code the inspector has for these views is its two atoms.

Other forms of data

JSON and CSV are two forms of many. The same pattern covers the others: find the model the view needs, and the smallest step from the form the data has to that model.

  • XML needs no step at all: an XSLT stylesheet reads it directly, as in the XSLT chapter, with no RDF anywhere.
  • SPARQL results already have a JSON form (application/sparql-results+json); with urn:json:query to pick out the bindings, a query’s answer is an array a row template can be filled from.
  • Plain lines of text, a log or a listing, have ikigai-text (grep, sort, head and the rest, as resources), which a view can splice as they are.

What was built, and what it rules out

One view and one row template over two sources, a CSV table and a JSON document, each source one template naming where its records come from, and each with its own golden thread. Ruled out: a second view for a second format, and a model the data has to be converted into before it can be shown.

Next: Live data and history.