Passage's Homepage
Smart Clients: Automate Partial Executions until Completion.
When a SPARQL query times out, Passage provides partial results along with a continuation query to retrieve the missing results. Therefore, an end-user must sends back the continuation query to the Passage server to get the missing results (possibly multiple times when continuation queries time out as well).
end-user passage-server
┃ ┃
┃ ── curl SPARQL query ─────────────────────────────> ┃
┃ ┣ 60s-timeout
┃ <─────────── partial results, continuation query ── ┃
┃ ┃
┃ ── curl continuation query ───────────────────────> ┃
┃ ┣ 60s-timeout
┃ <─────────────────────────────── partial results ── ┃
done ┛ ┃
To ease this tedious process, we rely on smart clients to implement the loop until completion:
end-user smart-client passage-server
┃ ┃ ┃
┃ ───────── ┃ ── SPARQL query ──────────────────────> ┃
┃ ┃ ┣ 60s-timeout
┃ ┃ <─ partial results, continuation query ┃
┃ ┃ ┃
┃ ┃ ── continuation query ────────────────> ┃
┃ ┃ ┣ 60s-timeout
┃ <──────── ┃ ──────────────────── partial results ── ┃
done ┛ ┃
We also rely on smart clients to fill the gap between SPARQL 1.1 and the current state of supported operators by Passage. When an operator is not (yet) supported by Passage, the smart client handles the operator itself. Together, they provide complete and correct results for SPARQL.
 |
Comunica
For our first release, we took advantage of the smart client Comunica
v4.0.x [1]. Comunica is a SPARQL
query engine with an emphasis on modularity. To ensure completeness of
query results, it already includes actors such as quad pattern
fragments (qpf), or brqpf. We wanted to provide one for continuation
queries. Following the recommendation, we implemented our own actor
which mostly ressembles their SPARQL source actor.
In this version v4.0.x, you can state the range of supported
operators, and the engine will decompose the query for you: operators
handled by Passage, and operators handled by Comunica.
Quickstart
# Retrieve the sources git clone git@github.com:passage-org/passage-comunica.git cd passage-comunica # Similarly to Comunica yarn install yarn run query-passage \ http://localhost:3000/watdiv10M.jnl/passage \ -f path/to/query --logLevel=debug > results.dat
In this script, we ask the already running Passage endpoint on
localhost:3000 to provide the results to the query
path/to/query. We enable debug to print the continuation queries
on the standard error output stream, while the results are printed in
results.dat.
Contributing
Keep the smart client up-to-date with Comunica.
For instance, the current implementation sets the subset of operations handled by Passage. Instead, the server should advertise it.
Implement the same logic in other smart clients.
While Comunica provides a convenient browser experience, it lacks some multithreading capabilities. Other engines such as HeFQUIN could fill this gap.
We still want to push as many operators as possible to the server. It improves on query execution times, reduces generated traffic, etc.