W3C test results

Per-suite pass/fail counts against the W3C SPARQL 1.1 and RDF 1.1 conformance suites.

Pass
1657
Fail
1
Skipped
4
Pass / runnable
99.9% of 1658
Total in suites
1662
Scope. These counts measure conformance only — whether the engine produces the expected result on each W3C-supplied fixture. They say nothing about performance, memory use, scale, or real-world query shape. Skipped tests are features not yet implemented (mostly HTTP-protocol / federated-query endpoints). Raw per-suite logs and machine-readable artifacts are linked below.

SPARQL 1.1 626 pass · 1 fail · 4 skip

add
8/0
aggregates
47/0
basic-update
13/0
bind
10/0
bindings
11/0
cast
6/0
clear
4/0
construct
7/0
copy
6/0
csv-tsv-res
6/0
delete
19/0
delete-data
6/0
delete-insert
17/0
delete-where
6/0
drop
4/0
entailment (SPARQL 1.1 regime — RDFS / D-entailment, 70 tests)
65/1 skip 4
exists
6/0
functions
75/0
grouping
6/0
http-rdf-update
19/0
json-res
4/0
move
6/0
negation
12/0
project-expression
7/0
property-path
33/0
protocol
34/0
service
7/0
service-description
3/0
subquery
14/0
syntax-fed
3/0
syntax-query
94/0
syntax-update-1
54/0
syntax-update-2
1/0
update-silent
13/0

RDF 1.1 1031 pass · 0 fail

rdf-mt
39/0
rdf-n-quads
87/0
rdf-n-triples
70/0
rdf-trig
356/0
rdf-turtle
313/0
rdf-xml
166/0
rdf-canon
59/26 skip 1

OWL 2 13 pass · 17 fail · out of ~3378 W3C OWL 2 tests in scope (≈0.4%)

Scope clarification. The SPARQL 1.1 entailment row in the SPARQL table above (65/70 pass) is the SPARQL 1.1 entailment-regime test suite — a narrow, well-defined slice of W3C testing that covers RDF / RDFS / D-entailment regimes attached to BGP matching. It is not a measure of OWL conformance. The full W3C OWL 2 Test Cases (~3378 tests across 9 categories below) are the larger, ultimate goal — currently only the profile-RL PositiveEntailmentTests slot (30 tests) is wired into the runner. The headline percentage above is against the full OWL 2 universe, not the profile-RL subset, so it reflects honest end-to-end coverage.

profile-RL PosEnt
13/17 of 30
profile-EL
— 174 tests · blocked on EL-profile closure rules
profile-QL
— 130 tests · blocked on QL-profile query rewrite
semantics-direct
— 976 tests · blocked on DL tableau (stage d/e)
syntax-dl
— 646 tests · blocked on DL syntactic-profile checker
type-positive-entailment
— 412 tests · blocked on closure-side bnode isomorphism
type-negative-entailment
— 46 tests · blocked on OWL negation support
type-consistency
— 708 tests · blocked on contradiction detection
type-inconsistency
— 256 tests · blocked on contradiction detection

OWL 2 (W3C conformance): We vendor the full W3C OWL 2 Test Cases at third_party/testing/owl/ (10 catalog files, ~2500 test:TestCase entries after overlap). Only profile-RL PositiveEntailmentTests are wired right now: the runner applies owl_rl_closure_with_reflexivity (fuel 100) and checks the conclusion’s triples against the closure (relaxed bnode match). The other OWL 2 categories are skipped for now—blocked on completing the RDF/S and SPARQL entailment baselines above. Counts are live from the vendored catalogs, not stale snapshots.

RDFC-1.0 59 pass · 26 fail · 1 stub · of 86 total

eval (HFDQ)
59/26 of 85
map / negative-eval
1 stub — runner does not emit map JSON / HNDQ termination guard

RDFC-1.0 (W3C RDF Dataset Canonicalization): We vendor the full W3C RDFC-1.0 Test Cases at third_party/testing/rdf-canon/. The canonical-labelling algorithm lives in F* at formal/fstar/RDF.Canonical.fst — Hash First Degree Quads (HFDQ) phase verifies cleanly with no --lax and no --admit_smt_queries. Eval tests check the canonical N-Quads form bytewise. Phase 2 work (HNDQ — Hash N-Degree Quads with bounded permutation enumeration) closes the remaining symmetric-cycle eval tests. Map tests are STUB pending a runner-side bnode → canonical-id JSON emitter; NegEval tests need an HNDQ termination guard.

Machine-readable artifacts

The raw runner logs (including per-test FAIL lines with diffs) are committed to formal/fstar/ocaml-output/sparql_results.log and formal/fstar/ocaml-output/rdf_results.log.

Raw per-suite numbers

SPARQL 1.1

631 total: 626 pass, 1 fail, 4 skip, 0 unsupported
  add                                 pass:8 fail:0 skip:0 unsupported:0
  aggregates                          pass:47 fail:0 skip:0 unsupported:0
  basic-update                        pass:13 fail:0 skip:0 unsupported:0
  bind                                pass:10 fail:0 skip:0 unsupported:0
  bindings                            pass:11 fail:0 skip:0 unsupported:0
  cast                                pass:6 fail:0 skip:0 unsupported:0
  clear                               pass:4 fail:0 skip:0 unsupported:0
  construct                           pass:7 fail:0 skip:0 unsupported:0
  copy                                pass:6 fail:0 skip:0 unsupported:0
  csv-tsv-res                         pass:6 fail:0 skip:0 unsupported:0
  delete                              pass:19 fail:0 skip:0 unsupported:0
  delete-data                         pass:6 fail:0 skip:0 unsupported:0
  delete-insert                       pass:17 fail:0 skip:0 unsupported:0
  delete-where                        pass:6 fail:0 skip:0 unsupported:0
  drop                                pass:4 fail:0 skip:0 unsupported:0
  entailment                          pass:65 fail:1 skip:4 unsupported:0
  exists                              pass:6 fail:0 skip:0 unsupported:0
  functions                           pass:75 fail:0 skip:0 unsupported:0
  grouping                            pass:6 fail:0 skip:0 unsupported:0
  http-rdf-update                     pass:19 fail:0 skip:0 unsupported:0
  json-res                            pass:4 fail:0 skip:0 unsupported:0
  move                                pass:6 fail:0 skip:0 unsupported:0
  negation                            pass:12 fail:0 skip:0 unsupported:0
  project-expression                  pass:7 fail:0 skip:0 unsupported:0
  property-path                       pass:33 fail:0 skip:0 unsupported:0
  protocol                            pass:34 fail:0 skip:0 unsupported:0
  service                             pass:7 fail:0 skip:0 unsupported:0
  service-description                 pass:3 fail:0 skip:0 unsupported:0
  subquery                            pass:14 fail:0 skip:0 unsupported:0
  syntax-fed                          pass:3 fail:0 skip:0 unsupported:0
  syntax-query                        pass:94 fail:0 skip:0 unsupported:0
  syntax-update-1                     pass:54 fail:0 skip:0 unsupported:0
  syntax-update-2                     pass:1 fail:0 skip:0 unsupported:0
  update-silent                       pass:13 fail:0 skip:0 unsupported:0

RDF 1.1

1031 total: 1031 pass, 0 fail, 0 skip, 0 unsupported
  rdf-mt                              pass:39 fail:0 skip:0 unsupported:0
  rdf-n-quads                         pass:87 fail:0 skip:0 unsupported:0
  rdf-n-triples                       pass:70 fail:0 skip:0 unsupported:0
  rdf-trig                            pass:356 fail:0 skip:0 unsupported:0
  rdf-turtle                          pass:313 fail:0 skip:0 unsupported:0
  rdf-xml                             pass:166 fail:0 skip:0 unsupported:0
How this page is generated

Source: formal/fstar/generate-report.sh. It shells out to the w3c_runner binary (extracted from F* specs, compiled via OCaml), scrapes per-suite counts, and writes index.html, latest.csv, and latest.json. Run ./generate-report.sh --run in formal/fstar/ to regenerate. CI re-runs on every push and nightly at 06:00 UTC.