www.openlinksw.com
docs.openlinksw.com

Book Home

Contents
Preface

Virtuoso Functions Guide

Administration
Aggregate Functions
Array Manipulation
BPEL APIs
Backup
Compression
Cursor
Date & Time Manipulation
Debug
Dictionary Manipulation
Encoding & Decoding
File Manipulation
Free Text
Hashing / Cryptographic
LDAP
Locale
Mail
Miscellaneous
Number
Phrases
RDF data
db.dba.rdf_64bit_upg...
db.dba.rdf_audit_met...
db.dba.rdf_backup_me...
db.dba.rdf_convert_r...
db.dba.rdf_datatype_...
db.dba.rdf_graph_col...
db.dba.rdf_graph_dif...
db.dba.rdf_graph_gro...
db.dba.rdf_graph_gro...
db.dba.rdf_langmatch...
db.dba.rdf_load_rdfa
db.dba.rdf_load_rdfx...
db.dba.rdf_load_rdfx...
db.dba.rdf_quad_uri
db.dba.rdf_quad_uri_...
db.dba.rdf_quad_uri_...
db.dba.rdf_regex
db.dba.rdf_restore_m...
db.dba.rdf_suo_apply...
db.dba.rdf_suo_diff_...
db.dba.rdf_triples_t...
db.dba.rdf_triples_t...
db.dba.rdf_ttl2hash
db.dba.rdf_void_stor...
db.dba.sparql_eval
db.dba.sparql_eval_t...
db.dba.sparql_rdb2rd...
db.dba.sparql_rdb2rd...
db.dba.sparql_rexec
db.dba.sparql_rexec_...
db.dba.sparql_rexec_...
db.dba.sparql_select...
db.dba.ttlp
db.dba.ttlp_mt
db.dba.ttlp_mt_local...
group_concat
group_digest
isref
rdf_view_sync_to_phy...
sample
__xml_get_ns_prefix
__xml_get_ns_uri
__xml_ns_iristr
__xml_ns_uname
__xml_nsexpand_irist...
http_nt_triple
http_ttl_triple
iri_split
ld_dir
rdfs_rule_set
sparql_to_sql_text
Remote SQL Data Source
Replication
SOAP
SQL
String
Transaction
Type Mapping
UDDI
User Defined Types & The CLR
VAD
Virtuoso Java PL API
Virtuoso Server Extension Interface (VSEI)
Web & Internet
XML
XPATH & XQUERY

Functions Index

DB.DBA.TTLP_MT

parses TTL (TURTLE), TTL(N-Triple) or NQ (N-quad) resources, and places its triples into DB.DBA.RDF_QUAD on multiple threads.
DB.DBA.TTLP_MT (in strg any, in base varchar, [in graph varchar], [in flags integer], [in log_mode integer], [in threads integer]);
Description

Loads the TTL (TURTLE or N3 resource) file on multiple threads, using parallel I/O and multiprocessing if available. The function commit partial transactions while it runs so the transaction log may contain part of loading. Moreover, the function may or may not leave a transaction log, depending on log_mode. Hence, after successful load, one may need to execute the checkpoint statement to make sure that a server restart does not wipe out the results.

Parameters
strg – text of the resource
base – base IRI to resolve relative IRIs to absolute
graph – target graph IRI, parsed triples will appear in that graph.
flags – bitmask of parsing flags. Permits some sorts of syntax errors in resource. Default is 0, meaning no permitted deviations from the spec. Other supported bits are:
  1 - Single quoted and double quoted strings may with newlines.
  2 - Allows bnode predicates (but SPARQL processor may ignore them!).
  4 - Allows variables, but triples with variables are ignored.
  8 - Allows literal subjects, but triples with them are ignored.
 16 - Allows '/', '#', '%' and '+' in local part of QName ("Qname with path")
 32 - Allows invalid symbols between '<' and '>', i.e. in relative IRIs.
 64 - Relax TURTLE syntax to include popular violations.
128 - Try to recover from lexical errors as much as it is possible.
256 - Allows TriG syntax, thus loading data in more than one graph.
512 - Allows loading N-quad dataset files with and optional context value to indicate provenance as detailed
<ulink url="http://sw.deri.org/2008/07/n-quads/">here</ulink>.
log_mode – detail level of writing the effect of loading to the transaction log. 0 means log nothing, 1 means log only allocations of internal IDs for new IRIs and literals, 2 means log everything. If database crashes when the loading is in progress or after the loading but before checkpoint is made, 0 will means that the database become inconsistent, 1 means that the database is consistent but loaded quads may disappear so the loading should be repeated and log replay may produce wrong results if actions in it depend on the content of quad store, 2 means no danger (so the default is 2). Hence loading with mode 1 and especially mode 0 are faster than usual mode 2 but they require checkpoints after data loadings and mode 0 additionally requires a checkpoint and database backup right before the loading.
threads – number of threads that insert quads into the database. It should not be less than 1, obviously; it is better to not set it greater than ((N-2)/k)-1 where N is the number of available CPU cores and k is the number of loadings that happen at the same time.
Return Types

The return value is not specified and may be changed in future versions.

Examples

Example 1

Load ttl from local file:

SQL> DB.DBA.TTLP_MT (file_to_string_output ('tmp/users.ttl'), '', 'http://mytest.com');

Done. -- 381 msec.
SQL>sparql
select *
from <http://mytest.com>
where {?s ?p ?o}
limit 10;
s             p                   o
VARCHAR       VARCHAR             VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/virtrdf-data-formats#SysUsers-GranteeId-format          http://www.w3.org/1999/02/22-rdf-syntax-ns#type            http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-GranteeId-format          http://www.openlinksw.com/schemas/virtrdf#qmfSuperFormats  nodeID://1000272018
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-GranteeId-format          http://www.openlinksw.com/schemas/virtrdf#inheritFrom      http://www.openlinksw.com/virtrdf-data-formats#sql-integer-uri-fn
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-GranteeId-format          http://www.openlinksw.com/schemas/virtrdf#noInherit        http://www.openlinksw.com/schemas/virtrdf#qmfCustomString1
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-GranteeId-format          http://www.openlinksw.com/schemas/virtrdf#qmfCustomString1 DB.DBA.RDF_DF_GRANTEE_ID_URI
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-UserId-format             http://www.w3.org/1999/02/22-rdf-syntax-ns#type            http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-UserId-format             http://www.openlinksw.com/schemas/virtrdf#qmfSuperFormats  nodeID://1000272019
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-UserId-format             http://www.openlinksw.com/schemas/virtrdf#inheritFrom      http://www.openlinksw.com/virtrdf-data-formats#sql-integer-uri
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-UserId-format             http://www.openlinksw.com/schemas/virtrdf#noInherit        http://www.openlinksw.com/schemas/virtrdf#qmfCustomString1
http://www.openlinksw.com/virtrdf-data-formats#SysUsers-UserId-format             http://www.openlinksw.com/schemas/virtrdf#qmfCustomString1 http://localhost/sys/user?id=%d

10 Rows. -- 30 msec.

Example 2

Usage of the ttlp_mt function in a procedure:

SQL>create procedure SPARQL_DAWG_LOAD_REMOTE_DATFILE (in full_uri varchar, in in_resultset integer := 0)
{
  declare REPORT varchar;
  declare graph_uri, dattext varchar;
  declare app_env any;
  app_env := null;
  whenever sqlstate '*' goto err_rep;
  if (not in_resultset)
    result_names (REPORT);
  dattext := cast (XML_URI_GET_AND_CACHE (full_uri) as varchar);
  SPARQL_REPORT (sprintf ('Downloading %s: %d bytes',
      full_uri, length (dattext) ) );
  graph_uri := full_uri;
  delete from RDF_QUAD where G = DB.DBA.RDF_MAKE_IID_OF_QNAME (graph_uri);
  if ((full_uri like '%.ttl') or (full_uri like '%.nt') or (full_uri like '%.n3'))
    DB.DBA.TTLP_MT (dattext, full_uri, graph_uri);
  else -- if (rel_path like '%.rdf')
    DB.DBA.RDF_LOAD_RDFXML_MT (dattext, full_uri, graph_uri);
  return graph_uri;
err_rep:
  result (sprintf ('%s: %s', __SQL_STATE, __SQL_MESSAGE));
  return graph_uri;
}
;
Done. -- 891 msec
SQL> select SPARQL_DAWG_LOAD_REMOTE_DATFILE('http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl');
REPORT
VARCHAR
_______________________________________________________________________________

Downloading http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl: 12768 bytes
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl

2 Rows. -- 1382 msec.

Example 3

Fourth item in record of NQuad format is optional. If present then it is used as a graph. If missed then the default graph is used. The purpose is to make SPARQL dataset serialization possible in a "natural" way: the content of default graph is written without any graph name specified whereas the content of named graphs is written with fourth field in place. Similarly, TriG uses default graph for triples outside graph blocks. In all these cases, base IRI is used to resolve all relative IRIs of the document, no matter what is the destination graph. Suppose the following scenario:

There is a simple nquad.nq file:

<http://www.w3.org/2002/01/tr-automation/tr.rdf> <http://purl.org/dc/elements/1.1/title> "W3C Standards and Technical Reports" <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/PersonalProfileDocument> <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card> <http://creativecommons.org/ns#license> <http://creativecommons.org/licenses/by-nc/3.0/> <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card> <http://purl.org/dc/elements/1.1/title> "Tim Berners-Lee's FOAF file" <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card> <http://xmlns.com/foaf/0.1/maker> <http://www.w3.org/People/Berners-Lee/card#i> <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card> <http://xmlns.com/foaf/0.1/primaryTopic> <http://www.w3.org/People/Berners-Lee/card#i> <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card#cm> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card#cm> <http://www.w3.org/2000/01/rdf-schema#seeAlso> <http://www.koalie.net/foaf.rdf> <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card#cm> <http://xmlns.com/foaf/0.1/mbox> <mailto:coralie@w3.org> <http://www.w3.org/People/Berners-Lee/card> .
<http://www.w3.org/People/Berners-Lee/card#cm> <http://xmlns.com/foaf/0.1/name> "Coralie Mercier" <http://www.w3.org/People/Berners-Lee/card> .

Load the file:

SQL>DB.DBA.TTLP_MT (file_to_string_output ('./nquad.nq'), '', 'http://localhost:8890/DAV/test', 512);
Done. -- 48 msec.

Select all triples from the graph with URI the fourth provenance URI from the NQuad file i.e.: <http://www.w3.org/People/Berners-Lee/card>:

 SQL> sparql select * from <http://www.w3.org/People/Berners-Lee/card> where {?s ?p ?o};
 s                                               p                                                  o
 VARCHAR                                         VARCHAR                                            VARCHAR
 ________________________________________________

 http://www.w3.org/2002/01/tr-automation/tr.rdf  http://purl.org/dc/elements/1.1/title              W3C Standards and Technical Reports
 http://www.w3.org/People/Berners-Lee/card       http://www.w3.org/1999/02/22-rdf-syntax-ns#type    http://xmlns.com/foaf/0.1/PersonalProfileDocument
 http://www.w3.org/People/Berners-Lee/card       http://xmlns.com/foaf/0.1/primaryTopic             http://www.w3.org/People/Berners-Lee/card#i
 http://www.w3.org/People/Berners-Lee/card       http://purl.org/dc/elements/1.1/title              Tim Berners-Lee's FOAF file
 http://www.w3.org/People/Berners-Lee/card       http://xmlns.com/foaf/0.1/maker                    http://www.w3.org/People/Berners-Lee/card#i
 http://www.w3.org/People/Berners-Lee/card       http://creativecommons.org/ns#license              http://creativecommons.org/licenses/by-nc/3.0/
 http://www.w3.org/People/Berners-Lee/card#cm    http://www.w3.org/1999/02/22-rdf-syntax-ns#type    http://xmlns.com/foaf/0.1/Person
 http://www.w3.org/People/Berners-Lee/card#cm    http://xmlns.com/foaf/0.1/mbox                     mailto:coralie@w3.org
 http://www.w3.org/People/Berners-Lee/card#cm    http://xmlns.com/foaf/0.1/name                     Coralie Mercier
 http://www.w3.org/People/Berners-Lee/card#cm    http://www.w3.org/2000/01/rdf-schema#seeAlso       http://www.koalie.net/foaf.rdf

 10 Rows. -- 7 msec.