com.bigdata.rdf.sail

Class BigdataSail

java.lang.Object

com.bigdata.rdf.sail.SailBase

com.bigdata.rdf.sail.BigdataSail

All Implemented Interfaces:

org.openrdf.sail.Sail

public class BigdataSail
extends SailBase
implements org.openrdf.sail.Sail

Sesame 2.x integration.

Read-write operations use getConnection()to obtain a mutable view. This delegates to either getUnisolatedConnection() or getReadWriteConnection() depending on how the Sail is configured.

Concurrent unisolated writers are supported if group commit is enabled, but at most one unisolated writer will execute against a given Sail instance at a time - other writers will be serialized. See getUnisolatedConnection() for more information on how to enable group commit and how to write code that supports group commit.

Concurrent readers are possible, and can be very efficient. However, readers MUST use a database commit point corresponding to a desired state of the store, e.g., after loading some data set and (optionally) after computing the closure of that data set. Use getReadOnlyConnection() to obtain a read-only view of the database as of the last commit time, or getReadOnlyConnection(long) to obtain a read-only view of the database as of some other historical point. These connections are safe to use concurrently with the unisolated connection from getConnection().

Concurrent fully isolated read/write transactions against the same Sail are also implemented in the bigdata SAIL. To turn on read/write transactions, use the option BigdataSail.Options.ISOLATABLE_INDICES. If this option is set to true, then getConnection() will return an isolated read/write view of the database. Multiple concurrent read/write transactions are allowed, and the database can resolve add/add conflicts between transactions. Unlike group commit, these fully isolated read/write transactions will in fact execute concurrently against the same Sail instance (rather than being serialized). However, there is additional overhead for fully isolated read/write transactions and graph update patterns can lead to unreconcilable conflicts during updates (in which case one transaction will be a failed during the validation phase). The highest throughput is generally obtained using getUnisolatedConnection() in which case you do not need to enabled BigdataSail.Options.ISOLATABLE_INDICES.

The BigdataSail may be configured as as to provide a triple store with statement-level provenance using statement identifiers. See AbstractTripleStore.Options#STATEMENT_IDENTIFIERS and Reification Done Right .

Quads may be enabled using AbstractTripleStore.Options#QUADS. However, note that BigdataSail.Options.TRUTH_MAINTENANCE is not supported for AbstractTripleStore.Options#QUADS at this time. This may change in the future once we decide how to handle eager materialization of entailments with multiple named graphs. The basic problem is that:

 merge(closure(graphA), closure(graphB))
 

IS NOT EQUALS TO

 closure(merge(graphA, graphB))
 

There are two ways to handle this. One is to compute all inferences at query time, in which case we are not doing eager materialization and therefore we are not using Truth Maintenance. The other is to punt and use the merge of their individual closures.

Author:

Bryan Thompson

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	BigdataSail.BigdataSailConnection Inner class implements the SailConnection.
static interface	BigdataSail.Options Additional parameters understood by the Sesame 2.x SAIL implementation.
static class	BigdataSail.UnisolatedCallable<T> Abstract base class permits some patterns in which the Callable owns the locks required to obtain the unisolated connection.

Field Summary

Fields

Modifier and Type	Field and Description
static org.openrdf.model.URI	NULL_GRAPH Sesame has the notion of a "null" graph which we use for the quad store mode.

Fields inherited from class com.bigdata.rdf.sail.SailBase

CONNECTION_TIMEOUT, connectionTimeOut, DEFAULT_CONNECTION_TIMEOUT, initializationLock, logger

Constructor Summary

Constructors

Constructor and Description
BigdataSail() Create or re-open a database instance configured using defaults.
BigdataSail(AbstractTripleStore database) Constructor used to wrap an existing AbstractTripleStore instance.
BigdataSail(AbstractTripleStore database, IIndexManager mainIndexManager) Core ctor.
BigdataSail(Properties properties) Create or open a database instance configured using the specified properties.
BigdataSail(String namespace, IIndexManager indexManager) Constructor used when the namespace and index manager are known (standard use case).
BigdataSail(String namespace, IIndexManager indexManager, IIndexManager mainIndexManager) Core constructor.

Method Summary

Methods

Modifier and Type	Method and Description
void	__tearDownUnitTest() DO NOT INVOKE FROM APPLICATION CODE - this method deletes the KB instance and destroys the backing database instance.
protected void	assertOpenSail()
static void	checkProperties(Properties properties) Check properties to make sure that they are consistent.
void	create(Properties properties) Create a the configured namespace.
void	destroy() Destroy the configured namespace.
boolean	exists() Return true iff the namespace exists at the moment when this check is performed.
protected void	finalize() Invokes shutDown().
BigdataSail.BigdataSailConnection	getConnection()
protected org.openrdf.sail.NotifyingSailConnection	getConnectionInternal() Return a read-write SailConnection.
IIndexManager	getIndexManager() The index manager associated with the database (when there is a focus store and a main database, then this is the focus store index manager).
String	getNamespace() Return the blazegraph namespace associated with the BigdataSail.
QueryEngine	getQueryEngine()
BigdataSail.BigdataSailConnection	getReadOnlyConnection() Return a read-only connection based on the last commit point.
BigdataSail.BigdataSailConnection	getReadOnlyConnection(long timestamp) Obtain a read-historical view that reads from the specified commit point.
BigdataSail.BigdataSailConnection	getReadWriteConnection() Return a connection backed by a read-write transaction.
protected ITransactionService	getTxService() Return the ITransactionService.
BigdataSail.BigdataSailConnection	getUnisolatedConnection() Return an unisolated connection to the database.
<T> T	getUnisolatedConnectionLocksAndRunLambda(BigdataSail.UnisolatedCallable<T> lambda) Obtain the locks required by the unisolated connection and invoke the caller's lambda.
org.openrdf.model.ValueFactory	getValueFactory() A BigdataValueFactory
protected void	initializeInternal() Do store-specific operations to initialize the store.
boolean	isOpen() Return true if the BigdataSail has been SailBase.initialize()d and has not been shutDown().
boolean	isWritable() Return false iff the Sail can not provide a writable connection.
void	shutDown()
protected void	shutDownInternal() If the backing database was created/opened by the BigdataSail then it is closed.

Methods inherited from class com.bigdata.rdf.sail.SailBase

connectionClosed, debugEnabled, getDataDir, initialize, isInitialized, manageConnection, setDataDir, toString

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.openrdf.sail.Sail

getDataDir, initialize, setDataDir

Field Detail

NULL_GRAPH

public static final transient org.openrdf.model.URI NULL_GRAPH

Sesame has the notion of a "null" graph which we use for the quad store mode. Any time you insert a statement into a quad store and the context position is not specified, it is actually inserted into this "null" graph. If SPARQL DATASET is not specified, then all contexts are queried and you will see statements from the "null" graph as well as from any other context. BigdataSail.BigdataSailConnection.getStatements(Resource, URI, Value, boolean, Resource...) will return statements from the "null" graph if the context is either unbound or is an array whose sole element is null.

See Also:

BigdataSail.BigdataSailConnection.addStatement(Resource, URI, Value, Resource...), BigdataSail.BigdataSailConnection.getStatements(Resource, URI, Value, boolean, Resource...)

Constructor Detail

BigdataSail

public BigdataSail()

Create or re-open a database instance configured using defaults.

BigdataSail

public BigdataSail(Properties properties)

Create or open a database instance configured using the specified properties.

See Also:

BigdataSail.Options

BigdataSail

public BigdataSail(AbstractTripleStore database)

Constructor used to wrap an existing AbstractTripleStore instance.

Note: Since BLZG-2041, this delegates through to the core constructor which accepts (namespace, IIndexManager).

Parameters:

database - The instance.

BigdataSail

public BigdataSail(AbstractTripleStore database,
           IIndexManager mainIndexManager)

Core ctor. You must use this variant for a scale-out triple store.

To create a BigdataSail backed by an IBigdataFederation use the ScaleOutTripleStore ctor and then AbstractTripleStore.create() the triple store if it does not exist.

Note: Since BLZG-2041, this delegates through to the core constructor which accepts (namespace, IIndexManager).

Parameters:

database - An existing AbstractTripleStore.

mainIndexManager - When database is a TempTripleStore, this is the IIndexManager used to resolve the QueryEngine. Otherwise it must be the same object as the database.

BigdataSail

public BigdataSail(String namespace,
           IIndexManager indexManager)

Constructor used when the namespace and index manager are known (standard use case).

Parameters:

namespace - The namespace.

indexManager - The index manager (typically a Journal).

BigdataSail

public BigdataSail(String namespace,
           IIndexManager indexManager,
           IIndexManager mainIndexManager)

Core constructor.

Note: Scale-out is shard-wise ACID. To create a BigdataSail backed by an IBigdataFederation use the ScaleOutTripleStore ctor and then AbstractTripleStore.create() the triple store if it does not exist. See CreateKBTask which encapsulates this.

Parameters:

namespace - The namespace.

indexManager - The index manager on which the data is stored.

mainIndexManager - Iff the data is stored on a TempTripleStore then this is the main index manager and will be used to locate the QueryEngine.

See Also:

BigdataSail should not locate the AbstractTripleStore until a connection is requested

Method Detail

getNamespace

public String getNamespace()

Return the blazegraph namespace associated with the BigdataSail.

See Also:

BigdataSail.Options.NAMESPACE, BigdataSail.getUnisolatedConnection() encapsulation

getIndexManager

public IIndexManager getIndexManager()

The index manager associated with the database (when there is a focus store and a main database, then this is the focus store index manager).

See Also:

BigdataSail.getUnisolatedConnection() encapsulation

checkProperties

public static void checkProperties(Properties properties)
                            throws UnsupportedOperationException

Check properties to make sure that they are consistent.

Parameters:

properties - The properties.

Throws:

UnsupportedOperationException

assertOpenSail

protected void assertOpenSail()

Throws:

IllegalStateException - if the BigdataSail has not been SailBase.initialize()d or has been shutDown().

isOpen

public boolean isOpen()

Return true if the BigdataSail has been SailBase.initialize()d and has not been shutDown().

create

public void create(Properties properties)
            throws InterruptedException,
                   ExecutionException

Create a the configured namespace.

Parameters:

properties - The configuration properties for the namespace.

Throws:

ExecutionException

InterruptedException

See Also:

BigdataSail.Options.NAMESPACE, (BigdataSail should not locate the AbstractTripleStore until a connection is requested)

destroy

public void destroy()
             throws org.openrdf.sail.SailException,
                    InterruptedException,
                    ExecutionException

Destroy the configured namespace.

Throws:

org.openrdf.sail.SailException

ExecutionException

InterruptedException

See Also:

BigdataSail.Options.NAMESPACE, (BigdataSail should not locate the AbstractTripleStore until a connection is requested)

exists

public boolean exists()

Return true iff the namespace exists at the moment when this check is performed.

Note: It is not possible to use this with any guarantees that the namespace still exists once control is returned to the caller unless the caller is holding an unisolated connection.

See Also:

(BigdataSail should not locate the AbstractTripleStore until a connection is requested)

initializeInternal

protected void initializeInternal()
                           throws org.openrdf.sail.SailException

Description copied from class: SailBase

Do store-specific operations to initialize the store. The default implementation of this method does nothing.

Overrides:

initializeInternal in class SailBase

Throws:

IllegalStateException - if the sail is already open.

org.openrdf.sail.SailException

finalize

protected void finalize()
                 throws Throwable

Invokes shutDown().

Overrides:

finalize in class Object

Throws:

Throwable

shutDown

public void shutDown()
              throws org.openrdf.sail.SailException

Specified by:

shutDown in interface org.openrdf.sail.Sail

Overrides:

shutDown in class SailBase

Throws:

org.openrdf.sail.SailException

shutDownInternal

protected void shutDownInternal()
                         throws org.openrdf.sail.SailException

If the backing database was created/opened by the BigdataSail then it is closed. Otherwise this is a NOP.

Specified by:

shutDownInternal in class SailBase

Throws:

org.openrdf.sail.SailException

__tearDownUnitTest

public void __tearDownUnitTest()

DO NOT INVOKE FROM APPLICATION CODE - this method deletes the KB instance and destroys the backing database instance. It is used to help tear down unit tests.

getValueFactory

public final org.openrdf.model.ValueFactory getValueFactory()

A BigdataValueFactory

Specified by:

getValueFactory in interface org.openrdf.sail.Sail

isWritable

public final boolean isWritable()
                         throws org.openrdf.sail.SailException

Return false iff the Sail can not provide a writable connection.

Specified by:

isWritable in interface org.openrdf.sail.Sail

Throws:

org.openrdf.sail.SailException

See Also:

(BigdataSail should not locate the AbstractTripleStore until a connection is requested)

getConnectionInternal

protected org.openrdf.sail.NotifyingSailConnection getConnectionInternal()
                                                                  throws org.openrdf.sail.SailException

Return a read-write SailConnection. This is used for both read-write transactions and the UNISOLATED connection.

Note: There is only one UNISOLATED connection and, when requested, this method will block until that connection is available.

Specified by:

getConnectionInternal in class SailBase

Returns:

A connection to the store.

Throws:

org.openrdf.sail.SailException

See Also:

for a non-blocking, read-only connection.

getConnection

public BigdataSail.BigdataSailConnection getConnection()
                                                throws org.openrdf.sail.SailException

If the triple store was provisioned to support full read/write transactions then this is delegated to getReadWriteConnection(). Otherwise, is delegated to getUnisolatedConnection() which returns the unisolated view of the database. Note that truth maintenance requires only one connection at a time and is therefore not compatible with full read/write transactions.

The correct pattern for obtaining an updatable connection, doing work with that connection, and committing or rolling back that update is as follows.

 
 BigdataSailConnection conn = null;
 boolean ok = false;
 try {
     conn = sail.getConnection();
     doWork(conn);
     conn.commit();
     ok = true;
 } finally {
     if (conn != null) {
         if (!ok) {
             conn.rollback();
         }
         conn.close();
     }
 }
 

This pattern can also be used with getUnisolatedConnection().

Specified by:

getConnection in interface org.openrdf.sail.Sail

Overrides:

getConnection in class SailBase

Throws:

org.openrdf.sail.SailException

getUnisolatedConnection

public BigdataSail.BigdataSailConnection getUnisolatedConnection()
                                                          throws InterruptedException

Return an unisolated connection to the database. The unisolated connection supports fast, scalable updates against the database. The unisolated connection is ACID when used with a local Journal and shard-wise ACID when used with an IBigdataFederation.

Group Commit Enabled

If group commit is enabled and you use the either the REST API or the AbstractApiTask.submitApiTask(IIndexManager, com.bigdata.rdf.task.IApiTask) pattern to submit tasks, then concurrent unisolated connections will be granted for different namespaces and multiple unisolated connections for the same namespace MAY be melded into the same commit group (whether they are or not depends on the timing of the tasks and whether they are being submitted by a single client thread or by a pool of clients).

Group Commit Disabled

In this mode the application decides when the database will go through a commit point. In order to guarantee that operations against the unisolated connection are ACID, only one unisolated connection is permitted at a time for a Journal. If there is an open unisolated connection against a local Journal, then the open connection must be closed before a new connection can be returned by this method.

This constraint that there can be only one unisolated connection is not enforced in scale-out since unisolated operations in scale-out are only shard-wise ACID.

The correct pattern for obtaining an updatable connection, doing work with that connection, and committing or rolling back that update is as follows.

 BigdataSailConnection conn = null;
 boolean ok = false;
 try {
    conn = sail.getUnisolatedConnection();
    doWork(conn);
    conn.commit();
    ok = true;
 } finally {
    if (conn != null) {
       if (!ok) {
          conn.rollback();
       }
       conn.close();
    }
 }
 

Returns:

The unisolated connection to the database

Throws:

InterruptedException

See Also:

getConnection(), Group Commit (wiki)

getUnisolatedConnectionLocksAndRunLambda

public <T> T getUnisolatedConnectionLocksAndRunLambda(BigdataSail.UnisolatedCallable<T> lambda)
                                           throws Exception

Obtain the locks required by the unisolated connection and invoke the caller's lambda.

Parameters:

lambda - The caller's lambda

Returns:

The value to which the lamba evaluates.

Throws:

Exception

getReadOnlyConnection

public BigdataSail.BigdataSailConnection getReadOnlyConnection()

Return a read-only connection based on the last commit point. This method is atomic with respect to the commit protocol.

Returns:

The view.

getReadOnlyConnection

public BigdataSail.BigdataSailConnection getReadOnlyConnection(long timestamp)

Obtain a read-historical view that reads from the specified commit point. This view is safe for concurrent readers and will not update if there are concurrent writes.

Parameters:

timestamp - The commit point.

Returns:

The view.

getReadWriteConnection

public BigdataSail.BigdataSailConnection getReadWriteConnection()
                                                         throws IOException,
                                                                InterruptedException

Return a connection backed by a read-write transaction.

Throws:

InterruptedException

DatasetNotFoundException

UnsupportedOperationException - unless BigdataSail.Options.ISOLATABLE_INDICES was specified when the backing triple store instance was provisioned.

IOException

getTxService

protected ITransactionService getTxService()

Return the ITransactionService.

getQueryEngine

public QueryEngine getQueryEngine()