com.bigdata.rdf.lexicon

Class Term2IdWriteProc

java.lang.Object

com.bigdata.btree.proc.AbstractIndexProcedure<T>

com.bigdata.btree.proc.AbstractKeyArrayIndexProcedure<Term2IdWriteProc.Result>

com.bigdata.rdf.lexicon.Term2IdWriteProc

All Implemented Interfaces:

IIndexProcedure<Term2IdWriteProc.Result>, IKeyArrayIndexProcedure<Term2IdWriteProc.Result>, IParallelizableIndexProcedure<Term2IdWriteProc.Result>, IReadOnly, IMutableRelationIndexWriteProcedure<Term2IdWriteProc.Result>, Externalizable, Serializable

public class Term2IdWriteProc
extends AbstractKeyArrayIndexProcedure<Term2IdWriteProc.Result>
implements IParallelizableIndexProcedure<Term2IdWriteProc.Result>, IMutableRelationIndexWriteProcedure<Term2IdWriteProc.Result>

This unisolated operation inserts terms into the term:id index, assigning identifiers to terms as a side-effect. The use of this operation MUST be followed by the the use of Id2TermWriteProc to ensure that the reverse mapping from id to term is defined before any statements are inserted using the assigned term identifiers. The client MUST NOT make assertions using the assigned term identifiers until the corresponding Id2TermWriteProc operation has succeeded.

In order for the lexicon to remain consistent if the client fails for any reason after the forward mapping has been made restart-safe and before the reverse mapping has been made restart-safe clients MUST always use a successful Term2IdWriteProc followed by a successful Id2TermWriteProc before inserting statements using term identifiers into the statement indices. In particular, a client MUST NOT treat lookup against the terms index as satisfactory evidence that the term also exists in the reverse mapping.

Note that it is perfectly possible that a concurrent client will overlap in the terms being inserted. The results will always be fully consistent if the rules of the road are observed since (a) unisolated operations are single-threaded; (b) term identifiers are assigned in an unisolated atomic operation by Term2IdWriteProc; and (c) the reverse mapping is made consistent with the assignments made/discovered by the forward mapping.

Note: The Term2IdWriteProc and Id2TermWriteProc operations may be analyzed as a batch variant of the following pseudo code.

  
  for each term:
  
  termId = null;
  
  synchronized (ndx) {
    
    counter = ndx.getCounter();
  
    termId = ndx.lookup(term.key);
    
    if(termId == null) {
 
       termId = counter.inc();
       
       ndx.insert(term.key,termId);
       
       }
  
  }
  
 

In addition, the actual operations against scale-out indices are performed on index partitions rather than on the whole index.

Author:

Bryan Thompson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Term2IdWriteProc.Result Object encapsulates the discovered / assigned term identifiers and provides efficient serialization for communication of those data to the client.
static class	Term2IdWriteProc.Term2IdWriteProcConstructor

Nested classes/interfaces inherited from class com.bigdata.btree.proc.AbstractKeyArrayIndexProcedure

AbstractKeyArrayIndexProcedure.ResultBitBuffer, AbstractKeyArrayIndexProcedure.ResultBitBufferCounter, AbstractKeyArrayIndexProcedure.ResultBitBufferHandler, AbstractKeyArrayIndexProcedure.ResultBuffer, AbstractKeyArrayIndexProcedure.ResultBufferHandler

Constructor Summary

Constructors

Modifier	Constructor and Description
	Term2IdWriteProc() De-serialization constructor.
protected	Term2IdWriteProc(IRabaCoder keySer, int fromIndex, int toIndex, byte[][] keys, boolean readOnly, boolean storeBlankNodes, int scaleOutTermIdBitsToReverse)

Method Summary

Methods

Modifier and Type	Method and Description
Term2IdWriteProc.Result	applyOnce(IIndex ndx, IRaba keys, IRaba vals) For each term whose serialized key is mapped to the current index partition, lookup the term in the terms index.
boolean	isReadOnly() Return true iff the procedure asserts that it will not write on the index.
boolean	isStoreBlankNodes()
protected IResultHandler<Term2IdWriteProc.Result,Term2IdWriteProc.Result>	newAggregator() Split-wise aggregation followed by combining the results across those splits in order to return an aggregated result whose iv[] is 1:1 with the original keys[][].
protected void	readMetadata(ObjectInput in) Reads metadata written by AbstractKeyArrayIndexProcedure.writeMetadata(ObjectOutput).
static VTE	VTE(byte code)
protected void	writeMetadata(ObjectOutput out) Writes metadata (not the keys or values, but just other metadata used by the procedure).

Methods inherited from class com.bigdata.btree.proc.AbstractKeyArrayIndexProcedure

apply, getKeys, getKeysCoder, getValues, getValuesCoder, readExternal, writeExternal

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.btree.proc.IIndexProcedure

apply

Constructor Detail

Term2IdWriteProc

public Term2IdWriteProc()

De-serialization constructor.

Term2IdWriteProc

protected Term2IdWriteProc(IRabaCoder keySer,
                int fromIndex,
                int toIndex,
                byte[][] keys,
                boolean readOnly,
                boolean storeBlankNodes,
                int scaleOutTermIdBitsToReverse)

Method Detail

isReadOnly

public final boolean isReadOnly()

Description copied from interface: IReadOnly

Return true iff the procedure asserts that it will not write on the index. When true, the procedure may be run against a view of the index that is read-only or which allows concurrent processes to read on the same index object. When false the procedure will be run against a mutable view of the index (assuming that the procedure is executed in a context that has access to a mutable index view).

Specified by:

isReadOnly in interface IReadOnly

isStoreBlankNodes

public final boolean isStoreBlankNodes()

applyOnce

public Term2IdWriteProc.Result applyOnce(IIndex ndx,
                                IRaba keys,
                                IRaba vals)

For each term whose serialized key is mapped to the current index partition, lookup the term in the terms index. If it is there then note its assigned termId. Otherwise, use the partition local counter to assign the term identifier, note the term identifier so that it can be communicated back to the client, and insert the {term,termId} entry into the terms index.

Specified by:

applyOnce in class AbstractKeyArrayIndexProcedure<Term2IdWriteProc.Result>

Parameters:

ndx - The terms index.

Returns:

The Term2IdWriteProc.Result, which contains the discovered / assigned term identifiers. TODO no point sending bnodes when readOnly.

readMetadata

protected void readMetadata(ObjectInput in)
                     throws IOException,
                            ClassNotFoundException

Description copied from class: AbstractKeyArrayIndexProcedure

Reads metadata written by AbstractKeyArrayIndexProcedure.writeMetadata(ObjectOutput).

Overrides:

readMetadata in class AbstractKeyArrayIndexProcedure<Term2IdWriteProc.Result>

Throws:

IOException

ClassNotFoundException

writeMetadata

protected void writeMetadata(ObjectOutput out)
                      throws IOException

Writes metadata (not the keys or values, but just other metadata used by the procedure).

The default implementation writes toIndex - fromIndex, which is the #of keys.

Overrides:

writeMetadata in class AbstractKeyArrayIndexProcedure<Term2IdWriteProc.Result>

Parameters:

out -

Throws:

IOException

VTE

public static final VTE VTE(byte code)

newAggregator

protected IResultHandler<Term2IdWriteProc.Result,Term2IdWriteProc.Result> newAggregator()

Split-wise aggregation followed by combining the results across those splits in order to return an aggregated result whose iv[] is 1:1 with the original keys[][].

Specified by:

newAggregator in class AbstractKeyArrayIndexProcedure<Term2IdWriteProc.Result>

Returns:

The IResultHandler -or- null

See Also:

NopAggregator, IParallelizableIndexProcedure, (Schedule more IOs when loading data)