Stream (Blazegraph Database Platform 2.1.5 API)

java.lang.Object
- com.bigdata.stream.Stream

All Implemented Interfaces:

ICheckpointProtocol, IReadWriteLockManager, ISimpleIndexAccess, ICounterSetAccess, ICommitter

Direct Known Subclasses:

SolutionSetStream
```
public abstract class Stream
extends Object
implements ICheckpointProtocol
```
A persistence capable stream of "index" entries. The stream maintains the order in which the entries were written.

Author:

Bryan Thompson FIXME GIST : The base Stream should handle byte[]s, much like the basic BTree or HTree. That way it can be a concrete class and used for a variety of things. It might also allow an append() method (similar to insert) for non-iterator based incremental write protocols. This would be really just a thin wrapper over the IPSOutputStream, but integrated with the ICheckpointProtocol and (ideally) the MVCC architecture.

See Also:
GIST

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`Stream.CompressionEnum` The type of compression used on the stream.
`static class`	`Stream.StreamIndexMetadata` Metadata for a named solution set.

Field Summary

Fields
Modifier and Type	Field and Description
`protected long`	`entryCount` The #of entries in the stream.
`protected Throwable`	`error`
`protected static String`	`ERROR_CLOSED` The index is already closed.
`protected static String`	`ERROR_ERROR_STATE` The index object is no longer valid.
`protected static String`	`ERROR_READ_ONLY` The index is read-only but a mutation operation was requested.
`protected Stream.StreamIndexMetadata`	`metadata` The metadata record for the index.
`protected AtomicBoolean`	`open` `true` iff the `Stream` is open.
`protected long`	`rootAddr` The address from which the stream may be read.

Constructor Summary

Constructors
Constructor and Description

Stream(IRawStore store, Checkpoint checkpoint, IndexMetadata metadata, boolean readOnly)
Required constructor.

Constructors
Constructor and Description
`Stream(IRawStore store, Checkpoint checkpoint, IndexMetadata metadata, boolean readOnly)` Required constructor.

Method Summary

Methods
Modifier and Type	Method and Description
`protected void`	`assertNotReadOnly()`
`protected void`	`assertNotTransient()` NOP.
`void`	`clear()`
`void`	`close()` The contract for `ICheckpointProtocol.close()` is to reduce the resource burden of the index while not rendering the index inoperative.
`static SolutionSetStream`	`create(IRawStore store, Stream.StreamIndexMetadata metadata)` Create a new `SolutionSetStream` or derived class.
`protected void`	`fireDirtyEvent()` Fire an event to the listener (iff set).
`protected void`	`flush()` Hook to flush anything which is dirty to the backing store.
`Checkpoint`	`getCheckpoint()` Returns the most recent `ICheckpoint` record.
`CounterSet`	`getCounters()` Return performance counters.
`IDirtyListener`	`getDirtyListener()` Return the `IDirtyListener`.
`Stream.StreamIndexMetadata`	`getIndexMetadata()` The metadata for the index.
`long`	`getLastCommitTime()` The timestamp associated with the last `IAtomicStore.commit()` in which writes buffered by this index were made restart-safe on the backing store.
`long`	`getMetadataAddr()` The address at which the most recent `IndexMetadata` record was written.
`int`	`getReadLockCount()` Return the #of read-locks held by the current thread for a mutable index view.
`long`	`getRecordVersion()` The value of the record version number that will be assigned to the next node or leaf written onto the backing store.
`long`	`getRootAddr()` The address of the last written root of the persistent data structure -or- `0L` if there is no root.
`IRawStore`	`getStore()` The backing store.
`long`	`handleCommit(long commitTime)` Flush dirty state to the store in preparation for an atomic commit and return the address from which the persistence capable data structure may be reloaded.
`void`	`invalidate(Throwable t)` Mark an `ICommitter` as invalid.
`boolean`	`isOpen()` An "open" index has may have some buffered data.
`boolean`	`isReadOnly()` Return `true` iff this B+Tree is read-only.
`static SolutionSetStream`	`load(IRawStore store, long addrCheckpoint, boolean readOnly)` Load an instance of a `HTree` or derived class from the store.
`protected boolean`	`needsCheckpoint()` Return true iff changes would be lost unless the `Stream` is flushed to the backing store using `writeCheckpoint()`.
`long`	`rangeCount()` Return the #of entries in the index.
`Lock`	`readLock()` Return a `Lock` that may be used to obtain a shared read lock which is used (in the absence of other concurrency control mechanisms) to permit concurrent readers on an unisolated index while serializing access to that index when a writer must run.
`protected int`	`recycle(long addr)` Recycle (aka delete) the allocation.
`void`	`removeAll()` Remove all entries in the index.
`void`	`reopen()` (Re-) open the index.
`abstract ICloseableIterator<?>`	`scan()` Visit all entries in the index in the natural order of the index (dereferencing visited tuples to the application objects stored within those tuples).
`protected void`	`setCheckpoint(Checkpoint checkpoint)` Sets the `checkpoint` and initializes the mutable fields from the checkpoint record.
`void`	`setDirtyListener(IDirtyListener listener)` Set or clear the listener (there can be only one).
`void`	`setLastCommitTime(long lastCommitTime)` Sets the lastCommitTime.
`protected InputStream`	`wrapInputStream(InputStream in)` Wrap for decompression.
`protected OutputStream`	`wrapOutputStream(OutputStream out)` Apply the configured `Stream.CompressionEnum` for the `Stream` to the caller's `OutputStream`.
`abstract void`	`write(ICloseableIterator<?> src)` Write entries onto the stream.
`long`	`writeCheckpoint()` Checkpoint operation must `#flush()` dirty nodes, dirty persistent data structures, etc, write a new `Checkpoint` record on the backing store, save a reference to the current `Checkpoint`, and return the address of that `Checkpoint` record.
`Checkpoint`	`writeCheckpoint2()` Checkpoint operation must `#flush()` dirty nodes, dirty persistent data structures, etc, write a new `Checkpoint` record on the backing store, save a reference to the current `Checkpoint`, and return the address of that `Checkpoint` record.
`Lock`	`writeLock()` Return a `Lock` that may be used to obtain an exclusive write lock which is used (in the absence of other concurrency control mechanisms) to serialize all processes accessing an unisolated index when a writer must run.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.btree.ICheckpointProtocol
dumpPages

- Field Detail
  - ERROR_CLOSED
```
protected static final String ERROR_CLOSED
```
    The index is already closed.
    
    See Also:
    Constant Field Values
  - ERROR_READ_ONLY
```
protected static final String ERROR_READ_ONLY
```
    The index is read-only but a mutation operation was requested.
    
    See Also:
    Constant Field Values
  - ERROR_ERROR_STATE
```
protected static final String ERROR_ERROR_STATE
```
    The index object is no longer valid.
    
    See Also:
    Constant Field Values
  - error
```
protected volatile Throwable error
```
  - open
```
protected final AtomicBoolean open
```
    true iff the Stream is open.
  - entryCount
```
protected long entryCount
```
    The #of entries in the stream.
  - rootAddr
```
protected long rootAddr
```
    The address from which the stream may be read. This is mutable. On update, it is the address at which the stream was last written. The current value gets propagated into the Checkpoint record as Checkpoint.getRootAddr().
  - metadata
```
protected Stream.StreamIndexMetadata metadata
```
    The metadata record for the index. This data rarely changes during the life of the solution set, but it CAN be changed.
- Constructor Detail
  - Stream
```
public Stream(IRawStore store,
      Checkpoint checkpoint,
      IndexMetadata metadata,
      boolean readOnly)
```
    Required constructor. This constructor is used both to create a new named solution set, and to load an existing named solution set from the store using a Checkpoint record.
    
    Parameters:
    store - The store.
    checkpoint - The Checkpoint record.
    metadata - The metadata record.
    readOnly - When true the view will be immutable.
    See Also:
    create(IRawStore, StreamIndexMetadata), load(IRawStore, long, boolean)
- Method Detail
  - getStore
```
public IRawStore getStore()
```
    Description copied from interface: ISimpleIndexAccess
    
    The backing store.
    
    Specified by:
    
    getStore in interface ISimpleIndexAccess
  - setCheckpoint
```
protected void setCheckpoint(Checkpoint checkpoint)
```
    Sets the checkpoint and initializes the mutable fields from the checkpoint record. In order for this operation to be atomic, the caller must be synchronized on the BTree or otherwise guaranteed to have exclusive access, e.g., during the ctor or when the BTree is mutable and access is therefore required to be single-threaded.
  - create
```
public static SolutionSetStream create(IRawStore store,
                       Stream.StreamIndexMetadata metadata)
```
    Create a new SolutionSetStream or derived class. This method works by writing the Stream.StreamIndexMetadata record on the store and then loading the SolutionSetStream from the Stream.StreamIndexMetadata record.
    
    Parameters:
    store - The store.
    metadata - The metadata record.
    
    Returns:
    The newly created SolutionSetStream.
    
    Throws:
    
    IllegalStateException - If you attempt to create two HTree objects from the same metadata record since the metadata address will have already been noted on the IndexMetadata object. You can use IndexMetadata.clone() to obtain a new copy of the metadata object with the metadata address set to 0L.
    
    IllegalStateException - if the IndexTypeEnum in the supplied IndexMetadata object is not IndexTypeEnum.BTree.
    See Also:
    load(IRawStore, long, boolean)
  - load
```
public static SolutionSetStream load(IRawStore store,
                     long addrCheckpoint,
                     boolean readOnly)
```
    Load an instance of a HTree or derived class from the store. The HTree or derived class MUST declare a constructor with the following signature: className(IRawStore store, Checkpoint checkpoint, IndexMetadata metadata, boolean readOnly)
    
    Parameters:
    store - The store.
    addrCheckpoint - The address of a Checkpoint record for the index.
    readOnly - When true the BTree will be marked as read-only. Marking has some advantages relating to the locking scheme used by Node.getChild(int) since the root node is known to be read-only at the time that it is allocated as per-child locking is therefore in place for all nodes in the read-only BTree. It also results in much higher concurrency for AbstractBTree.touch(AbstractNode).
    
    Returns:
    The HTree or derived class loaded from that Checkpoint record.
    
    Throws:
    
    IllegalArgumentException - if store is null.
  - clear
```
public void clear()
```
  - wrapOutputStream
```
protected OutputStream wrapOutputStream(OutputStream out)
```
    Apply the configured Stream.CompressionEnum for the Stream to the caller's OutputStream.
    
    Parameters:
    out - The OutputStream.
    
    Returns:
    The wrapped OutputStream.
    
    Throws:
    
    IOException
  - wrapInputStream
```
protected InputStream wrapInputStream(InputStream in)
```
    Wrap for decompression.
  - isReadOnly
```
public final boolean isReadOnly()
```
    Return true iff this B+Tree is read-only.
    
    Specified by:
    
    isReadOnly in interface IReadWriteLockManager
  - assertNotReadOnly
```
protected final void assertNotReadOnly()
```
    Throws:
    
    UnsupportedOperationException - if the B+Tree is read-only.
    See Also:
    isReadOnly()
  - assertNotTransient
```
protected final void assertNotTransient()
```
    NOP. Implemented for compatibility with the interior APIs of the HTree and BTree, but Stream does not support "transient" data (in the sense of data which is not evicted to a backing store).
  - getCounters
```
public CounterSet getCounters()
```
    Description copied from interface: ICounterSetAccess
    
    Return performance counters.
    
    Specified by:
    
    getCounters in interface ICounterSetAccess
  - handleCommit
```
public long handleCommit(long commitTime)
```
    Description copied from interface: ICommitter
    
    Flush dirty state to the store in preparation for an atomic commit and return the address from which the persistence capable data structure may be reloaded.
    
    Specified by:
    
    handleCommit in interface ICommitter
    
    Parameters:
    commitTime - The timestamp assigned to the commit.
    
    Returns:
    The address of the record from which the persistence capable data structure may be reloaded. If no changes have been made then the previous address should be returned as it is still valid.
  - invalidate
```
public void invalidate(Throwable t)
```
    Description copied from interface: ICommitter
    
    Mark an ICommitter as invalid. This will prevent it from allowing any writes through to the backing store.
    
    Specified by:
    
    invalidate in interface ICommitter
    
    Parameters:
    t - A cause (required).
    See Also:
    https://jira.blazegraph.com/browse/BLZG-1953
  - getRecordVersion
```
public long getRecordVersion()
```
    Description copied from interface: ICheckpointProtocol
    
    The value of the record version number that will be assigned to the next node or leaf written onto the backing store. This number is incremented each time a node or leaf is written onto the backing store. The initial value is ZERO (0). The first value assigned to a node or leaf will be ZERO (0). TODO Nobody is actually incrementing this value right now.
    
    Specified by:
    
    getRecordVersion in interface ICheckpointProtocol
  - getCheckpoint
```
public final Checkpoint getCheckpoint()
```
    Description copied from interface: ICheckpointProtocol
    
    Returns the most recent ICheckpoint record.
    
    Specified by:
    
    getCheckpoint in interface ICheckpointProtocol
    
    Returns:
    The most recent ICheckpoint record and never null.
  - writeCheckpoint
```
public final long writeCheckpoint()
```
    Description copied from interface: ICheckpointProtocol
    
    Checkpoint operation must #flush() dirty nodes, dirty persistent data structures, etc, write a new Checkpoint record on the backing store, save a reference to the current Checkpoint, and return the address of that Checkpoint record.
    Note: A checkpoint by itself is NOT an atomic commit. The commit protocol is at the store level and uses Checkpoints to ensure that the state of the persistence capable data structure is current on the backing store.
    
    Specified by:
    
    writeCheckpoint in interface ICheckpointProtocol
    
    Returns:
    The address at which the Checkpoint record for the persistence capable was written onto the store. The data structure can be reloaded from this Checkpoint record.
  - writeCheckpoint2
```
public final Checkpoint writeCheckpoint2()
```
    Description copied from interface: ICheckpointProtocol
    
    Checkpoint operation must #flush() dirty nodes, dirty persistent data structures, etc, write a new Checkpoint record on the backing store, save a reference to the current Checkpoint, and return the address of that Checkpoint record.
    Note: A checkpoint by itself is NOT an atomic commit. The commit protocol is at the store level and uses Checkpoints to ensure that the state of the persistence capable data structure is current on the backing store.
    
    Specified by:
    
    writeCheckpoint2 in interface ICheckpointProtocol
    
    Returns:
    The Checkpoint record for the persistent data structure which was written onto the store. The persistent data structure can be reloaded from this Checkpoint record.
  - needsCheckpoint
```
protected boolean needsCheckpoint()
```
    Return true iff changes would be lost unless the Stream is flushed to the backing store using writeCheckpoint().
    
    Returns:
    true true iff changes would be lost unless the Stream was flushed to the backing store using writeCheckpoint().
  - flush
```
protected void flush()
```
    Hook to flush anything which is dirty to the backing store. This is invoked by _writeCheckpoint2().
  - recycle
```
protected int recycle(long addr)
```
    Recycle (aka delete) the allocation. This method also adjusts the #of bytes released in the BTreeCounters.
    
    Parameters:
    addr - The address to be recycled.
    
    Returns:
    The #of bytes which were recycled and ZERO (0) if the address is IAddressManager.NULL.
  - getRootAddr
```
public final long getRootAddr()
```
    The address of the last written root of the persistent data structure -or- 0L if there is no root. A 0L return may be an indication that an empty data structure will be created on demand.
    
    Specified by:
    
    getRootAddr in interface ICheckpointProtocol
    
    Returns:
    The address from which the solutions may be read.
  - getMetadataAddr
```
public final long getMetadataAddr()
```
    Description copied from interface: ICheckpointProtocol
    
    The address at which the most recent IndexMetadata record was written.
    
    Specified by:
    
    getMetadataAddr in interface ICheckpointProtocol
  - getIndexMetadata
```
public Stream.StreamIndexMetadata getIndexMetadata()
```
    Description copied from interface: ICheckpointProtocol
    
    The metadata for the index. This is full of good stuff about the index.
    Note: The same method is also declared by IIndex in order to provide access to the IndexMetadata for remote clients in scale-out.
    
    Specified by:
    
    getIndexMetadata in interface ICheckpointProtocol
    
    See Also:
    IIndex.getIndexMetadata()
  - getDirtyListener
```
public final IDirtyListener getDirtyListener()
```
    Description copied from interface: ICheckpointProtocol
    
    Return the IDirtyListener.
    
    Specified by:
    
    getDirtyListener in interface ICheckpointProtocol
  - getLastCommitTime
```
public final long getLastCommitTime()
```
    Description copied from interface: ICheckpointProtocol
    
    The timestamp associated with the last IAtomicStore.commit() in which writes buffered by this index were made restart-safe on the backing store. The lastCommitTime is set when the index is loaded from the backing store and updated after each commit. It is ZERO (0L) when an index is first created and will remain ZERO (0L) until the index is committed. If the backing store does not support atomic commits, then this value will always be ZERO (0L).
    
    Specified by:
    
    getLastCommitTime in interface ICheckpointProtocol
  - setLastCommitTime
```
public final void setLastCommitTime(long lastCommitTime)
```
    Description copied from interface: ICheckpointProtocol
    
    Sets the lastCommitTime.
    Note: The lastCommitTime is set by a combination of the AbstractJournal and Name2Addr based on the actual commitTime of the commit during which an Name2Addr.Entry for that index was last committed. It is set for both historical index reads and unisolated index reads using Name2Addr.Entry.commitTime. The lastCommitTime for an unisolated index will advance as commits are performed with that index.
    
    Specified by:
    
    setLastCommitTime in interface ICheckpointProtocol
    
    Parameters:
    lastCommitTime - The timestamp of the last committed state of this index.
  - setDirtyListener
```
public final void setDirtyListener(IDirtyListener listener)
```
    Description copied from interface: ICheckpointProtocol
    
    Set or clear the listener (there can be only one).
    
    Specified by:
    
    setDirtyListener in interface ICheckpointProtocol
    
    Parameters:
    listener - The listener.
  - fireDirtyEvent
```
protected final void fireDirtyEvent()
```
    Fire an event to the listener (iff set).
  - rangeCount
```
public long rangeCount()
```
    Description copied from interface: ISimpleIndexAccess
    
    Return the #of entries in the index.
    Note: If the index supports deletion markers then the range count will be an upper bound and may double count tuples which have been overwritten, including the special case where the overwrite is a delete.
    
    Specified by:
    
    rangeCount in interface ISimpleIndexAccess
    
    Returns:
    The #of tuples in the index.
    See Also:
    IRangeQuery.rangeCount()
  - removeAll
```
public void removeAll()
```
    Description copied from interface: ISimpleIndexAccess
    
    Remove all entries in the index.
    
    Specified by:
    
    removeAll in interface ISimpleIndexAccess
  - close
```
public void close()
```
    Description copied from interface: ICheckpointProtocol
    
    The contract for ICheckpointProtocol.close() is to reduce the resource burden of the index while not rendering the index inoperative. An index that has been closed MAY be reopened at any time (conditional on the continued availability of the backing store). Such an index reference remains valid after a ICheckpointProtocol.close(). A closed index is transparently reopened by any access to the index data (scanning the index, probing the index, etc).
    Note: A ICheckpointProtocol.close() on a dirty index MUST discard writes rather than flushing them to the store and MUST NOT update its Checkpoint record - (ICheckpointProtocol.close() is used to discard indices with partial writes when an AbstractTask fails). If you are seeking to ICheckpointProtocol.close() a mutable index view that state can be recovered by ICheckpointProtocol.reopen() then you MUST write a new Checkpoint record before closing the index.
    
    Specified by:
    
    close in interface ICheckpointProtocol
  - reopen
```
public void reopen()
```
    Description copied from interface: ICheckpointProtocol
    
    (Re-) open the index. This method is part of a ICheckpointProtocol.close() / ICheckpointProtocol.reopen() protocol. That protocol may be used to reduce the resource burden of an index. This method is automatically invoked by a variety of methods that need to ensure that the index is available for use.
    
    Specified by:
    
    reopen in interface ICheckpointProtocol
    
    See Also:
    ICheckpointProtocol.close(), ICheckpointProtocol.isOpen(), #getRoot()
  - isOpen
```
public boolean isOpen()
```
    Description copied from interface: ICheckpointProtocol
    
    An "open" index has may have some buffered data. A closed index will have to re-read any backing data from the backing store.
    
    Specified by:
    
    isOpen in interface ICheckpointProtocol
    
    Returns:
    If the index is "open".
    See Also:
    ICheckpointProtocol.close(), ICheckpointProtocol.reopen()
  - scan
```
public abstract ICloseableIterator<?> scan()
```
    Description copied from interface: ISimpleIndexAccess
    
    Visit all entries in the index in the natural order of the index (dereferencing visited tuples to the application objects stored within those tuples).
    
    Specified by:
    
    scan in interface ISimpleIndexAccess
  - write
```
public abstract void write(ICloseableIterator<?> src)
```
    Write entries onto the stream.
    
    Parameters:
    src - An iterator visiting the entries.
  - readLock
```
public final Lock readLock()
```
    Description copied from interface: IReadWriteLockManager
    
    Return a Lock that may be used to obtain a shared read lock which is used (in the absence of other concurrency control mechanisms) to permit concurrent readers on an unisolated index while serializing access to that index when a writer must run. This is exposed for processes which need to obtain the write lock to coordinate external operations.
    Note: If the persistence capable data structure is read-only then the returned Lock is a singleton that ignores all lock requests. This is because our read-only persistence capable data structures are already thread-safe for concurrent readers.
    
    Specified by:
    
    readLock in interface IReadWriteLockManager
    
    Returns:
    The lock.
  - writeLock
```
public final Lock writeLock()
```
    Description copied from interface: IReadWriteLockManager
    
    Return a Lock that may be used to obtain an exclusive write lock which is used (in the absence of other concurrency control mechanisms) to serialize all processes accessing an unisolated index when a writer must run. This is exposed for processes which need to obtain the write lock to coordinate external operations.
    
    Specified by:
    
    writeLock in interface IReadWriteLockManager
    
    Returns:
    The lock.
  - getReadLockCount
```
public final int getReadLockCount()
```
    Description copied from interface: IReadWriteLockManager
    
    Return the #of read-locks held by the current thread for a mutable index view.
    
    Specified by:
    
    getReadLockCount in interface IReadWriteLockManager
    
    Returns:
    The #of reentrant read locks held by the current thread -or- ZERO if the index is read-only (read locks are not tracked for a read-only index view).

Class Stream

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface com.bigdata.btree.ICheckpointProtocol

Field Detail

ERROR_CLOSED

ERROR_READ_ONLY

ERROR_ERROR_STATE

error

open

entryCount

rootAddr

metadata

Constructor Detail

Stream

Method Detail

getStore

setCheckpoint

create

load

clear

wrapOutputStream

wrapInputStream

isReadOnly

assertNotReadOnly

assertNotTransient

getCounters

handleCommit

invalidate

getRecordVersion

getCheckpoint

writeCheckpoint

writeCheckpoint2

needsCheckpoint

flush

recycle

getRootAddr

getMetadataAddr

getIndexMetadata

getDirtyListener

getLastCommitTime

setLastCommitTime

setDirtyListener

fireDirtyEvent

rangeCount

removeAll

close

reopen

isOpen

scan

write

readLock

writeLock

getReadLockCount