com.bigdata.journal

Class Name2Addr

java.lang.Object

com.bigdata.btree.AbstractBTree

com.bigdata.btree.BTree

com.bigdata.journal.Name2Addr

All Implemented Interfaces:

IAutoboxBTree, IBTreeStatistics, ICheckpointProtocol, IIndex, IIndexLocalCounter, ILinearList, ILocalBTreeView, IRangeQuery, IReadWriteLockManager, ISimpleBTree, ISimpleIndexAccess, ISimpleTreeIndexAccess, ICounterSetAccess, ICommitter

public class Name2Addr
extends BTree

Name2Addr is a BTree mapping index names to an Name2Addr.Entry containing the last Checkpoint record committed for the named index and the timestamp of that commit. The keys are Unicode strings using the default Locale. The Name2Addr.Entrys in Name2Addr are the set of registered named indices for an AbstractJournal.

The AbstractJournal maintains an instance of this class that evolves with each AbstractJournal.commit() and tracks the Checkpoint records of the registered ITx.UNISOLATED indices. However, the journal also makes use of historical states for the Name2Addr index in order to resolve the historical state of a named index. Of necessity, the Name2Addr objects used for this latter purpose MUST be distinct from the evolving instance otherwise the current version of the named index would be resolved. Note further that the historical Name2Addr states are accessed using a canonicalizing mapping but that current evolving Name2Addr instance is NOT part of that mapping.

Concurrent reads are permitted against the historical Name2Addr objects since the BTree is thread-safe for read-only operations. Likewise, writes are only allowed on the ITx.UNISOLATED Name2Addr instance. Write access to the underlying BTree MUST be synchronized on the ITx.UNISOLATED Name2Addr instance since the BTree is NOT safe for concurrent writers. Further, read access to ITx.UNISOLATED Name2Addr object MUST be synchronized so as to not conflict with writes on that BTree. Therefore all write methods on this class are declared as synchronized but the caller MUST synchronize on Name2Addr if they are performing reads on the ITx.UNISOLATED Name2Addr instance. This allows readers on historical Name2Addr instances to have full concurrency.

Note: Name2Addr by itself is NOT sufficient to handle commits with concurrent task execution, such as arises with the group commit protocol. The problem is concurrency in the data structure that keeps track of add/drop for named indices and also tracks which named indices are dirty. In order to account for tasks running concurrent with commit processing, AbstractTask isolates Name2Addr and makes the set of changes {registering indices, dropping indices, and updating the Name2Addr.Entry in Name2Addr to reflect the current Checkpoint record for an index) an atomic state change that is performed IFF the task completes successfully and is synchronized on Name2Addr to prevent that happening concurrent with commit processing.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Name2Addr.Entry An entry in the persistent index.
static class	Name2Addr.EntrySerializer The values are Name2Addr.Entrys.
static class	Name2Addr.Name2AddrTupleSerializer Encapsulates key and value formation for Name2Addr.

Nested classes/interfaces inherited from class com.bigdata.btree.BTree

BTree.Counter, BTree.LeafCursor, BTree.NodeFactory, BTree.PartitionedCounter, BTree.Stack

Nested classes/interfaces inherited from class com.bigdata.btree.AbstractBTree

AbstractBTree.IBTreeCounters

Field Summary

Fields inherited from class com.bigdata.btree.BTree

counter, height, nentries, nleaves, nnodes, recordVersion

Fields inherited from class com.bigdata.btree.AbstractBTree

branchingFactor, debug, DEBUG, dumpLog, error, ERROR_CLOSED, ERROR_ERROR_STATE, ERROR_LESS_THAN_ZERO, ERROR_READ_ONLY, ERROR_TOO_LARGE, ERROR_TRANSIENT, INFO, metadata, ndistinctOnWriteRetentionQueue, nodeSer, readOnly, root, store, storeCache, writeRetentionQueue

Fields inherited from interface com.bigdata.btree.IRangeQuery

ALL, CURSOR, DEFAULT, DELETED, FIXED_LENGTH_SUCCESSOR, KEYS, NONE, PARALLEL, READONLY, REMOVEALL, REVERSE, VALS

Constructor Summary

Constructors

Constructor and Description
Name2Addr(IRawStore store, Checkpoint checkpoint, IndexMetadata metadata, boolean readOnly) Load from the store (de-serialization constructor).

Method Summary

Methods

Modifier and Type	Method and Description
protected void	assertUnisolatedInstance() Many methods on this class will throw an IllegalStateException unless they are invoked on the ITx.UNISOLATED Name2Addr instance.
static Name2Addr	create(IRawStore store) Create a new instance.
void	dropIndex(String name) Removes the entry for the named index.
Name2Addr.Entry	getEntry(String name) Return the Name2Addr.Entry for the named index.
ICheckpointProtocol	getIndex(String name) Return the ITx.UNISOLATED view of the named persistence capable data structure - this method tests a cache of the named persistence capable data structures and will return the existing instance if the index is found in the cache and will otherwise load the ITx.UNISOLATED view of the data structure from the backing store.
protected ICheckpointProtocol	getIndexCache(String name) Return the current entry, if any, for the named ITx.UNISOLATED index in the indexCache.
int	getIndexCacheSize() Return the approximate number of indices in the live index cache.
protected CounterSet	getIndexCounters(CounterSet counterSet, Set<String> found) Return a CounterSet reflecting the named indices that are currently open (more accurately, those open named indices whose references are in Name2Addrs internal indexCache).
long	handleCommit(long commitTime) Commit processing for named indices.
static Iterator<String>	indexNameScan(String prefix, IIndex n2a) Prefix scan of a Name2Addr index.
void	invalidate(Throwable t) Mark an ICommitter as invalid.
protected boolean	isUnisolatedInstance() Return true iff this is the ITx.UNISOLATED Name2Addr instance.
protected void	putIndexCache(String name, ICheckpointProtocol btree, boolean replace) Adds the named index to the ITx.UNISOLATED index cache.
protected void	putOnCommitList(String name, ICheckpointProtocol btree, boolean needsCheckpoint) Adds the named index to the commit list and sets a DirtyListener on the ICheckpointProtocol so that this Name2Addr object will be informed if the associated persistent data structure becomes dirty.
void	registerIndex(String name, ICheckpointProtocol btree) Add an entry for the named index.
protected void	setupCache(int cacheCapacity, long cacheTimeout) Setup the indexCache.
boolean	willCommit(String name) Return true iff the named index is on the commit list.

Methods inherited from class com.bigdata.btree.BTree

_reopen, asReadOnly, create, createTransient, createViewCheckpoint, fireDirtyEvent, getBloomFilter, getCheckpoint, getCounter, getDirtyListener, getEntryCount, getHeight, getLastCommitTime, getLeafCount, getMetadataAddr, getMutableBTree, getNodeCount, getRecordVersion, getRevisionTimestamp, getRootAddr, getSourceCount, getSources, getStore, load, needsCheckpoint, newLeafCursor, newLeafCursor, readBloomFilter, removeAll, setDirtyListener, setIndexMetadata, setLastCommitTime, writeCheckpoint, writeCheckpoint2

Methods inherited from class com.bigdata.btree.AbstractBTree

assertNotReadOnly, assertNotTransient, close, contains, contains, decodeRecordAddr, dump, dump, dumpPages, encodeRecordAddr, getBranchingFactor, getBtreeCounters, getContainsTuple, getCounters, getIndexMetadata, getLevel, getLevel, getLookupTuple, getNodeSerializer, getReadLockCount, getResourceMetadata, getRightMostNode, getRoot, getRootOrFinger, getStatistics, getUtilization, getWriteTuple, indexOf, insert, insert, insert, isBalanced, isOpen, isReadOnly, isTransient, keyAt, lookup, lookup, lookup, putIfAbsent, rangeCheck, rangeCopy, rangeCount, rangeCount, rangeCount, rangeCountExact, rangeCountExactWithDeleted, rangeIterator, rangeIterator, rangeIterator, rangeIterator, rangeIterator, readLock, readNodeOrLeaf, recycle, remove, remove, remove, reopen, scan, setBTreeCounters, submit, submit, submit, toString, touch, valueAt, valueAt, writeLock, writeNodeOrLeaf, writeNodeRecursive, writeNodeRecursiveCallersThread, writeNodeRecursiveConcurrent

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.bigdata.btree.ICheckpointProtocol

close, dumpPages, getIndexMetadata, isOpen, reopen

Methods inherited from interface com.bigdata.counters.ICounterSetAccess

getCounters

Methods inherited from interface com.bigdata.btree.ISimpleIndexAccess

rangeCount, scan

Methods inherited from interface com.bigdata.btree.IReadWriteLockManager

getReadLockCount, isReadOnly, readLock, writeLock

Constructor Detail

Name2Addr

public Name2Addr(IRawStore store,
         Checkpoint checkpoint,
         IndexMetadata metadata,
         boolean readOnly)

Load from the store (de-serialization constructor).

Parameters:

store - The backing store.

checkpoint - The Checkpoint record.

metadata - The metadata record for the index.

Method Detail

create

public static Name2Addr create(IRawStore store)

Create a new instance.

Parameters:

store - The backing store.

Returns:

The new instance.

assertUnisolatedInstance

protected final void assertUnisolatedInstance()

Many methods on this class will throw an IllegalStateException unless they are invoked on the ITx.UNISOLATED Name2Addr instance. This method is used to test that assertion for those methods.

Throws:

IllegalStateException - unless this is the ITx.UNISOLATED Name2Addr instance.

isUnisolatedInstance

protected final boolean isUnisolatedInstance()

Return true iff this is the ITx.UNISOLATED Name2Addr instance.

setupCache

protected void setupCache(int cacheCapacity,
              long cacheTimeout)

Setup the indexCache.

Note: This cache is null unless initialized and is ONLY used by the "live" version of the Name2Addr index. The only method that creates or loads the "live" Name2Addr index is AbstractJournal#setupName2AddrBTree().

Parameters:

cacheCapacity - The capacity of the inner LRUCache for the WeakValueCache.

cacheTimeout - The timeout in milliseconds for stale entries in the cache.

See Also:

Options.LIVE_INDEX_CACHE_CAPACITY, Options.LIVE_INDEX_CACHE_TIMEOUT

getIndexCacheSize

public int getIndexCacheSize()

Return the approximate number of indices in the live index cache.

Throws:

IllegalStateException - unless this is the ITx.UNISOLATED instance.

willCommit

public boolean willCommit(String name)

Return true iff the named index is on the commit list.

Note: This is synchronized even through the commitList is thread-safe in order to make the test atomic with respect to handleCommit(long).

Parameters:

name - The index name.

Throws:

IllegalStateException - unless this is the ITx.UNISOLATED instance.

handleCommit

public long handleCommit(long commitTime)

Commit processing for named indices.

This method applies the commitList and then flushes the backing ICheckpointProtocol object to the store. The commitList consists of DirtyListeners. If the listener has its DirtyListener#needsCheckpoint flag set, then the ICheckpointProtocol implementation to which that listener is attached will have its checkpoint written . Otherwise the current Checkpoint address is recovered. Either way, the Name2Addr.Entry in Name2Addrs backing BTree is updated to reflect the commitTime and Checkpoint address for the index.

Finally Name2Addr Checkpoints itself using ICommitter.handleCommit(long) and returns the address from which Name2Addr may be reloaded.

Note: The commitList MUST be protected against concurrent modification during the commit otherwise concurrent tasks could be reporting dirty objects while we are doing a commit and those notices would be lost. Persistence capable objects (ICheckpointProtocol implementations) get onto the commitList via the DirtyListener, so it is also synchronized.

Note: Name2Addr DOES NOT obtain a resource lock on the ICheckpointProtocol implementation. Therefore it MUST NOT checkpoint an index on which an AbstractTask has obtained a resource lock. Otherwise we have concurrent writers on the BTree and the BTree is not thread-safe for concurrent writers. Instead, the AbstractTask checkpoints the BTree itself while it is holding the resource lock and then sets DirtyListener#needsCheckpoint to false using putOnCommitList(String, ICheckpointProtocol, boolean) as an indication to Name2Addr that it MUST persist the current checkpointAddr for the BTree on its next commit (and MUST NOT write on the index when it does that commit).

Specified by:

handleCommit in interface ICommitter

Overrides:

handleCommit in class BTree

Parameters:

commitTime - The timestamp assigned to the commit.

Returns:

The address of a Checkpoint record from which the btree may be reloaded.

See Also:

Flush indices in parallel during checkpoint to reduce IO latency

getIndex

public ICheckpointProtocol getIndex(String name)

Return the ITx.UNISOLATED view of the named persistence capable data structure - this method tests a cache of the named persistence capable data structures and will return the existing instance if the index is found in the cache and will otherwise load the ITx.UNISOLATED view of the data structure from the backing store.

Parameters:

name - The index name.

Returns:

The named index or null iff there is no index with that name.

Throws:

IllegalArgumentException - if name is null.

IllegalStateException - if this is not the ITx.UNISOLATED Name2Addr instance.

getEntry

public Name2Addr.Entry getEntry(String name)

Return the Name2Addr.Entry for the named index.

Note: This is a lower-level access mechanism that is used by Journal#getIndex(String, ICommitRecord) when accessing historical named indices from an ICommitRecord.

Parameters:

name - The index name.

Returns:

The Name2Addr.Entry for the named index -or- null if there is no entry for that name.

registerIndex

public void registerIndex(String name,
                 ICheckpointProtocol btree)

Add an entry for the named index.

Parameters:

name - The index name.

btree - The index.

Throws:

IllegalArgumentException - if name is null.

IllegalArgumentException - if btree is null.

IndexExistsException - if there is already an index registered under that name.

putOnCommitList

protected void putOnCommitList(String name,
                   ICheckpointProtocol btree,
                   boolean needsCheckpoint)

Adds the named index to the commit list and sets a DirtyListener on the ICheckpointProtocol so that this Name2Addr object will be informed if the associated persistent data structure becomes dirty.

Parameters:

name - The index name.

btree - The persistence capable data structure.

needsCheckpoint - Specify true if Name2Addr should invoke ICheckpointProtocol.writeCheckpoint() rather than just updating the Name2Addr.Entry for the persistenc capable data structure using ICheckpointProtocol.getCheckpoint()

putIndexCache

protected void putIndexCache(String name,
                 ICheckpointProtocol btree,
                 boolean replace)

Adds the named index to the ITx.UNISOLATED index cache.

Parameters:

name - The index name.

btree - The ITx.UNISOLATED view of the persistence capable data structure.

replace - If an existing entry for that name may be replaced.

getIndexCache

protected ICheckpointProtocol getIndexCache(String name)

Return the current entry, if any, for the named ITx.UNISOLATED index in the indexCache.

Note: This method is more direct than getIndex(String). AbstractTask uses this method together with putIndexCache(String, ICheckpointProtocol, boolean) to allow different tasks access to the same pool of ITx.UNISOLATED indices.

Parameters:

name - The index name.

Returns:

The index iff it was found in the cache.

dropIndex

public void dropIndex(String name)

Removes the entry for the named index. The named index will no longer participate in commits.

Parameters:

name - The index name.

Throws:

IllegalArgumentException - if name is null.

NoSuchIndexException - if the index does not exist.

invalidate

public void invalidate(Throwable t)

Description copied from interface: ICommitter

Mark an ICommitter as invalid. This will prevent it from allowing any writes through to the backing store.

Specified by:

invalidate in interface ICommitter

Overrides:

invalidate in class BTree

Parameters:

t - A cause (required).

See Also:

https://jira.blazegraph.com/browse/BLZG-1953

getIndexCounters

protected CounterSet getIndexCounters(CounterSet counterSet,
                          Set<String> found)

Return a CounterSet reflecting the named indices that are currently open (more accurately, those open named indices whose references are in Name2Addrs internal indexCache). When index partitions are in use their CounterSets are reported under a path formed from name of the scale-out index and partition identifier. Otherwise the CounterSets are reported directly under the index name.

Parameters:

counterSet - When non-null the performance counters are entered into the caller's collection. Otherwise they are entered into a new collection.

found - When non-null, the names of the indices whose performance counters are being returned is reported as a side-effect on this Set.

Returns:

A new CounterSet reflecting the named indices that were open as of the time that this method was invoked.

See Also:

IndexManager.getIndexCounters(), Expose performance counters for read-only indices

indexNameScan

public static final Iterator<String> indexNameScan(String prefix,
                             IIndex n2a)

Prefix scan of a Name2Addr index. This scan assumes that the caller has provided for any possible thread-safety issues.

Parameters:

prefix - The prefix.

n2a - The index.

Returns:

The names of the indices spanned by that prefix in that index.

See Also:

Name2Addr.indexNameScan(prefix) uses scan + filter , AbstractTripleStore.destroy() does not filter for correct prefix