Package com.bigdata.resources

This package provides the logic to managed the live journal and the historical journals and index segments for a DataService.

See: Description

Interface Summary

Interface	Description
IndexManager.IIndexManagerCounters	Performance counters for the IndexManager.
IndexManager.Options	Options understood by the IndexManager.
IPartitionIdFactory	An interface which shields callers from handling RMI exceptions and which allows the use of mock implementations when you do not really want to assign partition identifiers using the IMetadataService.nextPartitionId(String).
OverflowManager.IIndexPartitionTaskCounters	Performance counters for the index partition tasks.
OverflowManager.IOverflowManagerCounters	Performance counters for the OverflowManager.
OverflowManager.Options	Options understood by the OverflowManager.
ResourceManager.IResourceManagerCounters	Interface defines and documents the counters and counter namespaces for the ResourceManager.
ResourceManager.Options	ResourceManager options.
StoreManager.IStoreManagerCounters	Performance counters for the StoreManager.
StoreManager.Options	Options for the StoreManager.

Class Summary

Class	Description
AbstractAtomicUpdateTask<T>	Abstract base class for tasks responsible for the atomic update of the view of an index partition.
AbstractPrepareTask<T>	Base class for the prepare phase which reads on the old journal.
AbstractResourceManagerBootstrapTestCase
AbstractResourceManagerTask<T>	Abstract base class for tasks run during post-processing of a journal by the ResourceManager.
AbstractResourceManagerTestCase	Base class for ResourceManager test suites that can use normal startup and shutdown.
AbstractResourceManagerTestCase.MockMetadataService	A minimal implementation of IMetadataService - only those methods actually used by the ResourceManager are implemented.
AbstractResult	Abstract base class for results when post-processing a named index partition on the old journal after an overflow operation.
AbstractTestSegSplitter	Unit tests for splitting an index segment based on its size on the disk, the nominal size of an index partition, and an optional application level constraint on the choice of the separator keys.
AbstractTestSegSplitter.MockPartitionIdFactory	Mock implementation assigns index partitions from a counter beginning with ZERO (0), which is the first legal index partition identifier.
AbstractTestSegSplitter.RejectSplitsAfterKey	Accepts the recommended separator key unless it is GTE the key given to the constructor, in which case it refuses to accept any splits.
AsynchronousOverflowTask	This class examines the named indices defined on the journal identified by the lastCommitTime and, for each named index registered on that journal, determines which of the following conditions applies and then schedules any necessary tasks based on that decision: Build a new IndexSegment from the writes buffered on the prior journal.
BuildResult	The result of an CompactingMergeTask.
CompactingMergeTask	Task builds an IndexSegment from the fused view of an index partition as of some historical timestamp and then atomically updates the view (aka a compacting merge).
CompactingMergeTask.AtomicUpdateCompactingMergeTask	The source view is pre-overflow (the last writes are on the old journal) while the current view is post-overflow (reflects writes made since overflow).
IncrementalBuildTask	Task builds an IndexSegment from the mutable BTree and zero or more additional sources in the index partition view and then atomically updates the view (aka an incremental build).
IncrementalBuildTask.AtomicUpdateIncrementalBuildTask	The source is an IndexSegment that was built from the mutable BTree associated with the lastCommitTime on old journal of some index partition.
IndexCache<H extends ILocalBTreeView>	Extends the ConcurrentWeakValueCache to track the earliest timestamp from which any local IIndex view is reading.
IndexManager	Class encapsulates logic and handshaking for tracking which indices (and their backing stores) are recently and currently referenced.
IndexManager.IndexSegmentStats	Statistics about the IndexSegments open in the cache.
IndexSegmentIndex	BTree mapping IndexSegmentStore createTimes to IResourceMetadata records.
JoinIndexPartitionTask	Task joins one or more index partitions and should be invoked when their is strong evidence that the index partitions have shrunk enough to warrant their being combined into a single index partition.
JoinIndexPartitionTask.AtomicUpdateJoinIndexPartition	Task performs an atomic update of the index partition view definitions on the live journal and the MetadataIndex, thereby putting into effect the changes made by a JoinIndexPartitionTask.
JoinResult	The result of a JoinIndexPartitionTask.
JournalIndex	BTree mapping IJournal createTimes (long integers) to JournalMetadata records describing the IJournal.
JournalIndex.TupleSerializer	Encapsulates key and value formation.
MoveResult	The object returned by MoveIndexPartitionTask.
MoveTask	Task moves an index partition to another IDataService.
MoveTask.AtomicUpdate	Moves an index partition from this data service to another data service.
MoveTask.ReceiveIndexPartitionTask	Receives an index partition comprised of a historical index segment store and an index segment store containing the buffered writes and registers the index partition on the data service on which this procedure is executed.
OverflowCounters	Utility class exposes some counter values while protecting the actual counters on the service from modification.
OverflowManager	Class encapsulates logic for handling journal overflow events.
OverflowManager.ResourceScores	Helper class reports performance counters of interest for this service.
OverflowMetadata	Metadata on the entire synchronous and asynchronous overflow task.
PurgeResult	A class that captures the results of StoreManager.purgeOldResources().
ResourceEvents	Class encapsulates reporting API for resource (index and store files) events.
ResourceFileFilter	The default implementation accepts directories under the configured IResourceManager.getDataDir() and files with either Options.JNL or Options.SEG file extensions.
ResourceManager	The ResourceManager has broad responsibility for journal files, index segment files, maintaining index views during overflow processing, and managing the transparent decomposition of scale-out indices and the distribution of the key-range index partitions for those scale-out indices.
ScatterSplitTask	Task splits an index partition into N equal sized index partitions and scatters those index partitions across data services in the federation.
SplitIndexPartitionTask	Task splits an index partition which is a compact view (no more than one journal and one index segment) and should be invoked when the size of the index segment on the disk exceeds the nominal size of an index partition.
SplitIndexPartitionTask.AtomicUpdateSplitIndexPartitionTask	An ITx.UNISOLATED operation that splits the live index using the same Split points, generating new index partitions with new partition identifiers.
SplitResult	The result of a SplitIndexPartitionTask including enough metadata to identify the index partitions to be created and the index partition to be deleted.
SplitTailTask	Splits the tail of an index partition and optionally submits a task to move the tail to a target data service specified by the caller.
SplitUtility	Utility methods for ISimpleSplitHandlers and friends.
SplitUtility.BuildIndexSegmentSplitTask	Task used to build an IndexSegment from a restricted key-range of an index during a SplitIndexPartitionTask.
StoreManager	Class encapsulates logic for managing the store files (journals and index segments), including the logic to compute the effective release time for the managed resources and to release those resources by deleting them from the file system.
TestAddDeleteResource	Unit test of the StoreManager when adding and deleting journals and index segments.
TestAll	Runs all tests for all journal implementations.
TestBuildTask	Basic test of building an index segment from an index partition on overflow.
TestBuildTask2	Basic test of building an index segment from an index partition on overflow.
TestFixedLengthPrefixShardSplits	Unit tests for splitting an index segment based on its size on the disk, the nominal size of an index partition, and an optional application level constraint on the choice of the separator keys.
TestMergeTask	Basic test of compacting merge for an index partition on overflow.
TestOverflow	Basic tests for ResourceManager#overflow(boolean, boolean)
TestReleaseResources	Test release (aka purge) of old resources.
TestReleaseResources.TestWithCopy_NonZeroMinReleaseAge	Test where the indices are copied during synchronous overflow processing and where a non-zero [minReleaseAge] was specified.
TestReleaseResources.TestWithCopyImmediateRelease	Test where the index view is copied in its entirety onto the new journal and the [minReleaseAge] is ZERO(0).
TestReleaseResources.TestWithCopyNoRelease	Test where the index view is copied in its entirety onto the new journal but the ResourceManager is not permitted to release old resources (it is configured as an immortal store).
TestResourceManagerBootstrap	Bootstrap test suite for the ResourceManager.
TestResourceManagerBootstrap.MyResourceManager
TestSegSplitter	Unit tests for splitting an index segment based on its size on the disk, the nominal size of an index partition, and an optional application level constraint on the choice of the separator keys.
TestSparseRowStoreSplitHandler	Tetst suite for LogicalRowSplitHandler.

Enum Summary

Enum	Description
OverflowActionEnum	The different actions that we can take to handle an index partition during overflow processing.
OverflowSubtaskEnum	Various kinds of subtasks for asynchronous index partition overflow tasks.
StaleLocatorReason	Typesafe enumeration of the possible reasons why an index partition locator is stale.

Exception Summary

Exception	Description
NoSuchStoreException	An instance of this class is thrown when a UUID does not identify any store known to the StoreManager.
StaleLocatorException	An instance of this class is thrown when an index partition has been split, joined, or moved to indicate that the client has a stale PartitionLocator and should refresh the locator for the key range covered by the named index partition and retry their request on the appropriate DataService.

Package com.bigdata.resources Description

This package provides the logic to managed the live journal and the historical journals and index segments for a DataService.

Overflow actions (Split, Join, Move, Build, Merge, etc.)

The critical distinction among the actions is whether or not they change the index partition identifier, in which case they define a new index partition. OverflowActionEnum#Copy, OverflowActionEnum#Build and OverflowActionEnum#Merge are "actions" which DO NOT change the index partition identifier.

For actions which define a new index partition, the critical distinction is whether the action is 1:N (OverflowActionEnum#Split) or N:1 (OverflowActionEnum#Join). Note that both OverflowActionEnum#Split and OverflowActionEnum#Join can conceptually locate the new index partition(s) on different data service(s). OverflowActionEnum#Move DOES change the index partition identifier and is in effect a 1:1 action. Since the locator for an index partition MUST NOT change, MOVE serves to "rename" an index partition, and in the process relocates it on another data service.

Neither OverflowActionEnum#Split nor OverflowActionEnum#Join is restricted to operate on only the local index partitions. The critical constaint when writing these operations is that they MUST hold exclusive write locks on the source index partition(s) such that the change in the index partitions can be made atomic. However, the manner in which the operation proceeds does differ when some of the output partition(s) will be located on a different data service. It is also more complex when some of the input partition(s) will be located on a different data service as the write lock must be coordinated across two or more data services.