Package com.bigdata.service.ndx.pipeline

Interface Summary

Interface	Description
IDuplicateRemover<O>	Interface allowing application specific elimination of duplicates.
IMasterTask<E,H>	Interface for task consuming data written on an application on an asynchronous write buffer.
KVOList.Op<O>	An operation which can be mapped over the duplicate list.
SubtaskOp<S extends AbstractSubtask>	An operation which can be mapped across subtasks.

Class Summary

Class	Description
AbstractKeyRangeMasterTestCase	Abstract base class for test suites for the AbstractMasterTask and friends using IPartitionMetadata locators.
AbstractMasterStats<L,HS extends AbstractSubtaskStats>	Abstract base class providing statistics for the AbstractMasterTask and a factory for the statistics for the subtasks.
AbstractMasterTask<H extends AbstractMasterStats<L,? extends AbstractSubtaskStats>,E,S extends AbstractSubtask,L>	Abstract base class for a master task which consumes chunks of elements written onto a BlockingBuffer and distributes those chunks to subtasks according to some abstraction which is not defined by this class.
AbstractMasterTestCase	Abstract base class for test suites for the AbstractMasterTask and friends.
AbstractPendingSetMasterStats<L,HS extends AbstractSubtaskStats>	Extended to report the moving average of the pending set size for the master and the sinks and to report the maximum pending set size for the sinks.
AbstractPendingSetMasterTask<H extends AbstractPendingSetMasterStats<L,? extends AbstractSubtaskStats>,E,S extends AbstractPendingSetSubtask,L>	Extends the master task to track outstanding asynchronous operations on work items.
AbstractPendingSetSubtask<HS extends AbstractSubtaskStats,M extends AbstractPendingSetMasterTask<? extends AbstractPendingSetMasterStats<L,HS>,E,? extends AbstractPendingSetSubtask,L>,E,L>	Extended to assign chunks of work items to a remote IAsynchronousClientTask, to track the set of outstanding asynchronous operations for a specific client task (the "pending set"), and to close the client task when the sink not assign any more work to that client.
AbstractRunnableMasterStats<L,HS extends AbstractSubtaskStats>	Statistics for the consumer, including several moving averages based on sampled data.
AbstractSubtask<HS extends AbstractSubtaskStats,M extends AbstractMasterTask<? extends AbstractMasterStats<L,HS>,E,? extends AbstractSubtask,L>,E,L>	Abstract implementation of a subtask for the AbstractMasterTask handles the protocol for startup and termination of the subtask.
AbstractSubtaskStats
DefaultDuplicateRemover<O>	Implementation which retains one instance of each tuple having the same unsigned byte[] key and the same byte[] value.
IndexAsyncWriteStats<L,HS extends IndexPartitionWriteStats>	Statistics for asynchronous index writes.
IndexPartitionWriteStats	Statistics reported for each index partition.
IndexPartitionWriteTask<H extends IndexAsyncWriteStats<L,HS>,O,E extends KVO<O>,L extends PartitionLocator,S extends IndexPartitionWriteTask,HS extends IndexPartitionWriteStats,M extends IndexWriteTask<H,O,E,S,L,HS,T,R,A>,T extends IKeyArrayIndexProcedure,R,A>	Class drains a BlockingBuffer writing on a specific index partition.
IndexWriteTask<H extends IndexAsyncWriteStats<L,HS>,O,E extends KVO<O>,S extends IndexPartitionWriteTask,L extends PartitionLocator,HS extends IndexPartitionWriteStats,T extends IKeyArrayIndexProcedure,R,A>	Task drains a BlockingBuffer containing KVO[] chunks, splits the chunks based on the separator keys for the scale-out index, and then assigns each chunk to per-index partition BlockingBuffer which is in turned drained by an IndexPartitionWriteTask that writes onto a specific index partition.
IndexWriteTask.M<T extends IKeyArrayIndexProcedure,O,R,A>	Concrete master hides most of the generic types leaving you with only those that are meaningfully parameterize for applications using the streaming write API.
KVOC<O>	Extends KVOList to provide handshaking with a KVOLatch.
KVOLatch	A synchronization aid that allows one or more threads to await asynchronous writes on one or more scale-out indices.
KVOList<O>	Extends KVO to allow duplicates to be gathered together in a doubly-linked list.
MockMaster<H extends MockMasterStats<L,HS>,O,E extends KVO<O>,S extends MockSubtask,L,HS extends MockSubtaskStats>	Class exists solely to make it easier to write the unit tests by aligning the various generic types across the master, the subtask, and their statistics objects.
MockMasterStats<L,HS extends MockSubtaskStats>	Class exists solely to make it easier to write the unit tests by aligning the various generic types across the master, the subtask, and their statistics objects.
MockSubtask<H extends MockMasterStats<L,HS>,O,E extends KVO<O>,L,S extends MockSubtask,HS extends MockSubtaskStats,M extends MockMaster<H,O,E,S,L,HS>>	Class exists solely to make it easier to write the unit tests by aligning the various generic types across the master, the subtask, and their statistics objects.
MockSubtaskStats	Class exists solely to make it easier to write the unit tests by aligning the various generic types across the master, the subtask, and their statistics objects.
NOPDuplicateRemover<O>	NOP implementation.
TestAll	Aggregates test suites in increasing dependency order.
TestDefaultDuplicateRemover	Test suite for DefaultDuplicateRemover.
TestFileSystemScanner
TestMasterTask	Unit tests for the control logic used by AbstractMasterTask and friends.
TestMasterTaskIdleTimeout	Unit tests of the idle timeout behavior for AbstractMasterTask and friends.
TestMasterTaskWithErrors	Unit tests for error handling in the control logic used by AbstractMasterTask and friends.
TestMasterTaskWithRedirect	Test ability to handle a redirect (subtask learns that the target service no longer accepts data for some locator and instead must send the data somewhere else).
TestMasterTaskWithSplits	Stress test using key-range partitioned index (IMetadataIndex), which allows us to test the AbstractMasterTask under split, move, join and other kinds of index partition operations.

Exception Summary

Exception	Description
IdleTimeoutException	An instance of this exception is thrown when a sink closes it's input queue due to an idle timeout (nothing has been written on its input queue for a period of timeout which exceeds its idle timeout).
MasterExhaustedException	An instance of this exception is thrown when a sink closes it's input queue due to the master being exhausted.