com.bigdata.resources

Class IncrementalBuildTask

java.lang.Object

com.bigdata.journal.AbstractTask<T>

com.bigdata.resources.AbstractResourceManagerTask<T>

com.bigdata.resources.AbstractPrepareTask<BuildResult>

com.bigdata.resources.IncrementalBuildTask

All Implemented Interfaces:

ITask<BuildResult>, Callable<BuildResult>

public class IncrementalBuildTask
extends AbstractPrepareTask<BuildResult>

Task builds an IndexSegment from the mutable BTree and zero or more additional sources in the index partition view and then atomically updates the view (aka an incremental build).

Build uses mutable BTree of the lastCommitTime for the old journal PLUS ZERO OR MORE additional source(s) taken in view order up to but not including the source in the view with significant content. This let's us keep the #of IndexSegments in the view down without incurring the cost of a compacting merge. (The cost of the compacting merge itself comes from having a large index segment in the view, generally in the last position of the view.) In turn, this keeps the cost of overflow down and can be a significant win if there are a number of large index partitions that receive a few writes in each overflow.

For example, assuming a large index segment exists from a previous compacting merge, then once the #of writes exceeds the "copy" threshold there will be an index build. The view will then have [live, smallSeg1, largeSeg1]. The next time the copy threshold is exceeded we would get [live, smallSeg2, smallSeg1, largeSeg1]. However if we include smallSeg1 in the build, then we get [live, smallSeg2, largeSeg1]. This can continue until we have enough data to warrant a split or until we have another "large" segment but not yet enough data to split, at which point we get [live, largeSeg2, largeSeg1] and then [live, smallSeg3, largeSeg2, largeSeg1].

Note: As its last action, this task submits a IncrementalBuildTask.AtomicUpdateIncrementalBuildTask which replaces the view with one defined by the current BTree on the journal and the newly built IndexSegment.

Note: If the task fails, then the output IndexSegment will be deleted.

Version:

$Id$

Author:

Bryan Thompson

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	IncrementalBuildTask.AtomicUpdateIncrementalBuildTask The source is an IndexSegment that was built from the mutable BTree associated with the lastCommitTime on old journal of some index partition.

Nested classes/interfaces inherited from class com.bigdata.journal.AbstractTask

AbstractTask.DelegateTask<T>, AbstractTask.InnerReadWriteTxServiceCallable<T>, AbstractTask.InnerWriteServiceCallable<T>, AbstractTask.ResubmitException

Field Summary

Fields inherited from class com.bigdata.resources.AbstractResourceManagerTask

DEBUG, INFO, log, resourceManager

Fields inherited from class com.bigdata.journal.AbstractTask

checkpointNanoTime, concurrencyManager, isReadWriteTx, nanoTime_assignedWorker, nanoTime_beginWork, nanoTime_finishedWork, nanoTime_submitTask, readOnly, taskCounters, timestamp, transactionManager, tx

Constructor Summary

Constructors

Constructor and Description
IncrementalBuildTask(com.bigdata.resources.ViewMetadata vmd)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	clearRefs() Method is responsible for clearing the SoftReferences held by ViewMetadata for the source view(s) on the old journal.
protected BuildResult	doTask() Build an IndexSegment from one or more sources for an index partition view.

Methods inherited from class com.bigdata.resources.AbstractResourceManagerTask

toString

Methods inherited from class com.bigdata.journal.AbstractTask

assertResource, assertRunning, assertUnisolated, call, clearLoggingContext, dropIndex, getCommitTime, getIndex, getJournal, getOnlyResource, getResource, getResourceManager, getTaskCounters, getTaskName, getTimestamp, isResource, registerIndex, setupLoggingContext, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

IncrementalBuildTask

public IncrementalBuildTask(com.bigdata.resources.ViewMetadata vmd)

Parameters:

vmd - Metadata about the index partition view.

Method Detail

clearRefs

protected void clearRefs()

Description copied from class: AbstractPrepareTask

Method is responsible for clearing the SoftReferences held by ViewMetadata for the source view(s) on the old journal.

Note: This method MUST be invoked in order to permit those references to be cleared more eagerly than the end of the entire asynchronous overflow operation (which is when the task references would themselves go out of scope and become available for GC).

Specified by:

clearRefs in class AbstractPrepareTask<BuildResult>

doTask

protected BuildResult doTask()
                      throws Exception

Build an IndexSegment from one or more sources for an index partition view. The sources are chosen in view order. New sources are incorporated until too much work would be performed for the lightweight semantics of "build". If all sources are incorporated by the build, then the result is identical a compacting merge.

Specified by:

doTask in class AbstractTask<BuildResult>

Returns:

The BuildResult.

Throws:

Exception - The exception that will be thrown by AbstractTask.call() iff the operation fails.

InterruptedException - This exception SHOULD be thrown if Thread.interrupted() becomes true during execution.