public class MoveTask extends AbstractPrepareTask<MoveResult>
IDataService.
This task runs as a historical read operation and copy the view of the index
partition as of the lastCommitTime of old journal to another
IDataService. Once that historical view has been copied, this task
then submits an AtomicUpdateMoveIndexPartitionTask. The atomic
update is an ITx.UNISOLATED operation. It is responsible copying any
writes buffered for the index partition on the live journal to the target
IDataService and then updating the MetadataIndex. Once the
atomic update task is finished, clients will discover that the source index
partition does not exist. When they query the MetadataService they
will discover that the key(-range) is now handled by the new index partition
on the target IDataService.
Note: This task is run on the target IDataService and it copies the
data from the source IDataService. This allows us to use standard
IRangeQuery operations to copy the historical view. However, the
AtomicUpdateMoveIndexPartitionTask is run on the source
IDataService since it needs to obtain an exclusive lock on the index
partition that is being moved in order to prevent concurrent writes during
the atomic cutover. For the same reason, the
AtomicUpdateMoveIndexPartitionTask can not use standard
IRangeQuery operations. Instead, it initiates a series of data
transfers while holding onto the exclusive lock until the target
IDataService has the current state of the index partition. At that
point it notifies the IMetadataService to perform the atomic cutover
to the new index partition.
Note: This task does NOT cause any resources associated with the current view
of the index partition to be released on the source IDataService.
The reason is two-fold. First, the IndexSegment(s) associated with
that view MAY be in used by historical views. Second, there MAY be historical
commit points for the index partition on the live journal before the atomic
cutover to the new IDataService - those historical commit points MUST
be preserved until the release policy for those views has been satisfied.
Note: The MOVE task MUST be explicitly coordinated with the target
IDataService. Failure to coordinate the move results in an error
message reported by the MetadataService indicating that the wrong
partition locator was found under the key. The cause is a MOVE operation
during which the target data service undergoes concurrent synchronous (and
then asynchronous) overflow. What happens is the MoveTask registers
the new index partition on the target data service. One registered on the
IDataService, the index partition it is visible during synchronous
overflow BEFORE the MOVE is complete and BEFORE the index is registered with
the MetadataService and hence discoverable to clients. If the target
IDataService then undergoes synchronous and asynchronous overflow and
chooses an action which would change the index partition definition (split,
join, or move) WHILE the index partition is still being moved onto the target
IDataService THEN the MOVE is not atomic and the definition of the
index partition in the MetadataService will not coherently reflect
either the MOVE or the action chosen by the target IDataService,
depending on which one makes its atomic update first.
The target IDataService MAY undergo both synchronous and asynchronous
overflow as IDataServices are designed to allow continued writes
during those operations. Further, it MAY choose to copy, build, or compact
the index partition while it is being moved. However, it MUST NOT choose any
action (split, join, or move) that would change the index partition
definition until the move is complete (whether it ends in success or
failure).
This issue is addressed by the following protocol:
MoveTask set the sourcePartitionId on the
LocalPartitionMetadata when it registers the index partition on the
target IDataService. When sourcePartitionId != -1.
the target IDataService is restricted to for that index partition to
overflows actions which do not change the index partition definition (copy,
build, or merge). Further, any index partition found on restart whose by the
target IDataService whose sourcePartitionId != -1 is
deleted as it was never successfully put into play (this prevents partial
moves from accumulating state which could not otherwise be released.)sourcePartitionId to be
set to -1 as one of its last actions, thereby allowing the
target IDataService to use operations that could re-define the index
partition (split, join, move) and also preventing the target index partition
from being deleted on restart. Note: There are only two entry points: a simple move and a move where the compacting merge has already been performed, e.g., by a split, and we just need to do the atomic update phase.
| Modifier and Type | Class and Description |
|---|---|
protected static class |
MoveTask.AtomicUpdate
Moves an index partition from this data service to another data service.
|
protected static class |
MoveTask.ReceiveIndexPartitionTask
Receives an index partition comprised of a historical index segment store
and an index segment store containing the buffered writes and registers
the index partition on the data service on which this procedure is
executed.
|
AbstractTask.DelegateTask<T>, AbstractTask.InnerReadWriteTxServiceCallable<T>, AbstractTask.InnerWriteServiceCallable<T>, AbstractTask.ResubmitExceptionDEBUG, INFO, log, resourceManagercheckpointNanoTime, concurrencyManager, isReadWriteTx, nanoTime_assignedWorker, nanoTime_beginWork, nanoTime_finishedWork, nanoTime_submitTask, readOnly, taskCounters, timestamp, transactionManager, tx| Constructor and Description |
|---|
MoveTask(com.bigdata.resources.ViewMetadata vmd,
UUID targetDataServiceUUID) |
| Modifier and Type | Method and Description |
|---|---|
protected void |
clearRefs()
Method is responsible for clearing the
SoftReferences held by
ViewMetadata for the source view(s) on the old journal. |
protected static MoveResult |
doAtomicUpdate(ResourceManager resourceManager,
String sourceIndexName,
BuildResult historicalWritesBuildResult,
UUID targetDataServiceUUID,
int targetIndexPartitionId,
Event parentEvent)
Submits an
MoveTask.AtomicUpdate and awaits and returns its outcome. |
protected MoveResult |
doTask()
Builds a compact index segment from the historical view as of the last
commit time on the old journal and then submits an atomic update
operation to move the source index partition to the target data service.
|
toStringassertResource, assertRunning, assertUnisolated, call, clearLoggingContext, dropIndex, getCommitTime, getIndex, getJournal, getOnlyResource, getResource, getResourceManager, getTaskCounters, getTaskName, getTimestamp, isResource, registerIndex, setupLoggingContext, toStringpublic MoveTask(com.bigdata.resources.ViewMetadata vmd,
UUID targetDataServiceUUID)
vmd - Metadata for the source index partition view.targetDataServiceUUID - The UUID for the target data service.protected void clearRefs()
AbstractPrepareTaskSoftReferences held by
ViewMetadata for the source view(s) on the old journal.
Note: This method MUST be invoked in order to permit those references to be cleared more eagerly than the end of the entire asynchronous overflow operation (which is when the task references would themselves go out of scope and become available for GC).
clearRefs in class AbstractPrepareTask<MoveResult>protected MoveResult doTask() throws Exception
doTask in class AbstractTask<MoveResult>MoveResult describing the move operation (this is
returned mainly for historical reasons).Exception - The exception that will be thrown by AbstractTask.call() iff the
operation fails.InterruptedException - This exception SHOULD be thrown if
Thread.interrupted() becomes true during
execution.protected static MoveResult doAtomicUpdate(ResourceManager resourceManager, String sourceIndexName, BuildResult historicalWritesBuildResult, UUID targetDataServiceUUID, int targetIndexPartitionId, Event parentEvent) throws InterruptedException, ExecutionException
MoveTask.AtomicUpdate and awaits and returns its outcome.resourceManager - The resource manager.sourceIndexName - The name of the source index partition.historicalWritesBuildResult - An index segment containing all data for the source view as of
the last commit time on the old journal. This index segment
should be generated by a compacting merge or by an index
partition split with the same semantics so that we will move
the minimum amount of data.targetDataServiceUUID - The UUID of the target data service.targetIndexPartitionId - The partition identifier assigned to the target index
partition.parentEvent - ExecutionExceptionInterruptedExceptionCopyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.