public class MoveTask extends AbstractPrepareTask<MoveResult>
This task runs as a historical read operation and copy the view of the index
partition as of the lastCommitTime of old journal to another
IDataService. Once that historical view has been copied, this task
then submits an
AtomicUpdateMoveIndexPartitionTask. The atomic
update is an
ITx.UNISOLATED operation. It is responsible copying any
writes buffered for the index partition on the live journal to the target
IDataService and then updating the
MetadataIndex. Once the
atomic update task is finished, clients will discover that the source index
partition does not exist. When they query the
will discover that the key(-range) is now handled by the new index partition
on the target
Note: This task is run on the target
IDataService and it copies the
data from the source
IDataService. This allows us to use standard
IRangeQuery operations to copy the historical view. However, the
AtomicUpdateMoveIndexPartitionTask is run on the source
IDataService since it needs to obtain an exclusive lock on the index
partition that is being moved in order to prevent concurrent writes during
the atomic cutover. For the same reason, the
AtomicUpdateMoveIndexPartitionTask can not use standard
IRangeQuery operations. Instead, it initiates a series of data
transfers while holding onto the exclusive lock until the target
IDataService has the current state of the index partition. At that
point it notifies the
IMetadataService to perform the atomic cutover
to the new index partition.
Note: This task does NOT cause any resources associated with the current view
of the index partition to be released on the source
The reason is two-fold. First, the
IndexSegment(s) associated with
that view MAY be in used by historical views. Second, there MAY be historical
commit points for the index partition on the live journal before the atomic
cutover to the new
IDataService - those historical commit points MUST
be preserved until the release policy for those views has been satisfied.
Note: The MOVE task MUST be explicitly coordinated with the target
IDataService. Failure to coordinate the move results in an error
message reported by the
MetadataService indicating that the wrong
partition locator was found under the key. The cause is a MOVE operation
during which the target data service undergoes concurrent synchronous (and
then asynchronous) overflow. What happens is the
the new index partition on the target data service. One registered on the
IDataService, the index partition it is visible during synchronous
overflow BEFORE the MOVE is complete and BEFORE the index is registered with
MetadataService and hence discoverable to clients. If the target
IDataService then undergoes synchronous and asynchronous overflow and
chooses an action which would change the index partition definition (split,
join, or move) WHILE the index partition is still being moved onto the target
IDataService THEN the MOVE is not atomic and the definition of the
index partition in the
MetadataService will not coherently reflect
either the MOVE or the action chosen by the target
depending on which one makes its atomic update first.
IDataService MAY undergo both synchronous and asynchronous
IDataServices are designed to allow continued writes
during those operations. Further, it MAY choose to copy, build, or compact
the index partition while it is being moved. However, it MUST NOT choose any
action (split, join, or move) that would change the index partition
definition until the move is complete (whether it ends in success or
This issue is addressed by the following protocol:
LocalPartitionMetadatawhen it registers the index partition on the target
sourcePartitionId != -1. the target
IDataServiceis restricted to for that index partition to overflows actions which do not change the index partition definition (copy, build, or merge). Further, any index partition found on restart whose by the target
sourcePartitionId != -1is deleted as it was never successfully put into play (this prevents partial moves from accumulating state which could not otherwise be released.)
sourcePartitionIdto be set to
-1as one of its last actions, thereby allowing the target
IDataServiceto use operations that could re-define the index partition (split, join, move) and also preventing the target index partition from being deleted on restart.
Note: There are only two entry points: a simple move and a move where the compacting merge has already been performed, e.g., by a split, and we just need to do the atomic update phase.
|Modifier and Type||Class and Description|
Moves an index partition from this data service to another data service.
Receives an index partition comprised of a historical index segment store and an index segment store containing the buffered writes and registers the index partition on the data service on which this procedure is executed.
DEBUG, INFO, log, resourceManager
|Constructor and Description|
|Modifier and Type||Method and Description|
Method is responsible for clearing the
Builds a compact index segment from the historical view as of the last commit time on the old journal and then submits an atomic update operation to move the source index partition to the target data service.
assertResource, assertRunning, assertUnisolated, call, clearLoggingContext, dropIndex, getCommitTime, getIndex, getJournal, getOnlyResource, getResource, getResourceManager, getTaskCounters, getTaskName, getTimestamp, isResource, registerIndex, setupLoggingContext, toString
public MoveTask(com.bigdata.resources.ViewMetadata vmd, UUID targetDataServiceUUID)
vmd- Metadata for the source index partition view.
targetDataServiceUUID- The UUID for the target data service.
protected void clearRefs()
SoftReferences held by
ViewMetadatafor the source view(s) on the old journal.
Note: This method MUST be invoked in order to permit those references to be cleared more eagerly than the end of the entire asynchronous overflow operation (which is when the task references would themselves go out of scope and become available for GC).
protected MoveResult doTask() throws Exception
MoveResultdescribing the move operation (this is returned mainly for historical reasons).
Exception- The exception that will be thrown by
AbstractTask.call()iff the operation fails.
InterruptedException- This exception SHOULD be thrown if
Thread.interrupted()becomes true during execution.
protected static MoveResult doAtomicUpdate(ResourceManager resourceManager, String sourceIndexName, BuildResult historicalWritesBuildResult, UUID targetDataServiceUUID, int targetIndexPartitionId, Event parentEvent) throws InterruptedException, ExecutionException
MoveTask.AtomicUpdateand awaits and returns its outcome.
resourceManager- The resource manager.
sourceIndexName- The name of the source index partition.
historicalWritesBuildResult- An index segment containing all data for the source view as of the last commit time on the old journal. This index segment should be generated by a compacting merge or by an index partition split with the same semantics so that we will move the minimum amount of data.
UUIDof the target data service.
targetIndexPartitionId- The partition identifier assigned to the target index partition.
Copyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.