com.bigdata.journal

Class AbstractTask<T>

java.lang.Object

com.bigdata.journal.AbstractTask<T>

All Implemented Interfaces:

ITask<T>, Callable<T>

Direct Known Subclasses:

AbstractResourceManagerTask, ApiTaskForJournal, DataService.GetIndexMetadataTask, DataService.RangeIteratorTask, DropIndexTask, IndexProcedureTask, JournalTransactionService.SinglePhaseCommit, JournalTransactionService.ValidateWriteSetTask, MetadataService.DropScaleOutIndexTask, MetadataService.JoinIndexPartitionTask, MetadataService.MoveIndexPartitionTask, MetadataService.NextPartitionIdTask, MetadataService.RegisterScaleOutIndexTask, MetadataService.SplitIndexPartitionTask, RegisterIndexTask, StressTestConcurrentUnisolatedIndices.WriteTask

public abstract class AbstractTask<T>
extends Object
implements Callable<T>, ITask<T>

Abstract base class for tasks that may be submitted to the ConcurrencyManager. Tasks may be isolated (by a transaction), unisolated, read-committed, or historical reads. Tasks access named resources (aka indices), which they pre-declare in their constructors.

A read-committed task runs against the most recently committed view of the named index. A historical read task runs against a historical view of the named index, but without guarantees of transactional isolation. Concurrent readers are permitted without locking on the same index.

An unisolated task reads and writes on the "live" index. Note that only a single thread may write on a BTree at a time. Therefore unisolated tasks (often referred to as writers) obtain an exclusive lock on the named index(s). When more than one named index is used, the locks are used to infer a partial ordering of the writers allowing as much concurrency as possible. Pre-declaration of locks allows us to avoid deadlocks in the lock system.

Isolated tasks are part of a larger transaction. Transactions are started and committed using an ITransactionManagerService. Transactional tasks run with full concurrency using an MVCC (Multi-Version Concurrency Control) strategy. When a transaction is committed (by the ITransactionManagerService) it must wait for lock(s) on the unisolated named indices on which it has written before it may validate and commit.

Note: You MUST submit a distinct instance of this task each time you ConcurrencyManager.submit(AbstractTask) it.

Author:

Bryan Thompson

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	AbstractTask.DelegateTask<T> Delegates various behaviors visible to the application code using the ITask interface to the AbstractTask object.
protected static class	AbstractTask.InnerReadWriteTxServiceCallable<T> Inner class used to wrap up the call to doTask() for read-write transactions.
protected static class	AbstractTask.InnerWriteServiceCallable<T> An instance of this class is used as the delegate to coordinate the acquisition of locks with the NonBlockingLockManager before the task can execute and to release locks after the task has completed (whether it succeeds or fails).
static class	AbstractTask.ResubmitException This is thrown if you attempt to reuse (re-submit) the same AbstractTask instance.

Field Summary

Fields

Modifier and Type	Field and Description
long	checkpointNanoTime The elapsed time in nanoseconds for a write task to checkpoint its index(s).
protected ConcurrencyManager	concurrencyManager The object used to manage exclusive access to the unisolated indices.
protected boolean	isReadWriteTx True iff the operation is isolated by a transaction.
protected static org.apache.log4j.Logger	log
long	nanoTime_assignedWorker The time at which this task was assigned to a worker thread for execution.
long	nanoTime_beginWork The time at which this task began to do its work.
long	nanoTime_finishedWork The time at which this task finished its work.
long	nanoTime_submitTask The time at which this task was submitted to the ConcurrencyManager.
protected boolean	readOnly True iff the operation is not permitted to write.
protected IResourceManager	resourceManager The object used to manage access to the resources from which views of the indices are created.
protected TaskCounters	taskCounters The AbstractTask increments various counters of interest to the ConcurrencyManager using this object.
protected long	timestamp The transaction identifier -or- ITx.UNISOLATED if the operation is NOT isolated by a transaction, -or- ITx.READ_COMMITTED, -or- timestamp to read from the most recent commit point not later than timestamp.
protected AbstractLocalTransactionManager	transactionManager The object used to manage local transactions.
protected Tx	tx The transaction object iff the operation is isolated by a transaction and otherwise null.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractTask(IConcurrencyManager concurrencyManager, long timestamp, String resource) Convenience constructor variant for one named resource.
protected	AbstractTask(IConcurrencyManager concurrencyManager, long timestamp, String[] resource)

Method Summary

Methods

Modifier and Type	Method and Description
protected String	assertResource(String resource) Asserts that the resource is one of the resource(s) declared to the constructor.
protected void	assertRunning() Assert that the task is still running (aborted is false).
protected void	assertUnisolated() Assert that the task is ITx.UNISOLATED.
T	call() Delegates the task behavior to doTask().
protected void	clearLoggingContext() Clear fields set by setupLoggingContext() from the MDC logging context.
protected abstract T	doTask() Implement the task behavior here.
void	dropIndex(String name) Drops the named index.
long	getCommitTime() The timestamp of the group commit for an ITx.UNISOLATED task which executes successfully and then iff the group commit succeeds.
ILocalBTreeView	getIndex(String name) Return an appropriate view of the named B+Tree that has the appropriate isolation level for the operation (non-GIST).
IJournal	getJournal() The journal against which the operation will be carried out.
String	getOnlyResource() Return the only declared resource.
String[]	getResource() Returns a copy of the array of resources declared to the constructor.
IResourceManager	getResourceManager() The object used to manage access to the resources from which views of the indices are created.
TaskCounters	getTaskCounters() The object used to track events and times for the task.
protected String	getTaskName() Returns the name of the class by default.
long	getTimestamp() The timestamp specified to the ctor.
boolean	isResource(String theRequestedResource) Return true iff the task declared this as a resource.
IIndex	registerIndex(String name, BTree btree) Registers an index
protected void	setupLoggingContext() Adds the following fields to the MDC logging context: taskname The name of the task as reported by getTaskName(). timestamp The timestamp specified to the ctor. resources The named resource(s) specified to the ctor IFF logging @ INFO or above.
String	toString() Returns Task{taskName,timestamp,elapsed,resource[]}

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

log

protected static final org.apache.log4j.Logger log

concurrencyManager

protected final ConcurrencyManager concurrencyManager

The object used to manage exclusive access to the unisolated indices.

transactionManager

protected final AbstractLocalTransactionManager transactionManager

The object used to manage local transactions.

resourceManager

protected final IResourceManager resourceManager

The object used to manage access to the resources from which views of the indices are created.

timestamp

protected final long timestamp

The transaction identifier -or- ITx.UNISOLATED if the operation is NOT isolated by a transaction, -or- ITx.READ_COMMITTED, -or- timestamp to read from the most recent commit point not later than timestamp.

isReadWriteTx

protected final boolean isReadWriteTx

True iff the operation is isolated by a transaction.

readOnly

protected final boolean readOnly

True iff the operation is not permitted to write.

tx

protected final Tx tx

The transaction object iff the operation is isolated by a transaction and otherwise null.

taskCounters

protected TaskCounters taskCounters

The AbstractTask increments various counters of interest to the ConcurrencyManager using this object.

nanoTime_submitTask

public long nanoTime_submitTask

The time at which this task was submitted to the ConcurrencyManager.

nanoTime_assignedWorker

public long nanoTime_assignedWorker

The time at which this task was assigned to a worker thread for execution.

nanoTime_beginWork

public long nanoTime_beginWork

The time at which this task began to do its work. If the task needs to acquire exclusive resource locks, then this timestamp is set once those locks have been acquired. Otherwise this timestamp will be very close to the nanoTime_assignedWorker.

nanoTime_finishedWork

public long nanoTime_finishedWork

The time at which this task finished its work. Tasks with write sets must still do abort processing or await the next commit group.

checkpointNanoTime

public long checkpointNanoTime

The elapsed time in nanoseconds for a write task to checkpoint its index(s).

Constructor Detail

AbstractTask

protected AbstractTask(IConcurrencyManager concurrencyManager,
            long timestamp,
            String resource)

Convenience constructor variant for one named resource.

Parameters:

concurrencyControl - The object used to control access to the local resources.

timestamp - The transaction identifier -or- ITx.UNISOLATED IFF the operation is NOT isolated by a transaction -or- - timestamp to read from the most recent commit point not later than the absolute value of timestamp (a historical read).

resource - The resource on which the task will operate. E.g., the names of the index. When the task is an unisolated write task an exclusive lock will be requested on the named resource and the task will NOT run until it has obtained that lock.

The name may identify either a namespace or a concrete index object. If a concrete index object is discovered, only that index is isolated. Otherwise all indices having the same prefix as the namespace are isolated.

AbstractTask

protected AbstractTask(IConcurrencyManager concurrencyManager,
            long timestamp,
            String[] resource)

Parameters:

concurrencyControl - The object used to control access to the local resources.

timestamp - The transaction identifier, ITx.UNISOLATED for an unisolated view, ITx.READ_COMMITTED for a view as of the most recent commit point, or timestamp to read from the most recent commit point not later than that timestamp.

resource - The resource(s) on which the task will operate. E.g., the names of the index(s). When the task is an unisolated write task an exclusive lock will be requested on each named resource and the task will NOT run until it has obtained those lock(s).

The name may identify either a namespace or a concrete index object. If a concrete index object is discovered, only that index is isolated. Otherwise all indices having the same prefix as the namespace are isolated.

Method Detail

getResourceManager

public final IResourceManager getResourceManager()

The object used to manage access to the resources from which views of the indices are created.

Specified by:

getResourceManager in interface ITask<T>

getJournal

public final IJournal getJournal()

Description copied from interface: ITask

The journal against which the operation will be carried out.

If the task is running against an ITx.UNISOLATED index, then this will be the IResourceManager.getLiveJournal(). If the operation is a historical read, then it will be whatever journal is appropriate to the historical commit point against which the task is being run.

Note: For ITx.UNISOLATED operations this exposes unconstrained access to the journal that could be used to violate the concurrency control mechanisms, therefore you SHOULD NOT use this unless you have a clear idea what you are about. You should be able to write all application level tasks in terms of ITask.getIndex(String) and operations on the returned index.

Note: For example, if you use the returned object to access a named index and modify the state of that named index, your changes WILL NOT be noticed by the checkpoint protocol in AbstractTask.InnerWriteServiceCallable.

Specified by:

getJournal in interface ITask<T>

Returns:

The corresponding journal for that timestamp -or- null if no journal has data for that timestamp, including when a historical journal with data for that timestamp has been deleted.

See Also:

IResourceManager.getJournal(long)

getCommitTime

public long getCommitTime()

The timestamp of the group commit for an ITx.UNISOLATED task which executes successfully and then iff the group commit succeeds. Otherwise ZERO (0L).

getIndex

public final ILocalBTreeView getIndex(String name)

Return an appropriate view of the named B+Tree that has the appropriate isolation level for the operation (non-GIST).

When the task is isolated by a transaction, then the index will be isolated by the transaction using the appropriate isolation level. If the transaction is read-only, then the index will not be writable.

When the task is a read-only unisolated operation, the index will be read-only and will read from the most recent committed state of the store prior to the time at which the task began to execute. If multiple index views are requested they will all use the same committed state of the store.

When the task is an unisolated write operation the index will be the unisolated writable (aka "live" or "current" index). Access to the unisolated writable indices is single-threaded. This constraint is enforced by a lock system using the named resources declared in the task constructor.

Note: There are two ways in which a task may access an ITx.UNISOLATED index, but in all cases access to the index is delegated to this method. First, the task can use this method directly. Second, the task can use getJournal() and then use IBTreeManager.getIndex(String) on that journal, which is simply delegated to this method. See IsolatedActionJournal.

Specified by:

getIndex in interface ITask<T>

Parameters:

name - The index name.

Returns:

An appropriate view of the named index.

See Also:

(GIST)

registerIndex

public IIndex registerIndex(String name,
                   BTree btree)

Registers an index

Parameters:

name - The index name.

btree - The BTree that will absorb writes for the index.

Returns:

The index on which writes may be made. Note that if the BTree describes an index partition with multiple sources then the returned object is a FusedView for that index partition as would be returned by IResourceManager.getIndex(String, long).

Throws:

UnsupportedOperationException - unless the task is ITx.UNISOLATED

IndexExistsException - if the index was already registered as of the time that this task began to execute.

See Also:

IBTreeManager.registerIndex(String, BTree)

dropIndex

public void dropIndex(String name)

Drops the named index.

Parameters:

name - The name of the index.

Throws:

IllegalArgumentException - if name is null.

UnsupportedOperationException - unless the task is ITx.UNISOLATED

NoSuchIndexException - if the named index is not registered as of the time that this task began to execute.

See Also:

IGISTManager.dropIndex(String)

getTaskCounters

public TaskCounters getTaskCounters()

Description copied from interface: ITask

The object used to track events and times for the task.

Specified by:

getTaskCounters in interface ITask<T>

getTimestamp

public long getTimestamp()

The timestamp specified to the ctor. This effects which index checkpoints are available to the task and whether the index(s) are read-only or mutable.

getResource

public String[] getResource()

Returns a copy of the array of resources declared to the constructor.

Specified by:

getResource in interface ITask<T>

getOnlyResource

public String getOnlyResource()

Return the only declared resource.

Specified by:

getOnlyResource in interface ITask<T>

Returns:

The declared resource.

Throws:

IllegalStateException - if more than one resource was declared.

isResource

public boolean isResource(String theRequestedResource)

Return true iff the task declared this as a resource.

Parameters:

theRequestedResource - The name of a resource.

Returns:

true iff name is a declared resource.

Throws:

IllegalArgumentException - if name is null.

assertResource

protected String assertResource(String resource)

Asserts that the resource is one of the resource(s) declared to the constructor. This is used to prevent tasks from accessing resources that they did not declare (and on which they may not hold a lock).

Parameters:

resource - A resource name.

Returns:

The resource name.

Throws:

IllegalStateException - if the resource was not declared to the constructor.

assertUnisolated

protected void assertUnisolated()

Assert that the task is ITx.UNISOLATED.

Throws:

UnsupportedOperationException - unless the task is ITx.UNISOLATED

assertRunning

protected void assertRunning()

Assert that the task is still running (aborted is false).

Throws:

RuntimeException - wrapping an InterruptedException if the task has been interrupted.

toString

public String toString()

Returns Task{taskName,timestamp,elapsed,resource[]}

Specified by:

toString in interface ITask<T>

Overrides:

toString in class Object

getTaskName

protected String getTaskName()

Returns the name of the class by default.

doTask

protected abstract T doTask()
                     throws Exception

Implement the task behavior here.

Note: Long-running implementations MUST periodically test Thread.interrupted() and MUST throw an exception, such as InterruptedException, if they are interrupted. This behavior allows tasks to be canceled in a timely manner.

If you ignore or fail to test Thread.interrupted() then your task CAN NOT be aborted. If it is Future.cancel(boolean) with false then the task will run to completion even though it has been cancelled (but the Future will appear to have been cancelled).

If you simply return rather than throwing an exception then the WriteExecutorService will assume that your task completed and your (partial) results will be made restart-safe at the next commit!

Returns:

The object that will be returned by call() iff the operation succeeds.

Throws:

Exception - The exception that will be thrown by call() iff the operation fails.

InterruptedException - This exception SHOULD be thrown if Thread.interrupted() becomes true during execution.

setupLoggingContext

protected void setupLoggingContext()

Adds the following fields to the MDC logging context:

taskname

The name of the task as reported by getTaskName().

timestamp

The timestamp specified to the ctor.

resources

The named resource(s) specified to the ctor IFF logging @ INFO or above.

clearLoggingContext

protected void clearLoggingContext()

Clear fields set by setupLoggingContext() from the MDC logging context.

call

public final T call()
             throws Exception

Delegates the task behavior to doTask().

For an unisolated operation, this method provides safe commit iff the task succeeds and otherwise invokes abort() so that partial task executions are properly discarded. When possible, the original exception is re-thrown so that we do not encapsulate the cause unless it would violate our throws clause.

Commit and abort are NOT invoked for an isolated operation regardless of whether the operation succeeds or fails. It is the responsibility of the "client" to commit or abort a transaction as it sees fit.

Note: Exceptions that are thrown from here will be wrapped as ExecutionExceptions by the ExecutorService. Use InnerCause to test for these exceptions.

Specified by:

call in interface Callable<T>

Throws:

StaleLocatorException - if the task requests an index partition which has been split, joined, or moved to another data service.

NoSuchIndexException - if the task requests an index that is not registered on the data service.

InterruptedException - can be thrown if the task is interrupted, for example while awaiting a lock, if the commit group is being discarded, or if the journal is being shutdown (which will cause the executor service running the task to be shutdown and thereby interrupt all running tasks).

Exception