com.bigdata.service

Class AbstractTransactionService

java.lang.Object

com.bigdata.service.AbstractService

com.bigdata.service.AbstractTransactionService

All Implemented Interfaces:

ICounterSetAccess, ITimestampService, ITransactionService, IService, IServiceShutdown, Remote

Direct Known Subclasses:

AbstractHATransactionService, DistributedTransactionService, TestTransactionService.MockTransactionService

public abstract class AbstractTransactionService
extends AbstractService
implements ITransactionService, IServiceShutdown, ICounterSetAccess

Centralized transaction manager service. In response to a client request, the transaction manager will distribute prepare/commit or abort operations to all data services on which writes were made by a transaction. The transaction manager also provides global timestamps required for non-transactional commit points and various other purposes.

Version:

$Id$

Author:

Bryan Thompson

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	AbstractTransactionService.Options Options understood by this service.
class	AbstractTransactionService.TxState Transaction state as maintained by the ITransactionService.

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	ERR_NO_SUCH If the transaction is not known to this service.
protected static String	ERR_NOT_ACTIVE If a transaction is no longer active.
protected static String	ERR_READ_ONLY If the transaction is read-only and a write operation was requested.
protected static String	ERR_SERVICE_NOT_AVAIL If the transaction service is not in a run state which permits the requested operation.
protected ReentrantLock	lock A lock used to serialize certain operations that must be atomic with respect to the state of the transaction service.
protected static org.apache.log4j.Logger	log Logger.
protected Condition	txDeactivate Signaled by deactivateTx(TxState) and based on lock.

Constructor Summary

Constructors

Constructor and Description
AbstractTransactionService(Properties properties)

Method Summary

Methods

Modifier and Type	Method and Description
void	abort(long tx) Abort the transaction (asynchronous).
void	abortAllTx() Abort all active transactions.
protected abstract void	abortImpl(AbstractTransactionService.TxState state) Implementation must abort the tx on the journal (standalone) or on each data service (federation) on which it has written.
protected void	activateTx(AbstractTransactionService.TxState state) Adds the transaction from to the local tables.
protected void	assertOpen()
protected AbstractTransactionService.TxState	assignTransactionIdentifier(long timestamp) Assign a transaction identifier for a new transaction.
long	commit(long tx) Request commit of the transaction write set.
protected abstract long	commitImpl(AbstractTransactionService.TxState state) Implementation must either single-phase commit (standalone journal or a transaction that only writes on a single data service) or 2-/3-phase commit (distributed transaction running on a federation).
protected void	deactivateTx(AbstractTransactionService.TxState state) Removes the transaction from the local tables.
void	destroy() Immediate/fast shutdown of the service and then destroys any persistent state associated with the service.
protected abstract long	findCommitTime(long timestamp) Find the commit time from which the tx will read (largest commitTime LTE timestamp).
protected abstract long	findNextCommitTime(long commitTime) Return the commit time for the successor of that commit point have the specified timestamp (a commit time strictly GT the given value).
protected long	findUnusedTimestamp(long commitTime, long nextCommitTime, long timeout, TimeUnit unit) Find a valid, unused timestamp.
long	getAbortCount() #of transaction aborted.
int	getActiveCount() The #of open transactions in any RunState.
long	getCommitCount() #of transaction committed.
CounterSet	getCounters() Return the CounterSet.
protected AbstractTransactionService.TxState	getEarliestActiveTx() Return the AbstractTransactionService.TxState for the earliest active Tx -or- null if there is no active tx.
protected long	getEffectiveReleaseTimeForHA() This method was introduced to compute the effective timestamp of the pinned history in support of the HA TXS.
abstract long	getLastCommitTime() Note: Declared abstract so that we can hide the IOException.
long	getMinReleaseAge() Return the minimum #of milliseconds of history that must be preserved.
protected Properties	getProperties() An object wrapping the properties used to initialize the service.
long	getReadOnlyActiveCount()
protected long	getReadsOnTime(long txId) Return the commit time on which the transaction is reading.
long	getReadWriteActiveCount()
long	getReleaseTime() Return the timestamp whose historical data MAY be released.
TxServiceRunState	getRunState() Return the RunState.
Class	getServiceIface() Return the most interesting interface for the service.
long	getStartCount() #of transaction started.
protected AbstractTransactionService.TxState	getTxState(long tx) Return the AbstractTransactionService.TxState associated with the specified transition identifier.
boolean	isOpen() Any state other than TxServiceRunState.Halted.
protected boolean	isReleaseTimeConsensusProtocol() Return true iff the release time consensus protocol is being used to update the releaseTime (HA and this service is either a leader or a follower).
long	newTx(long timestamp) Create a new transaction.
long	nextTimestamp() Return the next unique timestamp.
void	notifyCommit(long commitTime) The basic implementation advances the release time periodically as commits occur even when there are no transactions in use.
protected void	setReleaseTime(long newValue) Sets the new release time.
protected void	setRunState(TxServiceRunState newval) Change the TxServiceRunState.
void	shutdown() Polite shutdown.
void	shutdownNow() Fast shutdown (not immediate since it must abort active transactions).
AbstractTransactionService	start() Verifies that nextTimestamp() will not report a time before getLastCommitTime() and then changes the TxServiceRunState to TxServiceRunState.Running.
protected void	updateReleaseTime(long timestamp, AbstractTransactionService.TxState deactivatedTx) This method MUST be invoked each time a transaction completes with the absolute value of the transaction identifier that has just been deactivated.
protected void	updateReleaseTimeForBareCommit(long commitTime) If there are NO active transactions and the current releaseTime is LT (commitTime-1) then compute and set the new releaseTime.

Methods inherited from class com.bigdata.service.AbstractService

clearLoggingContext, getFederation, getHostname, getServiceName, getServiceUUID, setServiceUUID, setupLoggingContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.service.IService

getHostname, getServiceName, getServiceUUID

Field Detail

log

protected static final org.apache.log4j.Logger log

Logger.

ERR_READ_ONLY

protected static final transient String ERR_READ_ONLY

If the transaction is read-only and a write operation was requested.

See Also:

Constant Field Values

ERR_NO_SUCH

protected static final transient String ERR_NO_SUCH

If the transaction is not known to this service.

See Also:

Constant Field Values

ERR_NOT_ACTIVE

protected static final transient String ERR_NOT_ACTIVE

If a transaction is no longer active.

See Also:

Constant Field Values

ERR_SERVICE_NOT_AVAIL

protected static final transient String ERR_SERVICE_NOT_AVAIL

If the transaction service is not in a run state which permits the requested operation.

See Also:

Constant Field Values

lock

protected final ReentrantLock lock

A lock used to serialize certain operations that must be atomic with respect to the state of the transaction service. Mostly this is used to serialize the assignment of transaction identifiers and the update of the release time as transactions complete.

Note: To avoid lock ordering problems DO NOT acquire this lock if you are already holding a AbstractTransactionService.TxState.lock. This causes a lock ordering problem and can result in deadlock.

txDeactivate

protected final Condition txDeactivate

Signaled by deactivateTx(TxState) and based on lock.

Constructor Detail

AbstractTransactionService

public AbstractTransactionService(Properties properties)

Method Detail

getProperties

protected Properties getProperties()

An object wrapping the properties used to initialize the service.

getTxState

protected AbstractTransactionService.TxState getTxState(long tx)

Return the AbstractTransactionService.TxState associated with the specified transition identifier.

Note: This method is an internal API. The caller must adhere to the internal synchronization APIs for the transaction service.

Parameters:

tx - The transaction identifier (the signed value, NOT the absolute value).

Returns:

The AbstractTransactionService.TxState -or- null if there is no such active transaction.

getActiveCount

public final int getActiveCount()

The #of open transactions in any RunState.

isOpen

public boolean isOpen()

Any state other than TxServiceRunState.Halted.

Specified by:

isOpen in interface IServiceShutdown

assertOpen

protected void assertOpen()

getRunState

public TxServiceRunState getRunState()

Return the RunState.

setRunState

protected void setRunState(TxServiceRunState newval)

Change the TxServiceRunState.

Parameters:

newval - The new value.

Throws:

IllegalStateException - if the requested state is not a legal state change.

shutdown

public void shutdown()

Polite shutdown. New transactions will not start. This method will block until existing transactions (both read-write and read-only) are complete (either aborted or committed).

Specified by:

shutdown in interface IServiceShutdown

Overrides:

shutdown in class AbstractService

shutdownNow

public void shutdownNow()

Fast shutdown (not immediate since it must abort active transactions).

New transactions will not start and active transactions will be aborted. Transactions which are concurrently committing MAY fail (throwing exceptions from various methods, including nextTimestamp()) when the service halts.

Specified by:

shutdownNow in interface IServiceShutdown

Overrides:

shutdownNow in class AbstractService

abortAllTx

public void abortAllTx()

Abort all active transactions.

destroy

public void destroy()

Immediate/fast shutdown of the service and then destroys any persistent state associated with the service.

Specified by:

destroy in interface IService

Overrides:

destroy in class AbstractService

nextTimestamp

public long nextTimestamp()

Description copied from interface: ITimestampService

Return the next unique timestamp. Timestamps must be strictly increasing.

Note: This method MUST return strictly increasing values, even when it is invoked by concurrent threads. While other implementations are possible and may be more efficient, one way to insure thread safety is to synchronize on some object such that the implementaiton exhibits a FIFO behavior.

Specified by:

nextTimestamp in interface ITimestampService

See Also:

TimestampServiceUtil.nextTimestamp(ITimestampService)

newTx

public long newTx(long timestamp)

Create a new transaction.

Note: There is an upper bound of one read-write transaction that may be created per millisecond (the resolution of nextTimestamp()) and requests for new read-write transactions contend with other requests for nextTimestamp().

Note: The transaction service will refuse to start new transactions whose timestamps are LTE to getReleaseTime().

Specified by:

newTx in interface ITransactionService

Parameters:

timestamp - The timestamp may be:

• A timestamp (GT ZERO), which will result in a read-historical (read-only) transaction that reads from the most recent committed state whose commit timestamp is less than or equal to timestamp. (As a special case, a timestamp that is GT the lastCommitTime will produce a read-only transaction that is reading on the lastCommitTime with snapshot isolation (new writes will not become visible in the view). This is basically the same behavior as specifying ITx.READ_COMMITTED. While perhaps counter-intuitive, this behavior is preferred to throwing an exception when a user attempts to read from a commit time GT the most recent commit point.).
• The symbolic constant ITx.READ_COMMITTED to obtain a read-historical transaction reading from the most recently committed state of the database. The transaction will be assigned a start time corresponding to the most recent commit point of the database and will be a fully isolated read-only view of the state of the database as of that start time. (This is an atomic shorthand for newTx(getLastCommitTime())).
• ITx.UNISOLATED for a read-write transaction.

Returns:

The unique transaction identifier.

Throws:

RuntimeException - Wrapping TimeoutException if a timeout occurs awaiting a start time which would satisfy the request for a read-only transaction (this can occur only for read-only transactions which must contend for start times which will read from the appropriate historical commit point).

getStartCount

public long getStartCount()

#of transaction started.

getAbortCount

public long getAbortCount()

#of transaction aborted.

getCommitCount

public long getCommitCount()

#of transaction committed.

getReadOnlyActiveCount

public long getReadOnlyActiveCount()

getReadWriteActiveCount

public long getReadWriteActiveCount()

getEarliestActiveTx

protected AbstractTransactionService.TxState getEarliestActiveTx()

Return the AbstractTransactionService.TxState for the earliest active Tx -or- null if there is no active tx.

Note: The lock is required in order to make atomic decisions about the earliest active tx. Without the lock, the tx could stop or a new tx could start, thereby invalidating the "earliest active" guarantee.

Throws:

IllegalMonitorStateException - unless the lock is held by the caller.

getReleaseTime

public long getReleaseTime()

Return the timestamp whose historical data MAY be released. This time is derived from the minimum of the timestamp of the earliest running transaction and now-minReleaseAge and is updated whenever the earliest running transaction terminates. This value is monotonically increasing. It will always be LT the last non-zero last commit time. It will never be negative. It MAY be ZERO (0L) and will be ZERO (0L) on startup (unless explicitly set by the database to the last known commit time).

The returned values is used to identify the most recent commit point LTE the releaseTime. This is the earliest commit point on whose data MAY be released. (Consequently, the first commit point GT the releaseTime is the earliest visible commit point.)

Specified by:

getReleaseTime in interface ITransactionService

See Also:

AbstractTransactionService.Options.MIN_RELEASE_AGE

setReleaseTime

protected void setReleaseTime(long newValue)

Sets the new release time.

Note: For a joined service in HA (the leader or a follower), the release time is set by the consensus protocol. Otherwise it is automatically maintained by updateReleaseTime(long, TxState) and updateReleaseTimeForBareCommit(long).

Parameters:

newValue - The new value.

See Also:

Query on follower fails during UPDATE on leader

getEffectiveReleaseTimeForHA

protected long getEffectiveReleaseTimeForHA()

This method was introduced to compute the effective timestamp of the pinned history in support of the HA TXS. It ignores the releaseTime and reports the minimum of now - minReleaseAge and the readsOnCommitTime of the earliest active Tx. If the value would be negative, then ZERO (0L) is reported instead.

Note: This duplicates logic in #updateReleaseTime(long), but handles the special case in HA where the releaseTime is not being updated by updateReleaseTimeForBareCommit(long).

Returns:

The effective release time.

See Also:

HA TXS , #updateReleaseTime(long)

activateTx

protected void activateTx(AbstractTransactionService.TxState state)

Adds the transaction from to the local tables.

Parameters:

state - The transaction.

getReadsOnTime

protected long getReadsOnTime(long txId)

Return the commit time on which the transaction is reading.

Note: This method is exposed primarily for the unit tests.

Parameters:

txId - The transaction identifier.

Returns:

The commit time on which that transaction is reading.

Throws:

IllegalArgumentException - if there is no such transaction.

deactivateTx

protected void deactivateTx(AbstractTransactionService.TxState state)

Removes the transaction from the local tables.

Note: The caller MUST own AbstractTransactionService.TxState.lock across this method and MUST then do

 updateReleaseTime(long)
 deactivateTx.signallAll()
 

while holding the outer lock.

Note: Normally this method is invoked without the outer lock which necessitates lifting those method calls out of this method and into the caller.

Parameters:

state - The transaction.

Throws:

IllegalMonitorStateException - unless the caller is holding the AbstractTransactionService.TxState.lock.

isReleaseTimeConsensusProtocol

protected boolean isReleaseTimeConsensusProtocol()

Return true iff the release time consensus protocol is being used to update the releaseTime (HA and this service is either a leader or a follower). Return false iff the service should locally manage its own release time (non-HA and HA when the service is HAStatusEnum.NotReady).

Note: When we are using a 2-phase commit, the leader can not update the release time from commit() using this methods. It must rely on the consensus protocol to update the release time instead.

See Also:

Journal HA

updateReleaseTime

protected final void updateReleaseTime(long timestamp,
                     AbstractTransactionService.TxState deactivatedTx)

This method MUST be invoked each time a transaction completes with the absolute value of the transaction identifier that has just been deactivated. The method will remove the transaction entry in the ordered set of running transactions (startTimeIndex).

If the specified timestamp corresponds to the earliest running transaction, then the releaseTime will be updated and the new releaseTime will be set using setReleaseTime(long). For HA, the releaseTime is updated by a consensus protocol and the individual services MUST NOT advance their releaseTime as transactions complete.

Note: When we are using a 2-phase commit, the leader can not update the release time from commit() using this methods. It must rely on the consensus protocol to update the release time instead.

Parameters:

timestamp - The absolute value of a transaction identifier that has just been deactivated.

deactivatedTx - The transaction object that has been deactivated -or- null if there are known to be no active transactions remaining (e.g., startup and abortAll()).

See Also:

IllegalStateException trying to access lexicon index using RWStore with recycling , Query on follower fails during UPDATE on leader

notifyCommit

public void notifyCommit(long commitTime)

The basic implementation advances the release time periodically as commits occur even when there are no transactions in use.

Note: This needs to be a fairly low-latency operation since this method is invoked for all commits on all data services and will otherwise be a global hotspot.

Specified by:

notifyCommit in interface ITransactionService

Parameters:

commitTime - The commit time.

updateReleaseTimeForBareCommit

protected void updateReleaseTimeForBareCommit(long commitTime)

If there are NO active transactions and the current releaseTime is LT (commitTime-1) then compute and set the new releaseTime.

Note: This method was historically part of notifyCommit(long). It was moved into its own method so it can be overridden for some unit tests.

Note: When we are using a 2-phase commit, the leader can not update the release time from commit() using this methods. It must rely on the consensus protocol to update the release time instead.

getMinReleaseAge

public final long getMinReleaseAge()

Return the minimum #of milliseconds of history that must be preserved.

assignTransactionIdentifier

protected final AbstractTransactionService.TxState assignTransactionIdentifier(long timestamp)
                                                                        throws InterruptedException,
                                                                               TimeoutException

Assign a transaction identifier for a new transaction.

Parameters:

timestamp - The timestamp.

Returns:

The new transaction object.

Throws:

InterruptedException - if interrupted while awaiting a start time which would satisfy the request.

InterruptedException - if a timeout occurs while awaiting a start time which would satisfy the request.

TimeoutException

findCommitTime

protected abstract long findCommitTime(long timestamp)

Find the commit time from which the tx will read (largest commitTime LTE timestamp).

Parameters:

timestamp - The timestamp.

Returns:

The commit time and -1L if there is no such commit time.

findNextCommitTime

protected abstract long findNextCommitTime(long commitTime)

Return the commit time for the successor of that commit point have the specified timestamp (a commit time strictly GT the given value).

Parameters:

commitTime - The probe.

Returns:

The successor or -1L iff the is no successor for that commit time.

findUnusedTimestamp

protected long findUnusedTimestamp(long commitTime,
                       long nextCommitTime,
                       long timeout,
                       TimeUnit unit)
                            throws InterruptedException,
                                   TimeoutException

Find a valid, unused timestamp.

Note: Any timestamp in the half-open range [commitTime:nextCommitTime) MAY be assigned as all such timestamps will read from the commit point associated with [commitTime].

Parameters:

commitTime - The commit time for the commit point on which the tx will read (this must be the exact timestamp associated with the desired commit point).

nextCommitTime - The commit time for the successor of that commit point.

timeout - The maximum length of time to await an available timestamp.

unit - The unit in which timeout is expressed.

Throws:

InterruptedException

TimeoutException

getLastCommitTime

public abstract long getLastCommitTime()

Note: Declared abstract so that we can hide the IOException.

Specified by:

getLastCommitTime in interface ITransactionService

Returns:

The last known commit time.

abortImpl

protected abstract void abortImpl(AbstractTransactionService.TxState state)
                           throws Exception

Implementation must abort the tx on the journal (standalone) or on each data service (federation) on which it has written.

Pre-conditions:

1. The transaction is RunState.Active; and
2. The caller holds the AbstractTransactionService.TxState.lock.

Post-conditions:

1. The transaction is RunState.Aborted; and
2. The transaction write set has been discarded by each Journal or IDataService or which it has written (applicable for read-write transactions only).

Parameters:

state - The transaction state as maintained by the transaction server.

Throws:

Exception

commitImpl

protected abstract long commitImpl(AbstractTransactionService.TxState state)
                            throws Exception

Implementation must either single-phase commit (standalone journal or a transaction that only writes on a single data service) or 2-/3-phase commit (distributed transaction running on a federation).

Pre-conditions:

1. The transaction is RunState.Active; and
2. The caller holds the AbstractTransactionService.TxState.lock.

Post-conditions (success for read-only transaction or a read-write transaction with an empty write set):

1. The transaction is RunState.Committed; and
2. The returned commitTime is ZERO (0L).

Post-conditions (success for read-write transaction with a non-empty write set):

1. The transaction is RunState.Committed;
2. The transaction write set has been made restart-safe by each Journal or IDataService or which it has written (applicable for read-write transactions only); and
3. The application can read exactly the data written by the transaction from the commit point identified by the returned commitTime.

Post-conditions (failure):

1. The transaction is RunState.Aborted; and
2. The transaction write set has been discarded by each Journal or IDataService or which it has written (applicable for read-write transactions only).

Parameters:

tx - The transaction identifier.

Returns:

The commit time for the transaction -or- ZERO (0L) if the transaction was read-only or had an empty write set.

Throws:

Exception - if something else goes wrong. This will be (or will wrap) a ValidationError if validation fails.

abort

public void abort(long tx)

Abort the transaction (asynchronous).

Specified by:

abort in interface ITransactionService

Parameters:

tx - The transaction identifier.

commit

public long commit(long tx)
            throws ValidationError

Description copied from interface: ITransactionService

Request commit of the transaction write set. Committing a read-only transaction is necessary in order to release read locks (this is very fast). If a transaction has a write set, then this method does not return until that write set has been made restart safe or the transaction has failed.

The commit of a transaction with a write set on a single IDataService does not require either ITx.UNISOLATED tasks or other transactions to wait. The latency for such commits is directly related to the size of the transaction write set.

However, the commit of a transaction with writes on more than one IDataService requires a distributed commit protocol. The distributed commit protocol forces ALL tasks writing on those IDataServices to wait until the transaction is complete. This is necessary in order to obtain a global commit point that corresponds to the atomic commit state of the transaction (without this we would not have the Atomic property for distributed transaction commits).

Specified by:

commit in interface ITransactionService

Parameters:

tx - The transaction identifier.

Returns:

The commit time for the transaction -or- ZERO (0L) if the transaction was read-only or had an empty write set. This commit time identifies a global commit point on the database from which you may read the coherent post-commit state of the transaction.

Throws:

ValidationError - if the transaction could not be validated.

start

public AbstractTransactionService start()

Verifies that nextTimestamp() will not report a time before getLastCommitTime() and then changes the TxServiceRunState to TxServiceRunState.Running.

Specified by:

start in class AbstractService

Returns:

this (the return type should be strengthened by the concrete implementation to return the actual type).

getServiceIface

public Class getServiceIface()

Description copied from class: AbstractService

Return the most interesting interface for the service.

Specified by:

getServiceIface in interface IService

Specified by:

getServiceIface in class AbstractService

getCounters

public CounterSet getCounters()

Return the CounterSet.

Specified by:

getCounters in interface ICounterSetAccess