protected static class AbstractResourceManagerTestCase.MockMetadataService extends Object implements IMetadataService
IMetadataService - only those methods
actually used by the ResourceManager are implemented. This avoids
conflicts with the ResourceManager instance whose behavior we are
trying to test.| Modifier | Constructor and Description |
|---|---|
protected |
AbstractResourceManagerTestCase.MockMetadataService() |
| Modifier and Type | Method and Description |
|---|---|
void |
abort(long tx)
Request abort of the transaction by the data service.
|
void |
destroy()
Destroy the service.
|
void |
dropIndex(String name)
Drops the named index.
|
void |
dropScaleOutIndex(String name)
Drop the named scale-out index.
|
PartitionLocator |
find(String name,
long timestamp,
byte[] key)
Find and return the partition spanning the given key.
|
void |
forceOverflow(boolean immediate,
boolean compactingMerge)
Method sets a flag that will force overflow processing during the next
group commit and optionally forces a group commit (Note: This
method exists primarily for unit tests and benchmarking activities and
SHOULD NOT be used on a deployed federation as the overhead associated
with a compacting merge of each index partition can be significant).
|
PartitionLocator |
get(String name,
long timestamp,
byte[] key)
The partition with that separator key or
null (exact match
on the separator key). |
long |
getAsynchronousOverflowCounter()
The #of asynchronous overflows that have taken place on this data service
(the counter is not restart safe).
|
String |
getHostname()
The host on which this service is running.
|
IndexMetadata |
getIndexMetadata(String name,
long timestamp)
Return the metadata for the named index.
|
IQueryPeer |
getQueryEngine()
Return the
IQueryPeer running on this service. |
Class |
getServiceIface()
Return the most interesting interface for the service.
|
String |
getServiceName()
Return name by which a user might recognize this service.
|
UUID |
getServiceUUID()
The unique identifier for this service.
|
Session |
getSession() |
String |
getStatistics() |
boolean |
isOverflowActive()
Return
true iff the data service is currently engaged in
overflow processing. |
void |
joinIndexPartition(String name,
PartitionLocator[] oldLocators,
PartitionLocator newLocator)
Updates the
MetadataIndex for the named scale-out index to
reflect the join of N index partitions (which must be siblings) into a
single index partition. |
void |
moveIndexPartition(String name,
PartitionLocator oldLocator,
PartitionLocator newLocator)
Updates the
MetadataIndex for the named scale-out index to
reflect the move of an index partition from one data service to another. |
int |
nextPartitionId(String name)
Return the next unique partition identifier to be assigned to the named
scale-out index.
|
void |
prepare(long tx,
long revisionTime)
Request that the
IDataService participate in a 3-phase commit. |
boolean |
purgeOldResources(long timeout,
boolean truncateJournal)
This attempts to pause the service accepting
ITx.UNISOLATED
writes and then purges any resources that are no longer required based on
the StoreManager.Options#MIN_RELEASE_AGE. |
ResultSet |
rangeIterator(long tx,
String name,
byte[] fromKey,
byte[] toKey,
int capacity,
int flags,
IFilter filter)
Streaming traversal of keys and/or values in a key range.
|
IBlock |
readBlock(IResourceMetadata resource,
long addr)
Read a low-level record from the described
IRawStore described by
the IResourceMetadata. |
void |
registerIndex(String name,
IndexMetadata metadata)
Register a named mutable index on the
DataService. |
UUID |
registerScaleOutIndex(IndexMetadata metadata,
byte[][] separatorKeys,
UUID[] dataServices)
Register and statically partition a scale-out index.
|
void |
setReleaseTime(long releaseTime)
Notify a data service that it MAY release data required to support views
for up to the specified releaseTime .
|
long |
singlePhaseCommit(long tx)
Request commit of the transaction by the data service.
|
void |
splitIndexPartition(String name,
PartitionLocator oldLocator,
PartitionLocator[] newLocators)
Updates the
MetadataIndex for the named scale-out index to
reflect the split of an index partition into N new index partitions. |
Future<? extends Object> |
submit(Callable<? extends Object> proc)
|
Future |
submit(long tx,
String name,
IIndexProcedure proc)
Submit a procedure.
|
protected AbstractResourceManagerTestCase.MockMetadataService()
public int nextPartitionId(String name) throws IOException, InterruptedException, ExecutionException
IMetadataServicenextPartitionId in interface IMetadataServicename - The name of the scale-out index.InterruptedExceptionExecutionExceptionIOExceptionpublic UUID registerScaleOutIndex(IndexMetadata metadata, byte[][] separatorKeys, UUID[] dataServices) throws IOException, InterruptedException, ExecutionException
IMetadataServiceregisterScaleOutIndex in interface IMetadataServicemetadata - The metadata template describing the scale-out index,
including the name to be assigned to that index.separatorKeys - The array of separator keys. Each separator key is interpreted
as an unsigned byte[]. The first entry MUST be an
empty byte[]. The entries MUST be in sorted order.IOExceptionInterruptedExceptionExecutionExceptionpublic void dropScaleOutIndex(String name) throws IOException, InterruptedException, ExecutionException
IMetadataServicedropScaleOutIndex in interface IMetadataServicename - The name of the scale-out index.IOExceptionInterruptedExceptionExecutionExceptionpublic UUID getServiceUUID() throws IOException
IService
Note: Some service discovery frameworks (Jini) will assign the service a
UUID asynchronously after a new service starts, in which case
this method will return null until the service
UUID has been assigned.
getServiceUUID in interface IServiceIOException - since you can use this method with RMI.public String getStatistics() throws IOException
IOExceptionpublic void registerIndex(String name, IndexMetadata metadata) throws IOException, InterruptedException, ExecutionException
IDataServiceDataService.
Note: In order to register an index partition the
partition metadata property
MUST be set. The resources
property will then be overriden when the index is actually registered so
as to reflect the IResourceMetadata description of the journal on
which the index actually resides.
registerIndex in interface IDataServicename - The name that can be used to recover the index. In order to
create a partition of an index you must form the name of the
index partition using
DataService.getIndexPartitionName(String, int) (this
operation is generally performed by the
IMetadataService which manages scale-out indices).metadata - The metadata describing the index.
The LocalPartitionMetadata.getResources() property on
the IndexMetadata.getPartitionMetadata() SHOULD NOT be
set. The correct IResourceMetadata[] will be assigned
when the index is registered on the IDataService.
IOExceptionInterruptedExceptionExecutionExceptionpublic IndexMetadata getIndexMetadata(String name, long timestamp) throws IOException
IDataServicegetIndexMetadata in interface IDataServicename - The index name.timestamp - A transaction identifier, ITx.UNISOLATED for the
unisolated index view, ITx.READ_COMMITTED, or
timestamp for a historical view no later than
the specified timestamp.IOExceptionpublic void dropIndex(String name) throws IOException, InterruptedException, ExecutionException
IDataService
Note: In order to drop a partition of an index you must form the name of
the index partition using
DataService.getIndexPartitionName(String, int) (this operation is
generally performed by the IMetadataService which manages
scale-out indices).
dropIndex in interface IDataServicename - The index name.IOExceptionInterruptedExceptionExecutionExceptionpublic ResultSet rangeIterator(long tx, String name, byte[] fromKey, byte[] toKey, int capacity, int flags, IFilter filter) throws InterruptedException, ExecutionException, IOException
IDataServiceStreaming traversal of keys and/or values in a key range.
Note: In order to visit all keys in a range, clients are expected to
issue repeated calls in which the fromKey is incremented to the
successor of the last key visited until either an empty ResultSet
is returned or the ResultSet#isLast() flag is set, indicating
that all keys up to (but not including) the startKey have been
visited. See ClientIndexView (scale-out indices) and
DataServiceTupleIterator (unpartitioned indices), both of which
encapsulate this method.
Note: If the iterator can be determined to be read-only and it is
submitted as ITx.UNISOLATED then it will be run as
ITx.READ_COMMITTED to improve concurrency.
rangeIterator in interface IDataServicetx - The transaction identifier -or- ITx.UNISOLATED IFF the
operation is NOT isolated by a transaction -or-
- tx to read from the most recent commit point
not later than the absolute value of tx (a fully
isolated read-only transaction using a historical start time).name - The index name (required).fromKey - The starting key for the scan (or null iff
there is no lower bound).toKey - The first key that will not be visited (or null
iff there is no upper bound).capacity - When non-zero, this is the maximum #of entries to process.flags - One or more flags formed by bitwise OR of zero or more of the
constants defined by IRangeQuery.filter - An optional object that may be used to layer additional
semantics onto the iterator. The filter will be constructed on
the server and in the execution context for the iterator, so
it will execute directly against the index for the maximum
efficiency.InterruptedException - if the operation was interrupted.ExecutionException - If the operation caused an error. See
Throwable.getCause() for the underlying
error.IOExceptionpublic Future submit(long tx, String name, IIndexProcedure proc)
IDataServiceSubmit a procedure.
Unisolated operations SHOULD be used to achieve "auto-commit" semantics. Fully isolated transactions are useful IFF multiple operations must be composed into a ACID unit.
While unisolated batch operations on a single data service are ACID, clients are required to locate all index partitions for the logical operation and distribute their operation across the distinct data service instances holding the affected index partitions. In practice, this means that contract for ACID unisolated operations is limited to operations where the data is located on a single data service instance. For ACID operations that cross multiple data service instances the client MUST use a fully isolated transaction.
submit in interface IDataServicetx - The transaction identifier, ITx.UNISOLATED for an ACID
operation NOT isolated by a transaction,
ITx.READ_COMMITTED for a read-committed operation not
protected by a transaction (no global read lock), or any valid
commit time for a read-historical operation not protected by a
transaction (no global read lock).name - The name of the index partition.proc - The procedure to be executed.Future from which the outcome of the procedure may be
obtained.public IBlock readBlock(IResourceMetadata resource, long addr)
IDataServiceIRawStore described by
the IResourceMetadata.readBlock in interface IDataServiceresource - The description of the resource containing that block.addr - The address of the block in that resource.public void splitIndexPartition(String name, PartitionLocator oldLocator, PartitionLocator[] newLocators) throws IOException, InterruptedException, ExecutionException
IMetadataServiceMetadataIndex for the named scale-out index to
reflect the split of an index partition into N new index partitions. The
old index partition locator is removed from the MetadataIndex and
the new index partition locators are inserted in a single atomic
operation.splitIndexPartition in interface IMetadataServicename - The name of the scale-out index.oldLocator - The partition locator that is being split.newLocators - The locator information for the new index partitions that were
created by the split of the old index partition.IOExceptionInterruptedExceptionExecutionExceptionpublic void joinIndexPartition(String name, PartitionLocator[] oldLocators, PartitionLocator newLocator) throws IOException, InterruptedException, ExecutionException
IMetadataServiceMetadataIndex for the named scale-out index to
reflect the join of N index partitions (which must be siblings) into a
single index partition. The old index partition locators are removed from
the MetadataIndex and the new index partition locator is inserted
in a single atomic operation.joinIndexPartition in interface IMetadataServicename - The name of the scale-out index.oldLocators - The partition locators for the index partitions that were
joined.newLocator - The locator for the new index partition created by that join.IOExceptionInterruptedExceptionExecutionExceptionpublic void moveIndexPartition(String name, PartitionLocator oldLocator, PartitionLocator newLocator) throws IOException, InterruptedException, ExecutionException
IMetadataServiceMetadataIndex for the named scale-out index to
reflect the move of an index partition from one data service to another.
The old index partition locator is removed from the MetadataIndex
and the new index partition locator is inserted in a single atomic
operation.moveIndexPartition in interface IMetadataServicename - The name of the scale-out index.oldLocator - The partition locator for the source index partition.newLocator - The locator for the target index partition.IOExceptionInterruptedExceptionExecutionExceptionpublic PartitionLocator get(String name, long timestamp, byte[] key) throws InterruptedException, ExecutionException, IOException
IMetadataServicenull (exact match
on the separator key).get in interface IMetadataServicename - The name of the scale-out index.key - The separator key (the first key that would go into that
partition).null.InterruptedExceptionExecutionExceptionIOExceptionpublic PartitionLocator find(String name, long timestamp, byte[] key) throws InterruptedException, ExecutionException, IOException
IMetadataServicefind in interface IMetadataServicename - The name of the scale-out index.key - A key of interest for the scale-out index.null if
there are no partitions defined.InterruptedExceptionExecutionExceptionIOExceptionpublic void forceOverflow(boolean immediate,
boolean compactingMerge)
throws IOException
IDataService
Normally there is no reason to invoke this method directly. Overflow
processing is triggered automatically on a bottom-up basis when the
extent of the live journal nears the Options.MAXIMUM_EXTENT.
forceOverflow in interface IDataServiceimmediate - The purpose of this argument is to permit the caller to
trigger an overflow event even though there are no writes
being made against the data service. When true
the method will write a token record on the live journal in
order to provoke a group commit. In this case synchronous
overflow processing will have occurred by the time the method
returns. When false a flag is set and overflow
processing will occur on the next commit.compactingMerge - The purpose of this flag is to permit the caller to indicate
that a compacting merge should be performed for all indices on
the data service (at least, all indices whose data are not
simply copied onto the new journal) during the next
synchronous overflow. Note that compacting merges of indices
are performed automatically from time to time so this flag
exists mainly for people who want to force a compacting merge
for some reason.IOExceptionpublic boolean isOverflowActive()
throws IOException
IDataServicetrue iff the data service is currently engaged in
overflow processing.isOverflowActive in interface IDataServiceIOExceptionpublic long getAsynchronousOverflowCounter()
throws IOException
IDataServicegetAsynchronousOverflowCounter in interface IDataServiceIOExceptionpublic void destroy()
IServiceDestroyAdmin#destroy().public Future<? extends Object> submit(Callable<? extends Object> proc)
IDataServiceCallable and return its Future. The
Callable will execute on the
IBigdataFederation.getExecutorService().
Note: This interface is specialized by the IDataService for tasks
which need to gain access to the IDataService in order to gain
local access to index partitions, etc. Such tasks declare the
IDataServiceCallable. For example, scale-out joins use
this mechanism.
submit in interface IDataServicesubmit in interface IRemoteExecutorFuture for that task.IDataServiceCallablepublic String getHostname() throws IOException
IServicegetHostname in interface IServiceIOException - since you can use this method with RMI.public Class getServiceIface() throws IOException
IServicegetServiceIface in interface IServiceIOException - since you can use this method with RMI.public String getServiceName() throws IOException
IServicegetServiceName in interface IServiceIOException - since you can use this method with RMI.public boolean purgeOldResources(long timeout,
boolean truncateJournal)
throws IOException,
InterruptedException
IDataServiceITx.UNISOLATED
writes and then purges any resources that are no longer required based on
the StoreManager.Options#MIN_RELEASE_AGE.
Note: Resources are normally purged during synchronous overflow handling. However, asynchronous overflow handling can cause resources to no longer be needed as new index partition views are defined. This method MAY be used to trigger a release before the next overflow event.
purgeOldResources in interface IDataServicetimeout - The timeout (in milliseconds) that the method will await the
pause of the write service.truncateJournal - When true, the live journal will be truncated
to its minimum extent (all writes will be preserved but there
will be no free space left in the journal). This may be used
to force the DataService to its minimum possible
footprint for the configured history retention policy.true if successful and false if the
write service could not be paused after the specified timeout.IOExceptionInterruptedExceptionpublic void setReleaseTime(long releaseTime)
ITxCommitProtocolsetReleaseTime in interface ITxCommitProtocolreleaseTime - The new release time (strictly advanced by the transaction
manager).public void abort(long tx)
throws IOException
ITxCommitProtocolITransactionService.abort(long) to each
IDataService on which the transaction has written. It is NOT sent
for read-only transactions since they have no local state on the
IDataServices.abort in interface ITxCommitProtocoltx - The transaction identifier.IOException - if there is an RMI problem.public long singlePhaseCommit(long tx)
throws InterruptedException,
ExecutionException,
IOException
ITxCommitProtocol#prepare(long) message MUST be used.singlePhaseCommit in interface ITxCommitProtocoltx - The transaction identifier.InterruptedException - if interrupted.ExecutionException - This will wrap a ValidationError if validation fails.IOException - if there is an RMI problem.public void prepare(long tx,
long revisionTime)
throws InterruptedException,
ExecutionException,
IOException
ITxCommitProtocolIDataService participate in a 3-phase commit.
When the IDataService is sent the ITxCommitProtocol.prepare(long, long)
message it executes a task which will handle commit processing for the
transaction. That task MUST hold exclusive locks for the unisolated
indices to which the transaction write sets will be applied. While
holding those locks, the task must first validate the transaction's write
set and then merge down the write set onto the corresponding unisolated
indices using the specified revisionTime and checkpoint the
indices in order to reduce all possible sources of latency. Note that
each IDataService is able to independently prepare exactly those
parts of the transaction's write set which are mapped onto index
partitions hosted by a given IDataService.
Once validation is complete and all possible steps have been taken to
reduce sources of latency (e.g., checkpoint the indices and pre-extending
the store if necessary), the task notifies the
ITransactionService that it has prepared using
ITransactionService#prepared(long). The
ITransactionService will wait until all tasks have prepared. If a
task CAN NOT prepare the transaction, then it MUST throw an exception out
of its ITxCommitProtocol.prepare(long, long) method.
Once all tasks have send an ITransactionService#prepared(long)
message to the ITransactionService, it will assign a commitTime
to the transaction and permit those methods to return that commitTime to
the IDataServices. Once the task receives the assigned commit
time, it must obtain an exclusive write lock for the live journal (this
is a higher requirement than just an exclusive lock on the necessary
indices and will lock out all other write requests for the journal),
register the checkpointed indices on the commit list and then request a
commit of the journal using the specified commitTime. The task then
notifies the transaction service that it has completed its commit using
ITransactionService#committed(long) and awaits a response. If the
ITransactionService indicates that the commit was not successful,
the task rolls back the live journal to the prior commit point and throws
an exception out of ITxCommitProtocol.prepare(long, long).
A sample flow for successful a distributed transaction commit is shown
below. This example shows two IDataServices on which the client
has written. (If the client only writes on a single data service then we
use a single-phase commit protocol).
client -------+----txService----+--dataService1--+--dataService2--+... | [1] | commit(tx) -------- + [2] | | prepare(tx,rev) + | | [3] | | | prepare(tx,rev) ------------------+ | | | | | | <--prepared(tx) + | | | | | | <------------------- prepared(tx) + | | | "prepared" barrier [4] | | | | -- (commitTime) + | | -------------------- (commitTime) + | | [5] | | | | <--committed(tx)------------------+ | | [6] | | | <--committed(tx)+ | | | "committed" barrier [7] | | [8] | | ------ (success)+ | | [9] | | | (void)----------+ | | halt | | [10] | | ------------------------ (success)+ | | [11] | | | (void)----------------------------+ | [12] | halt | (commitTime)--------+ |
ITransactionService.commit(long)
request, in which it specifies the transaction identifier (tx). ITxCommitProtocol.prepare(long, long) requests to the participating
IDataServices, specifying the transaction identifier (tx) and
the revision timestamp (rev) to be used and then waits at a barrier until
it receives ITransactionService#prepared(long) messages from
those IDataServices.ITransactionService assigns a commitTime and returns that
commitTime as the return value for the prepared messages.IDataService obtains that commitTime, it
proceeds with its atomic commit using the specified commitTime and then
sends an ITransactionService#committed(long) message to the
ITransactionService.ITransactionService waits at another barrier.ITransactionService#committed(long) message from each
participating IDataService the transaction has been successfully
committed and the barrier breaks. The ITransactionService now
lets the ITransactionService#committed(long) messages return
true, indicating success.IDataServices return (void) from their
ITxCommitProtocol.prepare(long, long) message and the threads running their side
of the commit protocol halt.ITransactionService returns the commit time which
it assigned and which was used by each participating IDataService
to the client.
IDataServices and discard any local
state associated with the transaction and throw an exception out of
ITxCommitProtocol.prepare(long, long). Once the first barrier has been
satisfied, persistent side-effects MAY occur. Error handling in this
case must rollback the state of the live journal for each of the
participating IDataServices. If error handling was performed in
response to a local error, then the IDataService must throw that
error out of ITxCommitProtocol.prepare(long, long). However, if error handling
was initiated because ITransactionService#committed(long)
returned false then it should return normally (after
rolling back the journal).prepare in interface ITxCommitProtocoltx - The transaction identifier.revisionTime - The timestamp that will be written into the ITuples
when the write set of the validated transaction is merged down
onto the unisolated indices.IOException - if there is an RMI problem.InterruptedExceptionExecutionExceptionpublic Session getSession()
public IQueryPeer getQueryEngine() throws IOException
IDataServiceIQueryPeer running on this service.getQueryEngine in interface IDataServiceIOExceptionCopyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.