T - The generic type of the client or service.public abstract class AbstractFederation<T> extends Object implements IBigdataFederation<T>
IBigdataFederation implementations.| Modifier and Type | Class and Description |
|---|---|
static class |
AbstractFederation.ReportTask
Periodically report performance counter data to the
ILoadBalancerService. |
protected class |
AbstractFederation.StartDeferredTasksTask
This task starts an (optional)
AbstractStatisticsCollector, an
(optional) httpd service, and the (required) AbstractFederation.ReportTask. |
| Modifier and Type | Field and Description |
|---|---|
protected static org.apache.log4j.Logger |
log |
| Modifier | Constructor and Description |
|---|---|
protected |
AbstractFederation(IBigdataClient<T> client) |
| Modifier and Type | Method and Description |
|---|---|
ScheduledFuture<?> |
addScheduledTask(Runnable task,
long initialDelay,
long delay,
TimeUnit unit)
Adds a task which will run until canceled, until it throws an exception,
or until the service is shutdown.
|
protected void |
assertOpen() |
void |
destroy()
Destroys all discovered services belonging to the federation and their
persistent data and disconnects from the federation.
|
void |
didStart()
Delegated.
|
void |
dropIndex(String name)
Drop a scale-out index.
|
AbstractClient<T> |
getClient()
Return the client object that was used to connect to the federation.
|
boolean |
getCollectPlatformStatistics()
true iff performance counters will be collected for the
platform on which the client is running. |
boolean |
getCollectQueueStatistics()
true iff statistics will be collected for work queues. |
CounterSet |
getCounters()
The
CounterSet which the client will use report its statistics to
the ILoadBalancerService. |
IDataService[] |
getDataServices(UUID[] uuids)
Return an array of
IDataService references that is correlated
with the given array of IDataService UUIDs. |
ExecutorService |
getExecutorService()
A thread pool that may be used by clients to parallelize operations
against the federation.
|
BigdataGangliaService |
getGangliaService()
The embedded ganglia peer.
|
BigdataFileSystem |
getGlobalFileSystem()
Return the global file system used to store block-structured files and
their metadata and as a source and sink for map/reduce processing.
|
SparseRowStore |
getGlobalRowStore()
Return an unisolated view of the global
SparseRowStore used to
store named property sets. |
SparseRowStore |
getGlobalRowStore(long timestamp)
Return a view of the global
SparseRowStore used to store named
property sets as of the specified timestamp. |
CounterSet |
getHostCounterSet()
The node in
IBigdataFederation.getCounters() corresponding to the root of the
host on which the client or service is executing. |
int |
getHttpdPort()
The port on which the optional httpd service will be run.
|
String |
getHttpdURL()
The URL that may be used to access the local httpd service for this
client or service.
|
IClientIndex |
getIndex(String name,
long timestamp)
Applies an
AbstractIndexCache and strengthens the return type. |
protected abstract AbstractIndexCache<? extends IClientIndex> |
getIndexCache()
Return the cache for
IIndex objects. |
ScaleOutIndexCounters |
getIndexCounters(String name)
Return the
ScaleOutIndexCounters for the specified scale-out index
for this client. |
DefaultResourceLocator<?> |
getResourceLocator()
Return the default locator for resources that are logical index
containers (relations and relation containers).
|
ScheduledExecutorService |
getScheduledExecutorService()
A service which may be used to schedule performance counter sampling
tasks.
|
T |
getService()
Delegated.
|
String |
getServiceCounterPathPrefix()
The path prefix under which all of the client or service's counters are
located.
|
static String |
getServiceCounterPathPrefix(UUID serviceUUID,
Class serviceIface,
String hostname)
The path prefix under which all of the client or service's counters are
located.
|
CounterSet |
getServiceCounterSet()
The node in
IBigdataFederation.getCounters() corresponding to the root of the
client or service connected using this federation. |
Class<?> |
getServiceIface()
Delegated.
|
String |
getServiceName()
Delegated.
|
UUID |
getServiceUUID()
Delegated.
|
TaskCounters |
getTaskCounters()
Return the
TaskCounters which aggregate across all operations
performed by the client against the connected federation. |
TemporaryStore |
getTempStore()
A factory for
TemporaryStores. |
Iterator<String> |
indexNameScan(String prefix,
long timestamp)
Iterator visits the names of all indices spanned by the given prefix.
|
boolean |
isGroupCommit()
The
IBigdataFederation supports group commit (and always has). |
boolean |
isOpen() |
boolean |
isServiceReady()
Delegated.
|
AbstractHTTPD |
newHttpd(int httpdPort,
ICounterSetAccess accessor)
Delegated.
|
void |
reattachDynamicCounters()
Delegated.
|
void |
registerIndex(IndexMetadata metadata)
Register a scale-out index.
|
UUID |
registerIndex(IndexMetadata metadata,
byte[][] separatorKeys,
UUID[] dataServiceUUIDs)
Register and statically partition a scale-out index.
|
UUID |
registerIndex(IndexMetadata metadata,
UUID dataServiceUUID)
Register a scale-out index and assign the initial index partition to the
specified data service.
|
void |
reportCounters()
Forces the immediate reporting of the
CounterSet to the
ILoadBalancerService. |
protected void |
sendEvent(Event e)
Queues up an event to be sent to the
ILoadBalancerService. |
void |
serviceJoin(IService service,
UUID serviceUUID)
Delegated.
|
void |
serviceLeave(UUID serviceUUID)
Delegated.
|
void |
shutdown()
Normal shutdown allows any existing client requests to federation
services to complete but does not schedule new requests, disconnects from
the federation, and then terminates any background processing that is
being performed on the behalf of the client (service discovery, etc).
|
void |
shutdownNow()
Immediate shutdown terminates any client requests to federation services,
disconnects from the federation, and then terminate any background
processing that is being performed on the behalf of the client (service
discovery, etc).
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitgetAnyDataService, getDataService, getDataServiceByName, getDataServiceUUIDs, getLastCommitTime, getLoadBalancerService, getMetadataIndex, getMetadataService, getTransactionService, isDistributed, isJiniFederation, isScaleOut, isStablegetResourceLockServiceprotected AbstractFederation(IBigdataClient<T> client)
public AbstractClient<T> getClient()
IBigdataFederationgetClient in interface IBigdataFederation<T>public final boolean isOpen()
public void shutdown()
Note: concrete implementations MUST extend this method.
Note: Clients use IBigdataClient.disconnect(boolean) to
disconnect from a federation. The federation implements that disconnect
using either shutdown() or shutdownNow().
The implementation must be a NOP if the federation is already shutdown.
public void shutdownNow()
Note: concrete implementations MUST extend this method to either
disconnect from the remote federation or close the embedded federation
and then clear the #fed reference so that the client is no longer
"connected" to the federation.
Note: Clients use IBigdataClient.disconnect(boolean) to disconnect
from a federation. The federation implements that disconnect using either
shutdown() or shutdownNow().
The implementation must be a NOP if the federation is already shutdown.
public void destroy()
IBigdataFederationdestroy in interface IIndexStoredestroy in interface IBigdataFederation<T>protected final void assertOpen()
IllegalStateException - if the client has disconnected from the federation.public ScheduledExecutorService getScheduledExecutorService()
public boolean getCollectPlatformStatistics()
true iff performance counters will be collected for the
platform on which the client is running.getCollectPlatformStatistics in interface IIndexStoreIBigdataClient.Options.COLLECT_PLATFORM_STATISTICSpublic boolean getCollectQueueStatistics()
true iff statistics will be collected for work queues.getCollectQueueStatistics in interface IIndexStoreIBigdataClient.Options.COLLECT_QUEUE_STATISTICSpublic int getHttpdPort()
getHttpdPort in interface IIndexStoreIBigdataClient.Options.HTTPD_PORTpublic final String getHttpdURL()
IBigdataFederationgetHttpdURL in interface IBigdataFederation<T>null if the httpd service is not
running.public DefaultResourceLocator<?> getResourceLocator()
IIndexStoregetResourceLocator in interface IIndexStorepublic TaskCounters getTaskCounters()
TaskCounters which aggregate across all operations
performed by the client against the connected federation. These
TaskCounters are sampled by a
ThreadPoolExecutorStatisticsTask and the sampled data are
reported by the client to the ILoadBalancerService.public ScaleOutIndexCounters getIndexCounters(String name)
ScaleOutIndexCounters for the specified scale-out index
for this client. There is only a single instance per scale-out index and
all operations by this client on that index are aggregated by that
instance. These counters are reported by the client to the
ILoadBalancerService.name - The scale-out index name.public ScheduledFuture<?> addScheduledTask(Runnable task, long initialDelay, long delay, TimeUnit unit)
IIndexStoreaddScheduledTask in interface IIndexStoretask - The task.initialDelay - The initial delay.delay - The delay between invocations.unit - The units for the delay parameters.ScheduledFuture for that task.public final BigdataGangliaService getGangliaService()
public final CounterSet getCounters()
CounterSet which the client will use report its statistics to
the ILoadBalancerService.
Note: Applications MAY add their own counters (within a suitable
namespace) to the returned CounterSet in order to report their
own performance data to the ILoadBalancerService.
Note: This method must use stateful counters because the federation
services all need to be able to report their history. If this were to
return a new CounterSet each time, then the services would not
remember any history (including the load balancer). Returning a new
object every time here basically throws away the data we want.
getCounters in interface ICounterSetAccessgetCounters in interface IBigdataFederation<T>IBigdataFederation.getServiceCounterSet(),
IBigdataFederation.getServiceCounterPathPrefix()public CounterSet getHostCounterSet()
IBigdataFederationIBigdataFederation.getCounters() corresponding to the root of the
host on which the client or service is executing.getHostCounterSet in interface IBigdataFederation<T>public CounterSet getServiceCounterSet()
IBigdataFederationIBigdataFederation.getCounters() corresponding to the root of the
client or service connected using this federation.getServiceCounterSet in interface IBigdataFederation<T>public String getServiceCounterPathPrefix()
IBigdataFederationICounterSet.pathSeparator.getServiceCounterPathPrefix in interface IBigdataFederation<T>IBigdataFederation.getServiceCounterSet()public static String getServiceCounterPathPrefix(UUID serviceUUID, Class serviceIface, String hostname)
ICounterSet.pathSeparator.serviceUUID - The service UUID.serviceIface - The primary interface or class for the service.hostname - The fully qualified name of the host on which the service is
running.public ExecutorService getExecutorService()
IBigdataFederationClientIndexView.getExecutorService in interface IIndexStoregetExecutorService in interface IBigdataFederation<T>public boolean isGroupCommit()
IBigdataFederation supports group commit (and always has). The
client side API submits tasks. Those tasks are scheduled on the
IDataService using the group commit mechanisms.isGroupCommit in interface IIndexManager(NSS GROUP COMMIT)public void registerIndex(IndexMetadata metadata)
IBigdataFederationregisterIndex in interface IGISTManagerregisterIndex in interface IBigdataFederation<T>metadata - The metadata template used to create component indices for
BTrees this scale-out index (this also specifies the
name of the scale-out index).IGISTLocalManager.getIndexLocal(String, long)public UUID registerIndex(IndexMetadata metadata, UUID dataServiceUUID)
IBigdataFederationregisterIndex in interface IBigdataFederation<T>metadata - The metadata template used to create component indices for
BTrees this scale-out index (this also specifies the
name of the scale-out index).dataServiceUUID - The data service identifier (optional). When null,
a data service will be selected automatically. If
IndexMetadata.Options#INITIAL_DATA_SERVICE was
specified, then the identified service will be used. Otherwise
an underutilized service will be selected using the
ILoadBalancerService.IndexMetadata.Options#INITIAL_DATA_SERVICEpublic UUID registerIndex(IndexMetadata metadata, byte[][] separatorKeys, UUID[] dataServiceUUIDs)
IBigdataFederationregisterIndex in interface IBigdataFederation<T>metadata - The metadata template used to create component indices for
BTrees this scale-out index (this also specifies the
name of the scale-out index).separatorKeys - The array of separator keys. Each separator key is interpreted
as an unsigned byte[]. The first entry MUST be an
empty byte[]. The entries MUST be in sorted order.dataServiceUUIDs - The array of data services onto which each partition defined
by a separator key will be mapped (optional). When given, the
#of entries in this array MUST agree with the #of entries in
the separatorKeys array and all entries must be non-null.
When not given, the index partitions will be auto-assigned to
the discovered data services.protected abstract AbstractIndexCache<? extends IClientIndex> getIndexCache()
IIndex objects.public IClientIndex getIndex(String name, long timestamp)
AbstractIndexCache and strengthens the return type.
Obtain a view on a partitioned index.getIndex in interface IIndexManagergetIndex in interface IBigdataFederation<T>name - The index name.timestamp - A transaction identifier, ITx.UNISOLATED for the
unisolated index view, ITx.READ_COMMITTED, or
timestamp for a historical view no later than
the specified timestamp.null if the index does not exist.IGISTLocalManager.getIndexLocal(String, long)public void dropIndex(String name)
IBigdataFederationdropIndex in interface IGISTManagerdropIndex in interface IBigdataFederation<T>name - The index name.public Iterator<String> indexNameScan(String prefix, long timestamp)
Note: This implementation fully buffers the namespace scan.
indexNameScan in interface IGISTManagerprefix - The prefix (optional). When given, this MUST include a
. if you want to restrict the scan to only those
indices in a given namespace. Otherwise you can find indices
in kb2 if you provide the prefix kb
where both kb and kb2 are namespaces since the indices spanned
by kb would include both kb.xyz and
kb2.xyx.timestamp - A timestamp which represents either a possible commit time on
the store or a read-only transaction identifier.public SparseRowStore getGlobalRowStore()
IIndexStoreSparseRowStore used to
store named property sets.getGlobalRowStore in interface IIndexStoreGlobalRowStoreSchemapublic SparseRowStore getGlobalRowStore(long timestamp)
IIndexStoreSparseRowStore used to store named
property sets as of the specified timestamp.
The SparseRowStore only permits ITx.UNISOLATED writes, so
you MUST specify ITx.UNISOLATED as the timestamp if you intend to
write on the global row store!
You can request the most recent committed state of the global row store
by specifying ITx.READ_COMMITTED.
getGlobalRowStore in interface IIndexStoretimestamp - The timestamp of the view.null if no view
exists as of that timestamp.public BigdataFileSystem getGlobalFileSystem()
IIndexStoregetGlobalFileSystem in interface IIndexStoreBigdataFileSystempublic TemporaryStore getTempStore()
IIndexStoreTemporaryStores. TemporaryStores are
thread-safe and may be used by multiple processes at once. Old
TemporaryStores are eventually retired by the factory and their
storage is reclaimed once they are finalized (after they are no longer in
use by any process). The decision to retire a TemporaryStore is
either made implicitly, when it is no longer weakly reachable, or
explicitly, when it has grown large enough that no new processes should
begin using that TemporaryStore. In the latter case, the
TemporaryStore will remain available to the process(es) using it
and a new TemporaryStore will be allocated and made available to
the caller.
It is important that processes do not hold a hard reference to a
TemporaryStore beyond the end of the process as that will prevent
the TemporaryStore from being finalized. Holding reference to an
AbstractBTree created on a TemporaryStore is equivalent
to holding a hard reference to the TemporaryStore itself since
the AbstractBTree holds onto the backing IRawStore using
a hard reference.
getTempStore in interface IIndexStoreTemporaryStore.public void reportCounters()
CounterSet to the
ILoadBalancerService. Any errors will be logged, not thrown.public T getService()
getService in interface IFederationDelegate<T>public String getServiceName()
getServiceName in interface IFederationDelegate<T>public Class<?> getServiceIface()
ILoadBalancerService.getServiceIface in interface IFederationDelegate<T>null.public UUID getServiceUUID()
getServiceUUID in interface IFederationDelegate<T>AbstractService.setServiceUUID(UUID)public boolean isServiceReady()
true iff the service is ready to start.isServiceReady in interface IFederationDelegate<T>public void reattachDynamicCounters()
ILoadBalancerService. In general, implementations should limit
the frequency of update, e.g., to no more than once a second.
Note: For most purposes, this has been replaced by
ICounterSetAccess which is now passed into
CounterSetHTTPD. That provides the necessary indirection for
periodic refresh of the performance counters. The CounterSetHTTPD
now also handles the limitation on the update frequency for the
materialized counters.
However, there are still some counters which need to be dynamically reattached. For example, any counter set which is dynamic in its structure, such as the DirectBufferPool.
reattachDynamicCounters in interface IFederationDelegate<T>public void didStart()
AbstractFederation once the deferred startup tasks
are executed. Services may use this event to perform additional
initialization.didStart in interface IFederationDelegate<T>public AbstractHTTPD newHttpd(int httpdPort, ICounterSetAccess accessor) throws IOException
AbstractHTTPD instance.newHttpd in interface IFederationDelegate<T>accessor - Used to materialize the CounterSet that will be served
up.IOExceptionpublic void serviceJoin(IService service, UUID serviceUUID)
IBigdataClient.serviceJoin in interface IFederationDelegate<T>service - The service.serviceUUID - The service UUID.public void serviceLeave(UUID serviceUUID)
IBigdataClient when the service is no
longer available from any of its service registrars.serviceLeave in interface IFederationDelegate<T>serviceUUID - The service UUID.public IDataService[] getDataServices(UUID[] uuids)
IBigdataFederationIDataService references that is correlated
with the given array of IDataService UUIDs.
Note: This method will also resolve the UUID of an
IMetadataService.
getDataServices in interface IBigdataFederation<T>protected void sendEvent(Event e)
ILoadBalancerService.
Events are maintained on a non-blocking queue (no fixed capacity) and
sent by a scheduled task.e - SendEventsTaskCopyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.