T
- The generic type of the client or service.public abstract class AbstractFederation<T> extends Object implements IBigdataFederation<T>
IBigdataFederation
implementations.Modifier and Type | Class and Description |
---|---|
static class |
AbstractFederation.ReportTask
Periodically report performance counter data to the
ILoadBalancerService . |
protected class |
AbstractFederation.StartDeferredTasksTask
This task starts an (optional)
AbstractStatisticsCollector , an
(optional) httpd service, and the (required) AbstractFederation.ReportTask . |
Modifier and Type | Field and Description |
---|---|
protected static org.apache.log4j.Logger |
log |
Modifier | Constructor and Description |
---|---|
protected |
AbstractFederation(IBigdataClient<T> client) |
Modifier and Type | Method and Description |
---|---|
ScheduledFuture<?> |
addScheduledTask(Runnable task,
long initialDelay,
long delay,
TimeUnit unit)
Adds a task which will run until canceled, until it throws an exception,
or until the service is shutdown.
|
protected void |
assertOpen() |
void |
destroy()
Destroys all discovered services belonging to the federation and their
persistent data and disconnects from the federation.
|
void |
didStart()
Delegated.
|
void |
dropIndex(String name)
Drop a scale-out index.
|
AbstractClient<T> |
getClient()
Return the client object that was used to connect to the federation.
|
boolean |
getCollectPlatformStatistics()
true iff performance counters will be collected for the
platform on which the client is running. |
boolean |
getCollectQueueStatistics()
true iff statistics will be collected for work queues. |
CounterSet |
getCounters()
The
CounterSet which the client will use report its statistics to
the ILoadBalancerService . |
IDataService[] |
getDataServices(UUID[] uuids)
Return an array of
IDataService references that is correlated
with the given array of IDataService UUID s. |
ExecutorService |
getExecutorService()
A thread pool that may be used by clients to parallelize operations
against the federation.
|
BigdataGangliaService |
getGangliaService()
The embedded ganglia peer.
|
BigdataFileSystem |
getGlobalFileSystem()
Return the global file system used to store block-structured files and
their metadata and as a source and sink for map/reduce processing.
|
SparseRowStore |
getGlobalRowStore()
Return an unisolated view of the global
SparseRowStore used to
store named property sets. |
SparseRowStore |
getGlobalRowStore(long timestamp)
Return a view of the global
SparseRowStore used to store named
property sets as of the specified timestamp. |
CounterSet |
getHostCounterSet()
The node in
IBigdataFederation.getCounters() corresponding to the root of the
host on which the client or service is executing. |
int |
getHttpdPort()
The port on which the optional httpd service will be run.
|
String |
getHttpdURL()
The URL that may be used to access the local httpd service for this
client or service.
|
IClientIndex |
getIndex(String name,
long timestamp)
Applies an
AbstractIndexCache and strengthens the return type. |
protected abstract AbstractIndexCache<? extends IClientIndex> |
getIndexCache()
Return the cache for
IIndex objects. |
ScaleOutIndexCounters |
getIndexCounters(String name)
Return the
ScaleOutIndexCounters for the specified scale-out index
for this client. |
DefaultResourceLocator<?> |
getResourceLocator()
Return the default locator for resources that are logical index
containers (relations and relation containers).
|
ScheduledExecutorService |
getScheduledExecutorService()
A service which may be used to schedule performance counter sampling
tasks.
|
T |
getService()
Delegated.
|
String |
getServiceCounterPathPrefix()
The path prefix under which all of the client or service's counters are
located.
|
static String |
getServiceCounterPathPrefix(UUID serviceUUID,
Class serviceIface,
String hostname)
The path prefix under which all of the client or service's counters are
located.
|
CounterSet |
getServiceCounterSet()
The node in
IBigdataFederation.getCounters() corresponding to the root of the
client or service connected using this federation. |
Class<?> |
getServiceIface()
Delegated.
|
String |
getServiceName()
Delegated.
|
UUID |
getServiceUUID()
Delegated.
|
TaskCounters |
getTaskCounters()
Return the
TaskCounters which aggregate across all operations
performed by the client against the connected federation. |
TemporaryStore |
getTempStore()
A factory for
TemporaryStore s. |
Iterator<String> |
indexNameScan(String prefix,
long timestamp)
Iterator visits the names of all indices spanned by the given prefix.
|
boolean |
isGroupCommit()
The
IBigdataFederation supports group commit (and always has). |
boolean |
isOpen() |
boolean |
isServiceReady()
Delegated.
|
AbstractHTTPD |
newHttpd(int httpdPort,
ICounterSetAccess accessor)
Delegated.
|
void |
reattachDynamicCounters()
Delegated.
|
void |
registerIndex(IndexMetadata metadata)
Register a scale-out index.
|
UUID |
registerIndex(IndexMetadata metadata,
byte[][] separatorKeys,
UUID[] dataServiceUUIDs)
Register and statically partition a scale-out index.
|
UUID |
registerIndex(IndexMetadata metadata,
UUID dataServiceUUID)
Register a scale-out index and assign the initial index partition to the
specified data service.
|
void |
reportCounters()
Forces the immediate reporting of the
CounterSet to the
ILoadBalancerService . |
protected void |
sendEvent(Event e)
Queues up an event to be sent to the
ILoadBalancerService . |
void |
serviceJoin(IService service,
UUID serviceUUID)
Delegated.
|
void |
serviceLeave(UUID serviceUUID)
Delegated.
|
void |
shutdown()
Normal shutdown allows any existing client requests to federation
services to complete but does not schedule new requests, disconnects from
the federation, and then terminates any background processing that is
being performed on the behalf of the client (service discovery, etc).
|
void |
shutdownNow()
Immediate shutdown terminates any client requests to federation services,
disconnects from the federation, and then terminate any background
processing that is being performed on the behalf of the client (service
discovery, etc).
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getAnyDataService, getDataService, getDataServiceByName, getDataServiceUUIDs, getLastCommitTime, getLoadBalancerService, getMetadataIndex, getMetadataService, getTransactionService, isDistributed, isJiniFederation, isScaleOut, isStable
getResourceLockService
protected AbstractFederation(IBigdataClient<T> client)
public AbstractClient<T> getClient()
IBigdataFederation
getClient
in interface IBigdataFederation<T>
public final boolean isOpen()
public void shutdown()
Note: concrete implementations MUST extend this method.
Note: Clients use IBigdataClient.disconnect(boolean)
to
disconnect from a federation. The federation implements that disconnect
using either shutdown()
or shutdownNow()
.
The implementation must be a NOP if the federation is already shutdown.
public void shutdownNow()
Note: concrete implementations MUST extend this method to either
disconnect from the remote federation or close the embedded federation
and then clear the #fed
reference so that the client is no longer
"connected" to the federation.
Note: Clients use IBigdataClient.disconnect(boolean)
to disconnect
from a federation. The federation implements that disconnect using either
shutdown()
or shutdownNow()
.
The implementation must be a NOP if the federation is already shutdown.
public void destroy()
IBigdataFederation
destroy
in interface IIndexStore
destroy
in interface IBigdataFederation<T>
protected final void assertOpen()
IllegalStateException
- if the client has disconnected from the federation.public ScheduledExecutorService getScheduledExecutorService()
public boolean getCollectPlatformStatistics()
true
iff performance counters will be collected for the
platform on which the client is running.getCollectPlatformStatistics
in interface IIndexStore
IBigdataClient.Options.COLLECT_PLATFORM_STATISTICS
public boolean getCollectQueueStatistics()
true
iff statistics will be collected for work queues.getCollectQueueStatistics
in interface IIndexStore
IBigdataClient.Options.COLLECT_QUEUE_STATISTICS
public int getHttpdPort()
getHttpdPort
in interface IIndexStore
IBigdataClient.Options.HTTPD_PORT
public final String getHttpdURL()
IBigdataFederation
getHttpdURL
in interface IBigdataFederation<T>
null
if the httpd service is not
running.public DefaultResourceLocator<?> getResourceLocator()
IIndexStore
getResourceLocator
in interface IIndexStore
public TaskCounters getTaskCounters()
TaskCounters
which aggregate across all operations
performed by the client against the connected federation. These
TaskCounters
are sampled by a
ThreadPoolExecutorStatisticsTask
and the sampled data are
reported by the client to the ILoadBalancerService
.public ScaleOutIndexCounters getIndexCounters(String name)
ScaleOutIndexCounters
for the specified scale-out index
for this client. There is only a single instance per scale-out index and
all operations by this client on that index are aggregated by that
instance. These counters are reported by the client to the
ILoadBalancerService
.name
- The scale-out index name.public ScheduledFuture<?> addScheduledTask(Runnable task, long initialDelay, long delay, TimeUnit unit)
IIndexStore
addScheduledTask
in interface IIndexStore
task
- The task.initialDelay
- The initial delay.delay
- The delay between invocations.unit
- The units for the delay parameters.ScheduledFuture
for that task.public final BigdataGangliaService getGangliaService()
public final CounterSet getCounters()
CounterSet
which the client will use report its statistics to
the ILoadBalancerService
.
Note: Applications MAY add their own counters (within a suitable
namespace) to the returned CounterSet
in order to report their
own performance data to the ILoadBalancerService
.
Note: This method must use stateful counters because the federation
services all need to be able to report their history. If this were to
return a new CounterSet
each time, then the services would not
remember any history (including the load balancer). Returning a new
object every time here basically throws away the data we want.
getCounters
in interface ICounterSetAccess
getCounters
in interface IBigdataFederation<T>
IBigdataFederation.getServiceCounterSet()
,
IBigdataFederation.getServiceCounterPathPrefix()
public CounterSet getHostCounterSet()
IBigdataFederation
IBigdataFederation.getCounters()
corresponding to the root of the
host on which the client or service is executing.getHostCounterSet
in interface IBigdataFederation<T>
public CounterSet getServiceCounterSet()
IBigdataFederation
IBigdataFederation.getCounters()
corresponding to the root of the
client or service connected using this federation.getServiceCounterSet
in interface IBigdataFederation<T>
public String getServiceCounterPathPrefix()
IBigdataFederation
ICounterSet.pathSeparator
.getServiceCounterPathPrefix
in interface IBigdataFederation<T>
IBigdataFederation.getServiceCounterSet()
public static String getServiceCounterPathPrefix(UUID serviceUUID, Class serviceIface, String hostname)
ICounterSet.pathSeparator
.serviceUUID
- The service UUID
.serviceIface
- The primary interface or class for the service.hostname
- The fully qualified name of the host on which the service is
running.public ExecutorService getExecutorService()
IBigdataFederation
ClientIndexView
.getExecutorService
in interface IIndexStore
getExecutorService
in interface IBigdataFederation<T>
public boolean isGroupCommit()
IBigdataFederation
supports group commit (and always has). The
client side API submits tasks. Those tasks are scheduled on the
IDataService
using the group commit mechanisms.isGroupCommit
in interface IIndexManager
(NSS GROUP COMMIT)
public void registerIndex(IndexMetadata metadata)
IBigdataFederation
registerIndex
in interface IGISTManager
registerIndex
in interface IBigdataFederation<T>
metadata
- The metadata template used to create component indices for
BTree
s this scale-out index (this also specifies the
name of the scale-out index).IGISTLocalManager.getIndexLocal(String, long)
public UUID registerIndex(IndexMetadata metadata, UUID dataServiceUUID)
IBigdataFederation
registerIndex
in interface IBigdataFederation<T>
metadata
- The metadata template used to create component indices for
BTree
s this scale-out index (this also specifies the
name of the scale-out index).dataServiceUUID
- The data service identifier (optional). When null
,
a data service will be selected automatically. If
IndexMetadata.Options#INITIAL_DATA_SERVICE
was
specified, then the identified service will be used. Otherwise
an underutilized service will be selected using the
ILoadBalancerService
.IndexMetadata.Options#INITIAL_DATA_SERVICE
public UUID registerIndex(IndexMetadata metadata, byte[][] separatorKeys, UUID[] dataServiceUUIDs)
IBigdataFederation
registerIndex
in interface IBigdataFederation<T>
metadata
- The metadata template used to create component indices for
BTree
s this scale-out index (this also specifies the
name of the scale-out index).separatorKeys
- The array of separator keys. Each separator key is interpreted
as an unsigned byte[]. The first entry MUST be an
empty byte[]. The entries MUST be in sorted order.dataServiceUUIDs
- The array of data services onto which each partition defined
by a separator key will be mapped (optional). When given, the
#of entries in this array MUST agree with the #of entries in
the separatorKeys array and all entries must be non-null
.
When not given, the index partitions will be auto-assigned to
the discovered data services.protected abstract AbstractIndexCache<? extends IClientIndex> getIndexCache()
IIndex
objects.public IClientIndex getIndex(String name, long timestamp)
AbstractIndexCache
and strengthens the return type.
Obtain a view on a partitioned index.getIndex
in interface IIndexManager
getIndex
in interface IBigdataFederation<T>
name
- The index name.timestamp
- A transaction identifier, ITx.UNISOLATED
for the
unisolated index view, ITx.READ_COMMITTED
, or
timestamp
for a historical view no later than
the specified timestamp.null
if the index does not exist.IGISTLocalManager.getIndexLocal(String, long)
public void dropIndex(String name)
IBigdataFederation
dropIndex
in interface IGISTManager
dropIndex
in interface IBigdataFederation<T>
name
- The index name.public Iterator<String> indexNameScan(String prefix, long timestamp)
Note: This implementation fully buffers the namespace scan.
indexNameScan
in interface IGISTManager
prefix
- The prefix (optional). When given, this MUST include a
.
if you want to restrict the scan to only those
indices in a given namespace. Otherwise you can find indices
in kb2
if you provide the prefix kb
where both kb and kb2 are namespaces since the indices spanned
by kb
would include both kb.xyz
and
kb2.xyx
.timestamp
- A timestamp which represents either a possible commit time on
the store or a read-only transaction identifier.public SparseRowStore getGlobalRowStore()
IIndexStore
SparseRowStore
used to
store named property sets.getGlobalRowStore
in interface IIndexStore
GlobalRowStoreSchema
public SparseRowStore getGlobalRowStore(long timestamp)
IIndexStore
SparseRowStore
used to store named
property sets as of the specified timestamp.
The SparseRowStore
only permits ITx.UNISOLATED
writes, so
you MUST specify ITx.UNISOLATED
as the timestamp if you intend to
write on the global row store!
You can request the most recent committed state of the global row store
by specifying ITx.READ_COMMITTED
.
getGlobalRowStore
in interface IIndexStore
timestamp
- The timestamp of the view.null
if no view
exists as of that timestamp.public BigdataFileSystem getGlobalFileSystem()
IIndexStore
getGlobalFileSystem
in interface IIndexStore
BigdataFileSystem
public TemporaryStore getTempStore()
IIndexStore
TemporaryStore
s. TemporaryStore
s are
thread-safe and may be used by multiple processes at once. Old
TemporaryStore
s are eventually retired by the factory and their
storage is reclaimed once they are finalized (after they are no longer in
use by any process). The decision to retire a TemporaryStore
is
either made implicitly, when it is no longer weakly reachable, or
explicitly, when it has grown large enough that no new processes should
begin using that TemporaryStore
. In the latter case, the
TemporaryStore
will remain available to the process(es) using it
and a new TemporaryStore
will be allocated and made available to
the caller.
It is important that processes do not hold a hard reference to a
TemporaryStore
beyond the end of the process as that will prevent
the TemporaryStore
from being finalized. Holding reference to an
AbstractBTree
created on a TemporaryStore
is equivalent
to holding a hard reference to the TemporaryStore
itself since
the AbstractBTree
holds onto the backing IRawStore
using
a hard reference.
getTempStore
in interface IIndexStore
TemporaryStore
.public void reportCounters()
CounterSet
to the
ILoadBalancerService
. Any errors will be logged, not thrown.public T getService()
getService
in interface IFederationDelegate<T>
public String getServiceName()
getServiceName
in interface IFederationDelegate<T>
public Class<?> getServiceIface()
ILoadBalancerService
.getServiceIface
in interface IFederationDelegate<T>
null
.public UUID getServiceUUID()
getServiceUUID
in interface IFederationDelegate<T>
AbstractService.setServiceUUID(UUID)
public boolean isServiceReady()
true
iff the service is ready to start.isServiceReady
in interface IFederationDelegate<T>
public void reattachDynamicCounters()
ILoadBalancerService
. In general, implementations should limit
the frequency of update, e.g., to no more than once a second.
Note: For most purposes, this has been replaced by
ICounterSetAccess
which is now passed into
CounterSetHTTPD
. That provides the necessary indirection for
periodic refresh of the performance counters. The CounterSetHTTPD
now also handles the limitation on the update frequency for the
materialized counters.
However, there are still some counters which need to be dynamically reattached. For example, any counter set which is dynamic in its structure, such as the DirectBufferPool.
reattachDynamicCounters
in interface IFederationDelegate<T>
public void didStart()
AbstractFederation
once the deferred startup tasks
are executed. Services may use this event to perform additional
initialization.didStart
in interface IFederationDelegate<T>
public AbstractHTTPD newHttpd(int httpdPort, ICounterSetAccess accessor) throws IOException
AbstractHTTPD
instance.newHttpd
in interface IFederationDelegate<T>
accessor
- Used to materialize the CounterSet
that will be served
up.IOException
public void serviceJoin(IService service, UUID serviceUUID)
IBigdataClient
.serviceJoin
in interface IFederationDelegate<T>
service
- The service.serviceUUID
- The service UUID
.public void serviceLeave(UUID serviceUUID)
IBigdataClient
when the service is no
longer available from any of its service registrars.serviceLeave
in interface IFederationDelegate<T>
serviceUUID
- The service UUID
.public IDataService[] getDataServices(UUID[] uuids)
IBigdataFederation
IDataService
references that is correlated
with the given array of IDataService
UUID
s.
Note: This method will also resolve the UUID
of an
IMetadataService
.
getDataServices
in interface IBigdataFederation<T>
protected void sendEvent(Event e)
ILoadBalancerService
.
Events are maintained on a non-blocking queue (no fixed capacity) and
sent by a scheduled task.e
- SendEventsTask
Copyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.