com.bigdata.bop.fed

Class FederatedQueryEngine

java.lang.Object

com.bigdata.bop.engine.QueryEngine

com.bigdata.bop.fed.FederatedQueryEngine

All Implemented Interfaces:

IQueryClient, IQueryPeer, ICounterSetAccess, Remote

public class FederatedQueryEngine
extends QueryEngine

An IBigdataFederation aware QueryEngine.

Author:

Bryan Thompson

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	FederatedQueryEngine.Annotations Annotations understood by the QueryEngine.

Nested classes/interfaces inherited from class com.bigdata.bop.engine.QueryEngine

QueryEngine.IRunningQueryListener

Field Summary

Fields inherited from class com.bigdata.bop.engine.QueryEngine

counters, ERR_QUERY_NOT_RUNNING, geoSpatialCounters

Constructor Summary

Constructors

Constructor and Description
FederatedQueryEngine(DataService dataService) Constructor used on a DataService (a query engine peer).
FederatedQueryEngine(UUID thisService, IBigdataFederation<?> fed, IIndexManager indexManager, ManagedResourceService resourceService) Constructor used on a non-DataService node to expose a query controller.

Method Summary

Methods

Modifier and Type	Method and Description
void	bufferReady(IChunkMessage<IBindingSet> msg) Notify a service that a buffer having data for some BOp in some running query is available.
void	cancelQuery(UUID queryId, Throwable cause) Overridden to cancel all running operators for the query on this node.
void	declareQuery(IQueryDecl queryDecl) Deprecated.
protected void	didShutdown() Hook is notified by QueryEngine.shutdown() when all running queries have terminated.
IBigdataFederation<?>	getFederation() The IBigdataFederation iff running in scale-out.
IQueryClient	getProxy() Overridden to return an RMI proxy for this FederatedQueryEngine.
protected FederatedQueryEngineCounters	getQueryEngineCounters() The QueryEngineCounters object for this QueryEngine.
protected IQueryPeer	getQueryPeer(UUID serviceUUID) Resolve an IQueryPeer.
ManagedResourceService	getResourceService() The service used to expose ByteBuffers and managed index resources for transfer to remote services in support of distributed query evaluation.
FederatedRunningQuery	getRunningQuery(UUID queryId) Overridden to strengthen the return type.
UUID	getServiceUUID() The UUID of the service in which this QueryEngine is running.
void	init() Initialize the QueryEngine.
boolean	isDataService() Return true iff the query engine instance is hosted by a DataService.
boolean	isScaleOut() Return true iff running against an IBigdataFederation.
protected FederatedQueryEngineCounters	newCounters() Extension hook for new QueryEngineCounters instances.
protected FederatedRunningQuery	newRunningQuery(UUID queryId, boolean controller, IQueryClient clientProxy, UUID queryControllerId, PipelineOp query, IChunkMessage<IBindingSet> realSource) Factory for IRunningQuerys.
void	shutdownNow() Do not accept new queries and halt any running binding set chunk tasks.
String	toString()

Methods inherited from class com.bigdata.bop.engine.QueryEngine

acceptChunk, addListener, addPendingCancel, assertRunning, eval, eval, eval, eval, eval, eval, eval, eval, execute, finalize, getClientConnectionManager, getConcurrencyManager, getCounters, getGeoSpatialCounters, getIndexManager, getQuery, getRunningQueries, halt, haltOp, isRunning, newGeoSpatialCounters, pendingCancel, putIfAbsent, removeListener, shutdown, startOp

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FederatedQueryEngine

public FederatedQueryEngine(DataService dataService)

Constructor used on a DataService (a query engine peer).

Parameters:

dataService - The data service.

FederatedQueryEngine

public FederatedQueryEngine(UUID thisService,
                    IBigdataFederation<?> fed,
                    IIndexManager indexManager,
                    ManagedResourceService resourceService)

Constructor used on a non-DataService node to expose a query controller. Since the query controller is not embedded within a data service it needs to provide its own ResourceService and local IIndexManager.

Parameters:

fed -

indexManager -

resourceService -

Method Detail

getServiceUUID

public UUID getServiceUUID()

Description copied from interface: IQueryPeer

The UUID of the service in which this QueryEngine is running.

Specified by:

getServiceUUID in interface IQueryPeer

Overrides:

getServiceUUID in class QueryEngine

Returns:

The UUID of the service in which this QueryEngine is running -or- a unique and distinct UUID if the QueryEngine is not running against an IBigdataFederation.

See Also:

IService.getServiceUUID()

getFederation

public IBigdataFederation<?> getFederation()

Description copied from class: QueryEngine

The IBigdataFederation iff running in scale-out.

Note: The IBigdataFederation is required in scale-out in order to perform shard locator scans when mapping binding sets across the next join in a query plan.

Overrides:

getFederation in class QueryEngine

getResourceService

public ManagedResourceService getResourceService()

The service used to expose ByteBuffers and managed index resources for transfer to remote services in support of distributed query evaluation.

getProxy

public IQueryClient getProxy()

Overridden to return an RMI proxy for this FederatedQueryEngine.

The RMI proxy for this QueryEngine when used as a query controller. The default implementation returns this.

Overrides:

getProxy in class QueryEngine

isScaleOut

public final boolean isScaleOut()

Description copied from class: QueryEngine

Return true iff running against an IBigdataFederation.

Overrides:

isScaleOut in class QueryEngine

isDataService

public final boolean isDataService()

Return true iff the query engine instance is hosted by a DataService.

getRunningQuery

public FederatedRunningQuery getRunningQuery(UUID queryId)

Overridden to strengthen the return type.

Return the AbstractRunningQuery associated with that query identifier.

Overrides:

getRunningQuery in class QueryEngine

Parameters:

queryId - The query identifier.

Returns:

The AbstractRunningQuery -or- null if there is no query associated with that query identifier.

toString

public String toString()

Overrides:

toString in class Object

init

public void init()

Initialize the QueryEngine. It will accept binding set chunks and run them against running queries until it is shutdown.

Extended to also initialize a thread which will materialize IChunkMessage for consumption by this node. TODO The acceptTaskService is not used right now since we are always running the MaterializeMessageTask in the caller's thread. If it becomes used, then we should reconsider the pool size.

Overrides:

init in class QueryEngine

See Also:

acceptTaskService pool size on cluster

didShutdown

protected void didShutdown()

Hook is notified by QueryEngine.shutdown() when all running queries have terminated.

Extended to stop materializing chunks once all running queries are done.

Overrides:

didShutdown in class QueryEngine

shutdownNow

public void shutdownNow()

Do not accept new queries and halt any running binding set chunk tasks.

Extended to stop materializing chunks.

Overrides:

shutdownNow in class QueryEngine

declareQuery

@Deprecated
public void declareQuery(IQueryDecl queryDecl)
                  throws RemoteException

Deprecated.

Description copied from interface: IQueryPeer

Declare a query to a peer. This message is sent to the peer before any other message for that query and declares the query and the query controller with which the peer must communicate during query evaluation.

Specified by:

declareQuery in interface IQueryPeer

Overrides:

declareQuery in class QueryEngine

Parameters:

queryDecl - The query declaration.

Throws:

RemoteException

bufferReady

public void bufferReady(IChunkMessage<IBindingSet> msg)

Notify a service that a buffer having data for some BOp in some running query is available. The receiver may request the data when they are ready. If the query is cancelled, then the sender will drop the buffer. TODO The timing and responsibility for materializing chunks needs to be examined further when the data are being moved around using NIO rather than ThickChunkMessage. At stake is when the intermediate solutions are materialized on the node where they will be consumed. We can either do this synchronous or asynchronously when bufferReady() is caller or we can defer the transfer until the target operator on this node is ready to run.

Specified by:

bufferReady in interface IQueryPeer

Overrides:

bufferReady in class QueryEngine

Parameters:

msg - The message.

See Also:

acceptTaskService pool size on cluster , unselective joins on cluster

cancelQuery

public void cancelQuery(UUID queryId,
               Throwable cause)

Overridden to cancel all running operators for the query on this node.

Notify a service that the query has been terminated. The peer MUST NOT cancel the query synchronously as that can lead to a deadlock with the query controller. Instead, the peer should queue a task to cancel the query and then return.

The default implementation is a NOP.

Specified by:

cancelQuery in interface IQueryPeer

Overrides:

cancelQuery in class QueryEngine

Parameters:

queryId - The query identifier.

cause - The cause. When null, this is presumed to be normal query termination.

newRunningQuery

protected FederatedRunningQuery newRunningQuery(UUID queryId,
                                    boolean controller,
                                    IQueryClient clientProxy,
                                    UUID queryControllerId,
                                    PipelineOp query,
                                    IChunkMessage<IBindingSet> realSource)

Factory for IRunningQuerys.

Overridden to always use a FederatedRunningQuery.

Overrides:

newRunningQuery in class QueryEngine

See Also:

QueryEngine.Annotations.RUNNING_QUERY_CLASS

getQueryPeer

protected IQueryPeer getQueryPeer(UUID serviceUUID)

Resolve an IQueryPeer.

Note: This only resolves the peers running on the IDataServices. It will not resolve a query controller unless an IDataService is being used as the query controller.

Parameters:

serviceUUID - The service UUID.

Returns:

The proxy for the query peer.

newCounters

protected FederatedQueryEngineCounters newCounters()

Extension hook for new QueryEngineCounters instances.

Overrides:

newCounters in class QueryEngine

getQueryEngineCounters

protected FederatedQueryEngineCounters getQueryEngineCounters()

The QueryEngineCounters object for this QueryEngine.

Overridden to strengthen the return type.

Overrides:

getQueryEngineCounters in class QueryEngine