com.bigdata.bop

Interface PipelineOp.Annotations

All Superinterfaces:

BOp.Annotations, BufferAnnotations

All Known Subinterfaces:

AbstractAddRemoveStatementsOp.Annotations, AbstractSubqueryOp.Annotations, AccessPathJoinAnnotations, ArbitraryLengthPathOp.Annotations, ChunkedMaterializationOp.Annotations, ChunkedResolutionOp.Annotations, CommitOp.Annotations, ConditionalRoutingOp.Annotations, CopyOp.Annotations, DataSetJoin.Annotations, DistinctTermScanOp.Annotations, DropOp.Annotations, FastRangeCountOp.Annotations, FederatedQueryEngine.Annotations, GroupByOp.Annotations, HashIndexOp.Annotations, HashIndexOpBase.Annotations, HashJoinOp.Annotations, HTreeDistinctBindingSetsOp.Annotations, HTreeHashJoinAnnotations, HTreeHashJoinOp.Annotations, HTreeMergeJoin.Annotations, HTreeNamedSubqueryOp.Annotations, InlineMaterializeOp.Annotations, InsertOp.Annotations, IsInlineBOp.Annotations, JoinAnnotations, JoinGraph.Annotations, JVMDistinctBindingSetsOp.Annotations, JVMHashJoinAnnotations, JVMHashJoinOp.Annotations, JVMMergeJoin.Annotations, JVMNamedSubqueryOp.Annotations, MemoryGroupByOp.Annotations, MockTermResolverOp.Annotations, NestedLoopJoinOp.Annotations, ParseOp.Annotations, PipelinedAggregationOp.Annotations, PipelineDelayOp.Annotations, PipelinedHashIndexAndSolutionSetJoinOp.Annotations, PipelineJoin.Annotations, ProjectionOp.Annotations, QueryEngine.Annotations, ServiceCallJoin.Annotations, SliceOp.Annotations, SolutionSetHashJoinOp.Annotations, SortOp.Annotations, SubqueryJoinAnnotations, SubqueryOp.Annotations, VariableUnificationOp.Annotations, ZeroLengthPathOp.Annotations

Enclosing class:

PipelineOp

public static interface PipelineOp.Annotations
extends BOp.Annotations, BufferAnnotations

Field Summary

Fields

Modifier and Type	Field and Description
static String	ALT_SINK_REF The value of the annotation is the BOp.Annotations.BOP_ID of the ancestor in the operator tree which serves as the alternative sink for binding sets (default is no alternative sink).
static boolean	DEFAULT_LAST_PASS
static long	DEFAULT_MAX_MEMORY
static int	DEFAULT_MAX_MESSAGES_PER_TASK
static int	DEFAULT_MAX_PARALLEL
static int	DEFAULT_PIPELINE_QUEUE_CAPACITY
static boolean	DEFAULT_PIPELINED
static boolean	DEFAULT_REORDER_SOLUTIONS
static boolean	DEFAULT_SHARED_STATE
static String	LAST_PASS When true a final evaluation pass will be invoked once it is know that the operator can not be re-triggered by another IChunkMessage.
static String	MAX_MEMORY This annotation is only used for non-PIPELINED operators and specifies the maximum #of bytes which the operator may buffer on the native heap before evaluation of the operator is triggered (default 0L).
static String	MAX_MESSAGES_PER_TASK For a pipelined operator, this is the maximum number of messages that will be assigned to a single invocation of the evaluation task for that operator (default 10).
static String	MAX_PARALLEL This option may be used to place an optional limit on the #of concurrent tasks which may run for the same (bopId,shardId) for a given query (default 5).
static String	PIPELINE_QUEUE_CAPACITY For pipelined operators, this is the capacity of the input queue for that operator.
static String	PIPELINED Annotation used to mark pipelined (aka vectored) operators.
static String	REORDER_SOLUTIONS When true, the QueryEngine MAY reorder the solutions as they flow through the query plan (this is done as a throughput optimization).
static String	SHARED_STATE The value reported by PipelineOp.isSharedState() (default false).
static String	SINK_REF The value of the annotation is the BOp.Annotations.BOP_ID of the ancestor in the operator tree which serves as the default sink for binding sets (optional, default is the parent).

Fields inherited from interface com.bigdata.bop.BOp.Annotations

BOP_ID, CONTROLLER, DEFAULT_CONTROLLER, DEFAULT_EVALUATION_CONTEXT, DEFAULT_TIMEOUT, EVALUATION_CONTEXT, NAMESPACE, TIMEOUT

Fields inherited from interface com.bigdata.bop.BufferAnnotations

CHUNK_CAPACITY, CHUNK_OF_CHUNKS_CAPACITY, CHUNK_TIMEOUT, chunkTimeoutUnit, DEFAULT_CHUNK_CAPACITY, DEFAULT_CHUNK_OF_CHUNKS_CAPACITY, DEFAULT_CHUNK_TIMEOUT

Field Detail

SINK_REF

static final String SINK_REF

The value of the annotation is the BOp.Annotations.BOP_ID of the ancestor in the operator tree which serves as the default sink for binding sets (optional, default is the parent).

See Also:

BOpUtility.getEffectiveDefaultSink(BOp, BOp)

ALT_SINK_REF

static final String ALT_SINK_REF

The value of the annotation is the BOp.Annotations.BOP_ID of the ancestor in the operator tree which serves as the alternative sink for binding sets (default is no alternative sink).

Note: JOIN and SUBQUERY operators will route optional solutions to the altSink if the altSink is specified and to the default sink otherwise.

SHARED_STATE

static final String SHARED_STATE

The value reported by PipelineOp.isSharedState() (default false). This may be overridden to true to have instances of operators evaluated in the same query engine context share the same BOpStats instance.

Note: BOp.getEvaluationContext() MUST be overridden to return BOpEvaluationContext.CONTROLLER if this annotation is overridden to true.

When true, the QueryEngine will impose the necessary constraints when the operator is evaluated.

See Also:

IQueryContext

DEFAULT_SHARED_STATE

static final boolean DEFAULT_SHARED_STATE

See Also:

Constant Field Values

REORDER_SOLUTIONS

static final String REORDER_SOLUTIONS

When true, the QueryEngine MAY reorder the solutions as they flow through the query plan (this is done as a throughput optimization). When false, the QueryEngine MUST NOT reorder solutions.

See Also:

Solution order not always preserved.

DEFAULT_REORDER_SOLUTIONS

static final boolean DEFAULT_REORDER_SOLUTIONS

See Also:

Constant Field Values

MAX_PARALLEL

static final String MAX_PARALLEL

This option may be used to place an optional limit on the #of concurrent tasks which may run for the same (bopId,shardId) for a given query (default 5). The query is guaranteed to make progress as long as this is some positive integer. While limiting this value can limit the concurrency with which certain operators are evaluated and that can have a negative effect on the throughput, it controls both the demand on the JVM heap and the #of threads consumed.

Note: MAX_PARALLEL is the annotation for pipelined joins which has the strongest effect on performance. Changes to both MAX_MESSAGES_PER_TASK and PIPELINE_QUEUE_CAPACITY have less effect and performance tends to be best around a modest value (10) for those annotations.

See Also:

ISingleThreadedOp

DEFAULT_MAX_PARALLEL

static final int DEFAULT_MAX_PARALLEL

See Also:

MAX_PARALLEL, Constant Field Values

MAX_MESSAGES_PER_TASK

static final String MAX_MESSAGES_PER_TASK

For a pipelined operator, this is the maximum number of messages that will be assigned to a single invocation of the evaluation task for that operator (default 10). By default the QueryEngine MAY (and generally does) combine multiple IChunkMessages from the work queue of an operator for each evaluation pass made for that operator. When ONE (1), each IChunkMessage will be assigned to a new evaluation task for the operator. The value of this annotation must be a positive integer. If the operator is not-pipelined, then the maximum amount of data to be assigned to an evaluation task is governed by MAX_MEMORY instead.

DEFAULT_MAX_MESSAGES_PER_TASK

static final int DEFAULT_MAX_MESSAGES_PER_TASK

See Also:

MAX_MESSAGES_PER_TASK, Constant Field Values

PIPELINE_QUEUE_CAPACITY

static final String PIPELINE_QUEUE_CAPACITY

For pipelined operators, this is the capacity of the input queue for that operator. Producers will block if the input queue for the target operator is at its capacity. This provides an important limit on the amount of data which can be buffered on the JVM heap during pipelined query evaluation.

DEFAULT_PIPELINE_QUEUE_CAPACITY

static final int DEFAULT_PIPELINE_QUEUE_CAPACITY

See Also:

PIPELINE_QUEUE_CAPACITY, Constant Field Values

PIPELINED

static final String PIPELINED

Annotation used to mark pipelined (aka vectored) operators. When false the operator will use either "at-once" or "blocked" evaluation depending on how it buffers its data for evaluation.

See Also:

PipelineOp.isPipelinedEvaluation()

DEFAULT_PIPELINED

static final boolean DEFAULT_PIPELINED

See Also:

PIPELINED, Constant Field Values

MAX_MEMORY

static final String MAX_MEMORY

This annotation is only used for non-PIPELINED operators and specifies the maximum #of bytes which the operator may buffer on the native heap before evaluation of the operator is triggered (default 0L).

If an operator buffers its data on the Java heap then this MUST be ZERO (0L). At-once evaluation is always used for non-pipelined operators which buffer their data on the Java heap as there is no ready mechanism to bound their heap demand.

If an operator buffers its data on the native heap, then this MUST be some positive value which specifies the maximum #of bytes which may be buffered before the operator is evaluated. At-once evaluation for operators which buffer their data on the native heap is indicated with the value Long.MAX_VALUE.

Note: For a sharded operation, the value is the maximum #of bytes which may be buffered per shard.

See Also:

PipelineOp.isPipelinedEvaluation()

DEFAULT_MAX_MEMORY

static final long DEFAULT_MAX_MEMORY

See Also:

MAX_MEMORY, Constant Field Values

LAST_PASS

static final String LAST_PASS

When true a final evaluation pass will be invoked once it is know that the operator can not be re-triggered by another IChunkMessage. The final evaluation pass will be associated with an empty IChunkMessage as its source and BOpContext.isLastInvocation() will report true.

Note: A final evaluation pass will be triggered even if the operator was never triggered by a normal evaluation pass. This behavior is necessary for several language constructs. Operators are free to do nothing if they can ignore the final evaluation pass in this case.

DEFAULT_LAST_PASS

static final boolean DEFAULT_LAST_PASS

See Also:

Constant Field Values