public abstract class AbstractJoinNexus extends Object implements IJoinNexus
IJoinNexus
Modifier and Type | Field and Description |
---|---|
protected int |
chunkCapacity |
protected int |
chunkOfChunksCapacity |
protected IElementFilter<?> |
filter |
protected IIndexManager |
indexManager |
protected IEvaluationPlanFactory |
planFactory
The factory for rule evaluation plans.
|
protected long |
readTimestamp |
protected IResourceLocator<?> |
resourceLocator
Note: cached.
|
protected int |
solutionFlags |
ALL, BINDINGS, ELEMENT, RULE
Modifier | Constructor and Description |
---|---|
protected |
AbstractJoinNexus(IJoinNexusFactory joinNexusFactory,
IIndexManager indexManager) |
Modifier and Type | Method and Description |
---|---|
boolean |
bind(IPredicate<?> pred,
IConstraint[] constraints,
Object e,
IBindingSet bindings)
Binds variables from a visited element.
|
boolean |
bind(IRule rule,
int index,
Object e,
IBindingSet bindings)
Binds variables from a visited element.
|
boolean |
forceSerialExecution()
|
ActionEnum |
getAction()
The kind of operation that is being executed (Query, Insert, or Delete).
|
IStreamSerializer<IBindingSet[]> |
getBindingSetSerializer()
FIXME Custom serialization for binding sets, especially since there tends
to be a lot of redundancy in the data arising from how bindings are
propagated during JOINs.
|
int |
getChunkCapacity()
The #of elements in a chunk for query or mutation.
|
int |
getChunkOfChunksCapacity()
The #of chunks that can be held by an
IBuffer that is the target
or one or more UnsynchronizedArrayBuffer s. |
int |
getFullyBufferedReadThreshold()
The #of elements that will be materialized in a fully buffered read by an
IAccessPath . |
IRelation |
getHeadRelationView(IPredicate pred)
The head relation is what we write on for mutation operations and is also
responsible for minting new elements from computed
ISolution s. |
IIndexManager |
getIndexManager()
Used to locate indices, relations and relation containers.
|
IJoinNexusFactory |
getJoinNexusFactory()
The factory object is used to materialize appropriate
IJoinNexus
instances when the rule execution crosses an RMI boundary. |
int |
getMaxParallelSubqueries()
The maximum #of subqueries for the first join dimension that will be
issued in parallel.
|
IEvaluationPlanFactory |
getPlanFactory()
Return the factory for
IEvaluationPlan s. |
String |
getProperty(String name,
String defaultValue)
Resolve the property value using the
IIndexManager , the namespace
of the resource, and the Properties instance to be tested as
hidden parameters. |
<T> T |
getProperty(String name,
String defaultValue,
IValidator<T> validator)
Resolves, parses, and validates the property value.
|
IRangeCountFactory |
getRangeCountFactory()
The factory object for range counts used by
IEvaluationPlan s. |
long |
getReadTimestamp()
Equivalent to
IJoinNexusFactory.getReadTimestamp() . |
IResourceLocator |
getRelationLocator() |
IRuleStatisticsFactory |
getRuleStatisticsFactory()
The factory for rule statistics objects.
|
IRuleTaskFactory |
getRuleTaskFactory(boolean parallel,
IRule rule)
Return the effective
IRuleTaskFactory for the rule. |
IElementFilter<ISolution> |
getSolutionFilter()
Return the
IElementFilter that will be used to reject solutions
based on the bindings for the head of the rule -or- null
if no filter will be imposed. |
IStreamSerializer<ISolution[]> |
getSolutionSerializer()
FIXME Custom serialization for solution sets, especially since there
tends to be a lot of redundancy in the data arising from how bindings are
propagated during JOINs.
|
IAccessPath |
getTailAccessPath(IRelation relation,
IPredicate predicate)
Obtain an access path reading from relation for the specified predicate
(from the tail of some rule).
|
IRelation |
getTailRelationView(IPredicate pred)
Locate and return the view of the relation(s) identified by the
IPredicate . |
long |
getWriteTimestamp()
Equivalent to
IJoinNexusFactory.getWriteTimestamp() . |
protected boolean |
isEmptyProgram(IStep step)
Return true iff the step is an empty
IProgram . |
Iterator<PartitionLocator> |
locatorScan(AbstractScaleOutFederation<?> fed,
IPredicate<?> predicate)
Return an iterator visiting the
PartitionLocator for the index
partitions from which an IAccessPath must read in order to
materialize all elements which would be visited for that predicate. |
IBindingSet |
newBindingSet(IRule rule)
Factory for
IBindingSet implementations. |
IBuffer<ISolution[]> |
newDeleteBuffer(IMutableRelation relation)
Return a thread-safe buffer onto which chunks of computed
ISolution s will be written. |
IBuffer<ISolution[]> |
newInsertBuffer(IMutableRelation relation)
Return a thread-safe buffer onto which chunks of computed
ISolution s will be written. |
IBlockingBuffer<ISolution[]> |
newQueryBuffer()
Note:
ISolution (not relation elements) will be written on the
buffer concurrently by different rules so there is no natural order for
the elements in the buffer. |
ISolution |
newSolution(IRule rule,
IBindingSet bindingSet)
Create a new
ISolution . |
protected abstract ISortKeyBuilder<?> |
newSortKeyBuilder(IPredicate<?> head)
Return the
ISortKeyBuilder used to impose DISTINCT on the
solutions generated by a query. |
IBuffer<ISolution> |
newUnsynchronizedBuffer(IBuffer<ISolution[]> targetBuffer,
int chunkCapacity)
Return a buffer suitable for a single-threaded writer that flushes onto
the specified targetBuffer.
|
protected Object |
runDistributedProgram(IBigdataFederation<?> fed,
ActionEnum action,
IStep step)
Runs a distributed
IProgram (key-range partitioned indices, RMI,
and multi-machine). |
protected Object |
runLocalProgram(ActionEnum action,
IStep step)
This variant handles both local indices on a
TemporaryStore or
Journal WITHOUT concurrency controls (fast). |
long |
runMutation(IStep step)
Run as mutation operation (it will write any solutions onto the relations
named in the head of the various
IRule s). |
protected Object |
runProgram(ActionEnum action,
IStep step)
Core impl.
|
IChunkedOrderedIterator<ISolution> |
runQuery(IStep step)
Run as a query.
|
int |
solutionFlags()
The flags that effect the behavior of
IJoinNexus.newSolution(IRule, IBindingSet) . |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
fakeBinding, newBindingSetSortKeyBuilder
protected final IIndexManager indexManager
protected final IResourceLocator<?> resourceLocator
protected final long readTimestamp
protected final int chunkCapacity
protected final int chunkOfChunksCapacity
protected final int solutionFlags
protected final IElementFilter<?> filter
protected final IEvaluationPlanFactory planFactory
protected AbstractJoinNexus(IJoinNexusFactory joinNexusFactory, IIndexManager indexManager)
joinNexusFactory
- The object used to create this instance and which can be used
to create other instances as necessary for distributed rule
execution.indexManager
- The object used to resolve indices, relations, etc.public final int getChunkOfChunksCapacity()
IJoinNexus
IBuffer
that is the target
or one or more UnsynchronizedArrayBuffer
s. This is generally a
small value on the order of the #of parallel producers that might be
writing on the IBuffer
since the capacity of the
UnsynchronizedArrayBuffer
s is already quite large (10k or better
elements, defining a single "chunk" from a single producer).getChunkOfChunksCapacity
in interface IJoinNexus
IJoinNexus.getMaxParallelSubqueries()
,
IJoinNexus.getChunkCapacity()
public final int getChunkCapacity()
IJoinNexus
10,000
or better.getChunkCapacity
in interface IJoinNexus
public final int getFullyBufferedReadThreshold()
IJoinNexus
IAccessPath
. When this threshold is exceeded the
IAccessPath
will use an IAsynchronousIterator
instead.
This value should on the order of IJoinNexus.getChunkCapacity()
.getFullyBufferedReadThreshold
in interface IJoinNexus
IAccessPath.iterator(long,long, int)
,
AbstractResource.Options#FULLY_BUFFERED_READ_THRESHOLD
public final String getProperty(String name, String defaultValue)
IJoinNexus
IIndexManager
, the namespace
of the resource, and the Properties
instance to be tested as
hidden parameters.getProperty
in interface IJoinNexus
name
- The property name.defaultValue
- The default.Configuration
public final <T> T getProperty(String name, String defaultValue, IValidator<T> validator)
IJoinNexus
getProperty
in interface IJoinNexus
name
- The property name.defaultValue
- The default value.public IElementFilter<ISolution> getSolutionFilter()
IJoinNexus
IElementFilter
that will be used to reject solutions
based on the bindings for the head of the rule -or- null
if no filter will be imposed. This may be used for query or mutation.getSolutionFilter
in interface IJoinNexus
public IRuleStatisticsFactory getRuleStatisticsFactory()
IJoinNexus
getRuleStatisticsFactory
in interface IJoinNexus
public IJoinNexusFactory getJoinNexusFactory()
IJoinNexus
IJoinNexus
instances when the rule execution crosses an RMI boundary.getJoinNexusFactory
in interface IJoinNexus
public IRangeCountFactory getRangeCountFactory()
IJoinNexus
IEvaluationPlan
s.getRangeCountFactory
in interface IJoinNexus
public final boolean forceSerialExecution()
IJoinNexus
true
, rule level parallelism is disabled and the
ISolution
buffers are flushed after after every IStep
.
This can be enabled if you are exploring apparent concurrency problems
with the rules. It should normally be false
for better
performance.forceSerialExecution
in interface IJoinNexus
public final int getMaxParallelSubqueries()
IJoinNexus
ExecutorService
entirely and ONE (1) to submit a single task at a
time to the ExecutorService
.getMaxParallelSubqueries
in interface IJoinNexus
public final ActionEnum getAction()
IJoinNexus
getAction
in interface IJoinNexus
public final long getWriteTimestamp()
IJoinNexus
IJoinNexusFactory.getWriteTimestamp()
.getWriteTimestamp
in interface IJoinNexus
public final long getReadTimestamp()
IJoinNexus
IJoinNexusFactory.getReadTimestamp()
.getReadTimestamp
in interface IJoinNexus
public IRelation getHeadRelationView(IPredicate pred)
ISolution
s.
This method depends solely on the name of the head relation and the
timestamp of interest for the view.getHeadRelationView
in interface IJoinNexus
pred
- The IPredicate
, which MUST be the head of some
IRule
.IRelation
, which will never be a fused view and
which will accept writes iff the rules are being executed as a
mutation operation.public IRelation getTailRelationView(IPredicate pred)
IJoinNexus
IPredicate
.
Note: This method is responsible for returning a fused view when more
than one relation name was specified for the IPredicate
. It
SHOULD be used whenever the IRelation
is selected based on a
predicate in the tail of an IRule
and could therefore be a fused
view of more than one relation instance. (The head of the IRule
must be a simple IRelation
and not a view.)
Note: The implementation should choose the read timestamp for each
relation in the view using #getReadTimestamp(String)
.
getTailRelationView
in interface IJoinNexus
pred
- The IPredicate
, which MUST be a tail from some IRule
.IRelation
, which might be a
RelationFusedView
.public IAccessPath getTailAccessPath(IRelation relation, IPredicate predicate)
IJoinNexus
Note that passing in the IRelation
is important since it
otherwise must be discovered using the IResourceLocator
. By
requiring the caller to resolve it before hand and pass it into this
method the contention and demand on the IResourceLocator
cache is
reduced.
getTailAccessPath
in interface IJoinNexus
relation
- The relation.predicate
- The predicate. When IPredicate.getPartitionId()
is
set, the returned IAccessPath
MUST read on the
identified local index partition (directly, not via RMI).public Iterator<PartitionLocator> locatorScan(AbstractScaleOutFederation<?> fed, IPredicate<?> predicate)
IJoinNexus
PartitionLocator
for the index
partitions from which an IAccessPath
must read in order to
materialize all elements which would be visited for that predicate.
Note: You can use an IDataServiceCallable
to obtain the
reference of the IDataService
and pass that into your
AbstractTask
in order to have the federation reference available
when running under the ConcurrencyManager
.
locatorScan
in interface IJoinNexus
fed
- The federation, which is required in order to access the
IMetadataIndex
for a scale-out index.predicate
- The predicate, with whatever bindings already applied.public final IIndexManager getIndexManager()
IJoinNexus
getIndexManager
in interface IJoinNexus
public final boolean bind(IRule rule, int index, Object e, IBindingSet bindings)
IJoinNexus
Note: The bindings are propagated before the constraints are verified so this method will have a side-effect on the bindings even if the constraints were not satisfied. Therefore you should clone the bindings before calling this method.
bind
in interface IJoinNexus
rule
- The rule.index
- The index of the IPredicate
in the body of the
Rule
.e
- An element materialized by the IAccessPath
for that
IPredicate
.true
unless the new bindings would violate any of
the IConstraint
s declared for the Rule
).public final boolean bind(IPredicate<?> pred, IConstraint[] constraints, Object e, IBindingSet bindings)
IJoinNexus
Note: The bindings are propagated before the constraints are verified so this method will have a side-effect on the bindings even if the constraints were not satisfied. Therefore you should clone the bindings before calling this method.
bind
in interface IJoinNexus
pred
- The IPredicate
from which the element was read.constraints
- An array of constraints which must be satisfied (optional).e
- An element materialized by the IAccessPath
for that
IPredicate
.true
unless the new bindings would violate any of
the optional IConstraint
.public final ISolution newSolution(IRule rule, IBindingSet bindingSet)
IJoinNexus
ISolution
. The behavior of this method generally
depends on bit flags specified when the IJoinNexus
was created.
Note: For many purposes, it is only the computed IJoinNexus.ELEMENT
s that
are of interest. For high-level query, you will generally specify only
the IJoinNexus.BINDINGS
. The IJoinNexus.BINDINGS
are also useful for some
truth maintenance applications. The IJoinNexus.RULE
is generally only of
interest for inspecting the behavior of some rule set.
newSolution
in interface IJoinNexus
rule
- The rule.bindingSet
- The bindings (the implementation MUST clone the bindings if
they will be saved with the ISolution
).ISolution
.IJoinNexus.ELEMENT
,
IJoinNexus.BINDINGS
,
IJoinNexus.RULE
,
IJoinNexus.solutionFlags()
,
Solution
public final int solutionFlags()
IJoinNexus
IJoinNexus.newSolution(IRule, IBindingSet)
.solutionFlags
in interface IJoinNexus
public IStreamSerializer<ISolution[]> getSolutionSerializer()
getSolutionSerializer
in interface IJoinNexus
(needs to be written).
public IStreamSerializer<IBindingSet[]> getBindingSetSerializer()
getBindingSetSerializer
in interface IJoinNexus
SPOBindingSetSerializer, which has not been finished.
public final IBindingSet newBindingSet(IRule rule)
IJoinNexus
IBindingSet
implementations.
Note: The factory MUST apply any bound
constants
for the IRule
before
returning the IBindingSet
.
newBindingSet
in interface IJoinNexus
rule
- The rule whose bindings will be stored in the binding set.public final IRuleTaskFactory getRuleTaskFactory(boolean parallel, IRule rule)
IJoinNexus
IRuleTaskFactory
for the rule. When the rule
is a step of a sequential program writing on one or more
IMutableRelation
s, then the returned IStepTask
must
automatically flush the buffer after the rule executes in order to ensure
that the state of the IMutableRelation
(s) are updated before the
next IRule
is executed.getRuleTaskFactory
in interface IJoinNexus
parallel
- true
unless the rule is a step is a sequential
IProgram
. Note that a sequential step MUST flush its
buffer since steps are run in sequence precisely because they
have a dependency!rule
- A rule that is a step in some program. If the program is just
a rule then the value of parallel does not matter. The
buffer will is cleared when it flushed so a re-flushed is
always a NOP.IStepTask
to execute for that rule.RunRuleAndFlushBufferTaskFactory
,
RunRuleAndFlushBufferTask
public final IEvaluationPlanFactory getPlanFactory()
IJoinNexus
IEvaluationPlan
s.getPlanFactory
in interface IJoinNexus
public final IResourceLocator getRelationLocator()
public final IBuffer<ISolution> newUnsynchronizedBuffer(IBuffer<ISolution[]> targetBuffer, int chunkCapacity)
IJoinNexus
The returned buffer MUST apply the optional filter value returned by
IJoinNexus.getSolutionFilter()
in order to keep individual
ISolution
s out of the buffer. Filtering is done at this level
since the targetBuffer contains chunks of solutions.
newUnsynchronizedBuffer
in interface IJoinNexus
targetBuffer
- A thread-safe buffer for chunks of ISolution
s that
was allocated with IJoinNexus.newQueryBuffer()
,
IJoinNexus.newInsertBuffer(IMutableRelation)
, or
IJoinNexus.newDeleteBuffer(IMutableRelation)
.chunkCapacity
- The capacity of the new buffer. This should be maximum chunk
size that will be produced or IJoinNexus.getChunkCapacity()
if
you do not have better information.public final IBlockingBuffer<ISolution[]> newQueryBuffer()
ISolution
(not relation elements) will be written on the
buffer concurrently by different rules so there is no natural order for
the elements in the buffer.newQueryBuffer
in interface IJoinNexus
public IBuffer<ISolution[]> newInsertBuffer(IMutableRelation relation)
ISolution
s will be written. When the buffer is
flushed
the chunked ISolution
s will be
inserted into the IMutableRelation
.
Note: getSolutionFilter()
is applied by
newUnsynchronizedBuffer(IBuffer, int)
and NOT by the buffer
returned by this method.
newInsertBuffer
in interface IJoinNexus
relation
- The relation.public IBuffer<ISolution[]> newDeleteBuffer(IMutableRelation relation)
ISolution
s will be written. When the buffer is
flushed
the chunks of ISolution
s will be
deleted from the IMutableRelation
.
Note: getSolutionFilter()
is applied by
newUnsynchronizedBuffer(IBuffer, int)
and NOT by the buffer
returned by this method.
newDeleteBuffer
in interface IJoinNexus
relation
- The relation.protected abstract ISortKeyBuilder<?> newSortKeyBuilder(IPredicate<?> head)
ISortKeyBuilder
used to impose DISTINCT on the
solutions generated by a query.head
- The head of the rule.ISortKeyBuilder
.public IChunkedOrderedIterator<ISolution> runQuery(IStep step) throws Exception
IJoinNexus
runQuery
in interface IJoinNexus
step
- The IRule
or IProgram
.IllegalStateException
- unless this is an ActionEnum.Query
.IllegalArgumentException
- if either argument is null
.Exception
public final long runMutation(IStep step) throws Exception
IJoinNexus
IRule
s).runMutation
in interface IJoinNexus
step
- The IRule
or IProgram
.IllegalArgumentException
- unless ActionEnum.isMutation()
is true
.Exception
protected final boolean isEmptyProgram(IStep step)
IProgram
.step
- The step.protected final Object runProgram(ActionEnum action, IStep step) throws Exception
DataService
(highly efficient) or within the client
using the IClientIndex
to submit operations to the appropriate
DataService
(s) (not very efficient, even w/o RMI).IChunkedOrderedIterator
(query) or Long
(mutation count).Exception
protected final Object runLocalProgram(ActionEnum action, IStep step) throws Exception
TemporaryStore
or
Journal
WITHOUT concurrency controls (fast).Exception
protected final Object runDistributedProgram(IBigdataFederation<?> fed, ActionEnum action, IStep step) throws Exception
IProgram
(key-range partitioned indices, RMI,
and multi-machine).Exception
Copyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.