public class RWStrategy extends AbstractRawStore implements IBufferStrategy, IHABufferStrategy, IRWStrategy
IBufferStrategy wrapping the
RWStore which may be used as the backing store for a Journal.
The RWStore manages allocation slots. This can translate into an
enormous space savings on the disk for large data sets (when compared to the
WORM) since old revisions of B+Tree nodes and leaves may be recycled
efficiently.
RWStrategy supports access to historical commit states in
combination with the history retention policy of the
ITransactionService.
RWStore uses a distinct binary layout on the disk based which is
not directly compatible with the WORM binary storage layer. The WORM and the
RWStore uses the same file header and root blocks. However, the
RWStore defines some fields in the root blocks which are not used by
the WORM store such as the metabits info. In addition, some of the root block
fields defined by the WORM store are not used by the RWStore.RWStore.OptionsNULL| Modifier and Type | Method and Description |
|---|---|
void |
abort()
Calls through to store and then to WriteCacheService.reset
|
void |
abortContext(IAllocationContext context)
Indicates that the allocation context will no longer be used and that the
allocations made within the context should be discarded.
|
int |
checkDeferredFrees(AbstractJournal journal)
Lifted to provide a direct interface from the Journal so that the
CommitRecordIndex can be pruned prior to store commit.
|
void |
close()
Close the store immediately.
|
void |
closeForWrites()
Operation is not supported.
|
void |
commit()
A method that removes assumptions of how a specific strategy commits
data.
|
void |
computeDigest(Object snapshot,
MessageDigest digest)
Compute the digest of the entire backing store (including the magic, file
version, root blocks, etc).
|
void |
delete(long addr)
The default implementation is a NOP.
|
void |
delete(long addr,
IAllocationContext context)
Must check whether there are existing transactions which may access
this data, and if not free immediately, otherwise defer.
|
void |
deleteResources()
Deletes the backing file(s) (if any).
|
void |
destroy()
Closes the store immediately (if open) and deletes its persistent
resources.
|
void |
detachContext(IAllocationContext context)
Indicates that the allocation context will no longer be used, but that
the allocations made within the context should be preserved.
|
void |
force(boolean metadata)
Force the data to stable storage.
|
IAddressManager |
getAddressManager()
Return the delegate object that provides the
IAddressManager
implementation for this IRawStore. |
long |
getBlockSequence()
Return the #of
WriteCache blocks that were written out for the
last write set. |
BufferMode |
getBufferMode()
The buffer mode supported by the implementation
|
int |
getByteCount(long addr)
The length of the datum in bytes.
|
Lock |
getCommitLock()
Optionally return a
Lock that must be used (when non-
null) to make the IBufferStrategy.commit() /
IRWStrategy.postCommit() strategy atomic. |
CounterSet |
getCounters()
Return the performance counter hierarchy.
|
long |
getCurrentBlockSequence()
Return the then-current write cache block sequence.
|
long |
getExtent()
The current size of the journal in bytes.
|
File |
getFile()
The backing file -or-
null if there is no backing file
for the store. |
int |
getHeaderSize()
The size of the journal header, including MAGIC, version, and both root
blocks.
|
long |
getInitialExtent()
The initial extent.
|
InputStream |
getInputStream(long addr)
Return an input stream from which a previously written stream may be read
back.
|
long |
getLastReleaseTime()
If history is retained this returns the time for which data was most
recently released.
|
long |
getMaximumExtent()
The maximum extent allowable before a buffer overflow operation will be
rejected.
|
int |
getMaxRecordSize() |
long |
getMetaBitsAddr()
The RWStrategy requires meta allocation info in the root block, this
method is the hook to enable access.
|
long |
getMetaStartAddr()
The RWStrategy requires meta allocation info in the root block, this
method is the hook to enable access.
|
long |
getNextOffset()
The next offset at which a data item would be written on the store as an
offset into the user extent.
|
long |
getOffset(long addr)
The offset on the store at which the datum is stored.
|
int |
getOffsetBits()
Although the RW Store uses a latched addressing strategy it is not
meaningful to make this available in this interface.
|
IPSOutputStream |
getOutputStream()
Return an output stream which can be used to write on the backing store.
|
IPSOutputStream |
getOutputStream(IAllocationContext context)
Return an output stream which can be used to write on the backing store
within the given allocation context.
|
long |
getPhysicalAddress(long addr)
Determine the unencoded physical address
|
IResourceMetadata |
getResourceMetadata()
Not supported - this is available on the
AbstractJournal. |
RWStore |
getStore()
Return the backing
IStore. |
StoreState |
getStoreState()
A StoreState object references critical transient data that can be used
to determine a degree of consistency between stores, specifically for an
HA context.
|
long |
getUserExtent()
The size of the user data extent in bytes.
|
UUID |
getUUID()
|
WriteCacheService |
getWriteCacheService()
Return the
WriteCacheService (mainly for debugging). |
boolean |
inWriteCache(long addr)
Return
true iff the address was in the write cache as of the
moment the write cache was inspected. |
boolean |
isCommitted(long addr)
Return true if the address is marked as committed in the
RWStore
in memory bit maps. |
boolean |
isDirty()
Supports protocol in BigdataSailConnection to check for modifications
prior to calling rollback().
|
boolean |
isFullyBuffered()
True iff the store is fully buffered (all reads are against memory).
|
boolean |
isOpen()
true iff the store is open. |
boolean |
isReadOnly()
true iff the store does not allow writes. |
boolean |
isStable()
True iff backed by stable storage.
|
boolean |
lockAddress(long addr)
An assert oriented method that allows a finite number of addresses
to be monitored to ensure it is not freed.
|
IAllocationContext |
newAllocationContext(boolean isolated)
Creates a context to be used to isolate updates to within the context until it
is released to the parent environment.
|
IRawTx |
newTx()
A hook used to support session protection by incrementing and
decrementing a transaction counter within the
IStore. |
void |
postCommit()
Called post commit to dispose any transient commit state retained to
support reset/rollback.
|
void |
postHACommit(IRootBlockView rootBlock)
Provides a trigger for synchronization of transient state after a commit.
|
ByteBuffer |
read(long addr)
Read the data (unisolated).
|
ByteBuffer |
readFromLocalStore(long addr)
Read from the local store in support of failover reads on nodes in a
highly available
Quorum. |
ByteBuffer |
readRaw(long position,
ByteBuffer transfer)
Used to support the rebuild protocol.
|
ByteBuffer |
readRootBlock(boolean rootBlock0)
Read the specified root block from the backing file.
|
void |
registerExternalCache(ConcurrentWeakValueCache<Long,ICommitter> historicalIndexCache,
int byteCount)
Call made from AbstractJournal to register the cache used.
|
boolean |
requiresCommit(IRootBlockView block)
A method that removes assumptions of how a specific strategy determines
whether a transaction commit is required.
|
void |
resetFromHARootBlock(IRootBlockView rootBlock)
Reload from the current root block - CAUTION : THIS IS NOT A
RESET / ABORT.
|
long |
saveDeferrals()
Saves the current list of delete blocks, returning the address allocated.
|
Future<Void> |
sendHALogBuffer(IHALogRequest req,
IHAWriteMessage msg,
IBufferAccess b)
Send an
IHAWriteMessage and the associated raw buffer through the
write pipeline. |
Future<Void> |
sendRawBuffer(IHARebuildRequest req,
long sequence,
long quorumToken,
long fileExtent,
long offset,
int nbytes,
ByteBuffer b)
Send an
IHAWriteMessage and the associated raw buffer through the
write pipeline. |
void |
setExtentForLocalStore(long extent)
Called from HAGlue.receiveAndReplicate to ensure the correct file extent
prior to any writes.
|
long |
size()
The #of application data bytes written on the store (does not count any
headers or root blocks that may exist for the store).
|
Object |
snapshotAllocators()
Snapshot the allocators in preparation for computing a digest of the
committed allocations.
|
long |
toAddr(int nbytes,
long offset)
Converts a byte count and offset into a long integer.
|
String |
toString(long addr)
A human readable representation of the address.
|
long |
transferTo(RandomAccessFile out)
Operation is not supported.
|
void |
truncate(long extent)
Either truncates or extends the journal.
|
boolean |
useChecksums()
Return
true if the store uses per-record checksums. |
long |
write(ByteBuffer data)
Write the data (unisolated).
|
long |
write(ByteBuffer data,
IAllocationContext context)
Overridden to integrate with the shadow allocator support of the
RWStore. |
void |
writeOnStream(OutputStream os,
AbstractJournal.ISnapshotData snapshotData,
Quorum<HAGlue,QuorumService<HAGlue>> quorum,
long token)
Write a consistent snapshot of the committed state of the backing store.
|
void |
writeRawBuffer(HARebuildRequest req,
IHAWriteMessage msg,
ByteBuffer transfer)
Used to support the rebuild protocol
|
void |
writeRawBuffer(IHAWriteMessage msg,
IBufferAccess b)
Write a buffer containing data replicated from the master onto the local
persistence store.
|
void |
writeRootBlock(IRootBlockView rootBlock,
ForceEnum forceOnCommit)
Write the root block onto stable storage (ie, flush it through to disk).
|
public ByteBuffer readRootBlock(boolean rootBlock0)
IBufferStrategyreadRootBlock in interface IBufferStrategypublic ByteBuffer read(long addr)
IRawStoreread in interface IRawStoreaddr - A long integer that encodes both the offset from which the
data will be read and the #of bytes to be read. See
IAddressManager.toAddr(int, long).public long write(ByteBuffer data)
IRawStorewrite in interface IRawStoredata - The data. The bytes from the current
Buffer.position() to the
Buffer.limit() will be written and the
Buffer.position() will be advanced to the
Buffer.limit() . The caller may subsequently
modify the contents of the buffer without changing the state
of the store (i.e., the data are copied into the store).IAddressManager.public long write(ByteBuffer data, IAllocationContext context)
RWStore. Shadow allocators may be used to isolate allocation
changes (both allocating slots and releasing slots) across different
processes.write in interface IAllocationManagerStoredata - The data.context - The allocation context.public void delete(long addr)
AbstractRawStoredelete in interface IRawStoredelete in class AbstractRawStoreaddr - A long integer formed using Addr that encodes both the
offset at which the data was written and the #of bytes that
were written.public void delete(long addr,
IAllocationContext context)
delete in interface IAllocationManagerStoreaddr - The address whose allocation is to be deleted.context - The allocation context.public void detachContext(IAllocationContext context)
IAllocationManagerIStore is the top-level parent of
allocation contexts. The allocators associated with the allocation
context are return to the global list of available allocators.detachContext in interface IAllocationManagercontext - The application object which serves as the allocation context.public void abortContext(IAllocationContext context)
IAllocationManagerabortContext in interface IAllocationManagercontext - The application object which serves as the allocation context.public void closeForWrites()
closeForWrites in interface IBufferStrategyUnsupportedOperationException - always.public BufferMode getBufferMode()
IBufferStrategygetBufferMode in interface IBufferStrategypublic CounterSet getCounters()
IBufferStrategygetCounters in interface ICounterSetAccessgetCounters in interface IBufferStrategypublic long getExtent()
IBufferStrategyOptions.INITIAL_EXTENT.getExtent in interface IBufferStrategypublic int getHeaderSize()
IBufferStrategygetHeaderSize in interface IBufferStrategypublic long getInitialExtent()
IBufferStrategygetInitialExtent in interface IBufferStrategypublic long getMaximumExtent()
IBufferStrategy
Note: The semantics here differ from those defined by
Options.MAXIMUM_EXTENT. The latter specifies the threshold at
which a journal will overflow (onto another journal) while this specifies
the maximum size to which a buffer is allowed to grow.
Note: This is normally zero (0L), which basically means that
the maximum extent is ignored by the IBufferStrategy but
respected by the AbstractJournal, resulting in a soft limit
on journal overflow.
getMaximumExtent in interface IBufferStrategy0L
iff no limit is imposed.public boolean useChecksums()
IBufferStrategytrue if the store uses per-record checksums. When
true, an additional 4 bytes are written after the record on
the disk. Those bytes contain the checksum of the record.useChecksums in interface IBufferStrategypublic long getNextOffset()
IBufferStrategygetNextOffset in interface IBufferStrategypublic long getUserExtent()
IBufferStrategy
Note: The size of the user extent is always generally smaller than the
value reported by IBufferStrategy.getExtent() since the latter also reports the
space allocated to the journal header and root blocks.
getUserExtent in interface IBufferStrategypublic long transferTo(RandomAccessFile out) throws IOException
transferTo in interface IBufferStrategyout - The file to which the buffer contents will be transferred.UnsupportedOperationException - always.IOExceptionpublic void truncate(long extent)
IBufferStrategyNote: Implementations of this method MUST be synchronized so that the operation is atomic with respect to concurrent writers.
truncate in interface IBufferStrategyextent - The new extent of the journal. This value represent the total
extent of the journal, including any root blocks together with
the user extent.public void writeRootBlock(IRootBlockView rootBlock, ForceEnum forceOnCommit)
IBufferStrategywriteRootBlock in interface IBufferStrategyrootBlock - The root block. Which root block is indicated by
IRootBlockView.isRootBlock0().public void close()
IRawStorepublic void deleteResources()
IRawStoredeleteResources in interface IRawStorepublic void destroy()
IRawStoreIllegalStateException if the store
is already closed, but still deletes the backing resources.destroy in interface IRawStoreIRawStore.deleteResources()public void commit()
IBufferStrategycommit in interface IBufferStrategypublic void abort()
abort in interface IBufferStrategypublic void force(boolean metadata)
IRawStorepublic File getFile()
IRawStorenull if there is no backing file
for the store.public IResourceMetadata getResourceMetadata()
AbstractJournal.getResourceMetadata in interface IRawStoreUnsupportedOperationException - alwayspublic boolean isFullyBuffered()
IRawStoreNote: This does not guarantee that the OS will not swap the buffer onto disk.
isFullyBuffered in interface IRawStorepublic boolean isOpen()
IRawStoretrue iff the store is open.public boolean isReadOnly()
IRawStoretrue iff the store does not allow writes.isReadOnly in interface IRawStorepublic boolean isStable()
IRawStorepublic long size()
Overridden to return the #of bytes in the file rather than the user
bytes. This is because the RWStore does not know the #of bytes of
user data in each allocation slot. Therefore it is not able to keep
accurrate track of the user bytes as allocation slots are cycled.
public IAddressManager getAddressManager()
AbstractRawStoreIAddressManager
implementation for this IRawStore.
Note: LRUNexus depends on the delegation model to retain
references to the IAddressManager without causing the
IRawStore to be retained. It uses the IAddressManager to
decode the address in order to track the bytesOnDisk for the buffered
records.
getAddressManager in interface IBufferStrategygetAddressManager in class AbstractRawStorepublic int getByteCount(long addr)
IAddressManagerIRawStore.getByteCount in interface IAddressManageraddr - The opaque identifier that is the within store locator for
some datum.public long getOffset(long addr)
IAddressManagergetOffset in interface IAddressManageraddr - The opaque identifier that is the within store locator for
some datum.public long toAddr(int nbytes,
long offset)
IAddressManagertoAddr in interface IAddressManagernbytes - The byte count.offset - The byte offset.public String toString(long addr)
IAddressManagertoString in interface IAddressManageraddr - The opaque identifier that is the within store locator for
some datum.public boolean requiresCommit(IRootBlockView block)
The state of the provided block is not relevant since it does not hold
information on recent allocations (the meta allocations will only effect
the root block after a commit). This is passed through to the
RWStore which examines its internal state.
requiresCommit in interface IBufferStrategyblock - The root block held by the client, can be checked against the
state of the Buffer Strategypublic boolean isDirty()
isDirty in interface IBufferStrategypublic long getMetaBitsAddr()
IBufferStrategygetMetaBitsAddr in interface IBufferStrategypublic long getMetaStartAddr()
IBufferStrategygetMetaStartAddr in interface IBufferStrategypublic int getMaxRecordSize()
getMaxRecordSize in interface IBufferStrategypublic int getOffsetBits()
getOffsetBits in interface IBufferStrategypublic RWStore getStore()
IRWStrategyIStore.getStore in interface IRWStrategypublic long getPhysicalAddress(long addr)
IAddressManagergetPhysicalAddress in interface IAddressManageraddr - The encoded addresspublic void writeRawBuffer(IHAWriteMessage msg, IBufferAccess b) throws IOException, InterruptedException
IHABufferStrategywriteRawBuffer in interface IHABufferStrategyIOExceptionInterruptedExceptionpublic Future<Void> sendHALogBuffer(IHALogRequest req, IHAWriteMessage msg, IBufferAccess b) throws IOException, InterruptedException
IHABufferStrategyIHAWriteMessage and the associated raw buffer through the
write pipeline.sendHALogBuffer in interface IHABufferStrategyreq - The IHALogRequest for some HALog file.msg - The IHAWriteMessage.b - The raw buffer. Bytes from position to limit will be sent.
remaining() must equal IHAWriteMessageBase.getSize().Future for that request.IOExceptionInterruptedExceptionpublic Future<Void> sendRawBuffer(IHARebuildRequest req, long sequence, long quorumToken, long fileExtent, long offset, int nbytes, ByteBuffer b) throws IOException, InterruptedException
IHABufferStrategyIHAWriteMessage and the associated raw buffer through the
write pipeline.sendRawBuffer in interface IHABufferStrategyreq - The IHARebuildRequest to replicate the backing file to
the requesting service.sequence - The sequence of this IHAWriteMessage (origin ZERO
(0)).quorumToken - The quorum token of the leader, which must remain valid across
the rebuild protocol.fileExtent - The file extent as of the moment that the leader begins to
replicate the existing backing file.offset - The starting offset (relative to the root blocks).nbytes - The #of bytes to be sent.b - The raw buffer. The buffer will be cleared and filled with the
specified data, then sent down the write pipeline.Future for that request.IOExceptionInterruptedExceptionpublic void writeOnStream(OutputStream os, AbstractJournal.ISnapshotData snapshotData, Quorum<HAGlue,QuorumService<HAGlue>> quorum, long token) throws IOException, QuorumException
IHABufferStrategyNote: The caller is able to obtain both root blocks atomically, while the strategy may not be aware of the root blocks or may not be able to coordinate their atomic capture.
Note: The caller must ensure that the resulting snapshot will be consistent either by ensuring that no writes occur or by taking a read-lock that will prevent overwrites of committed state during this operation.
writeOnStream in interface IHABufferStrategyos - Where to write the data.quorum - The Quorum.token - The token that must remain valid during this operation.IOExceptionQuorumException - if the service is not joined with the met quorum for that
token at any point during the operation.public Object snapshotAllocators()
IHABufferStrategysnapshotAllocators in interface IHABufferStrategynull if the snapshot is a NOP for the
IBufferStrategy (e.g., for the WORM).public void computeDigest(Object snapshot, MessageDigest digest) throws DigestException, IOException
IHABufferStrategyNote: The digest is not reliable unless you either use a snapshot or suspend writes (on the quorum) while it is computed.
computeDigest in interface IHABufferStrategysnapshot - The allocator snapshot (optional). When given, the digest is
computed only for the snapshot. When null it is
computed for the entire file.DigestExceptionIOExceptionpublic ByteBuffer readFromLocalStore(long addr) throws InterruptedException
IHABufferStrategyQuorum.readFromLocalStore in interface IHABufferStrategyInterruptedExceptionpublic void setExtentForLocalStore(long extent)
throws IOException,
InterruptedException
setExtentForLocalStore in interface IHABufferStrategyInterruptedExceptionIOExceptionIHABufferStrategy.setExtentForLocalStore(long)public boolean lockAddress(long addr)
addr - - address to be lockedpublic long getLastReleaseTime()
IHistoryManagergetLastReleaseTime in interface IHistoryManagerpublic int checkDeferredFrees(AbstractJournal journal)
checkDeferredFrees in interface IHistoryManagerAbstractJournal#commitNow()public boolean isCommitted(long addr)
RWStore
in memory bit maps.isCommitted in interface IRWStrategyaddr - The address.true iff the address is currently committed.public boolean inWriteCache(long addr)
true iff the address was in the write cache as of the
moment the write cache was inspected.addr - The address.public IRawTx newTx()
IHistoryManagerIStore. As long as
a transaction is active we can not release data which is currently marked
as freed but was committed at the point the session started.newTx in interface IHistoryManagerpublic void registerExternalCache(ConcurrentWeakValueCache<Long,ICommitter> historicalIndexCache, int byteCount)
IHistoryManagerNote: It is not safe to clear at the point of the delete request since the data could still be loaded if the data is retained for a period due to a non-zero retention period or session protection.
registerExternalCache in interface IHistoryManagerpublic long saveDeferrals()
IHistoryManagerWrites the content of currentTxnFreeList to the store.
These are the current buffered frees that have yet been saved into a block referenced from the deferredFreeList
saveDeferrals in interface IHistoryManagerDeleteBlockCommitterpublic InputStream getInputStream(long addr)
IStreamStoregetInputStream in interface IStreamStoregetInputStream in class AbstractRawStoreaddr - The address at which the stream was written.public IPSOutputStream getOutputStream()
IStreamStoreIPSOutputStream.getOutputStream in interface IStreamStoregetOutputStream in class AbstractRawStorepublic IPSOutputStream getOutputStream(IAllocationContext context)
IAllocationManagerStoreIPSOutputStream.getOutputStream in interface IAllocationManagerStorecontext - The context within which any allocations are made by the
returned IPSOutputStream.public void resetFromHARootBlock(IRootBlockView rootBlock)
IHABufferStrategy
Note: This method is used when the root blocks of the leader are
installed onto a follower. This can change the UUID for the
backing store file. The IHABufferStrategy implementation MUST
update any cached value for that UUID.
Use IHABufferStrategy.postHACommit(IRootBlockView) rather than this method in the
2-phase commit on the follower.
resetFromHARootBlock in interface IHABufferStrategypublic long getBlockSequence()
IHABufferStrategyWriteCache blocks that were written out for the
last write set. This is used to communicate the #of write cache blocks in
the commit point back to AbstractJournal.commitNow(long). It is
part of the commit protocol.
Note: This DOES NOT reflect the current value of the block sequence
counter for ongoing writes. That counter is owned by the
WriteCacheService.
getBlockSequence in interface IHABufferStrategyWriteCacheService.resetSequence(),
FIXME I would prefer to expose the {@link WriteCacheService} to the
{@link AbstractJournal} and let it directly invoke
{@link WriteCacheService#resetSequence()}. The current pattern
requires the {@link IHABufferStrategy} implementations to track the
lastBlockSequence and is messy.public long getCurrentBlockSequence()
IHABufferStrategygetCurrentBlockSequence in interface IHABufferStrategyIHABufferStrategy.getBlockSequence()public ByteBuffer readRaw(long position, ByteBuffer transfer)
IHABufferStrategyreadRaw in interface IHABufferStrategyposition - - absolute file offsettransfer - - target buffer for readpublic void writeRawBuffer(HARebuildRequest req, IHAWriteMessage msg, ByteBuffer transfer) throws IOException
IHABufferStrategywriteRawBuffer in interface IHABufferStrategyIOExceptionpublic Lock getCommitLock()
IRWStrategyLock that must be used (when non-
null) to make the IBufferStrategy.commit() /
IRWStrategy.postCommit() strategy atomic.getCommitLock in interface IRWStrategypublic void postCommit()
IRWStrategy
Note: It is the responsibility of the commit protocol layers to wind up
calling IBufferStrategy.abort() if there is a failure during the
commit protocol.
postCommit in interface IRWStrategypublic void postHACommit(IRootBlockView rootBlock)
IHABufferStrategyFor the RWStore this is used to resynchronize the allocators during the 2-phase commit on the follower with the delta in the allocators from the write set associated with that commit.
postHACommit in interface IHABufferStrategyrootBlock - The newly installed root block.public WriteCacheService getWriteCacheService()
IHABufferStrategyWriteCacheService (mainly for debugging).getWriteCacheService in interface IHABufferStrategypublic StoreState getStoreState()
IHABufferStrategygetStoreState in interface IHABufferStrategypublic IAllocationContext newAllocationContext(boolean isolated)
IAllocationManagernewAllocationContext in interface IAllocationManagerCopyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.