public class RWStrategy extends AbstractRawStore implements IBufferStrategy, IHABufferStrategy, IRWStrategy
IBufferStrategy
wrapping the
RWStore
which may be used as the backing store for a Journal
.
The RWStore
manages allocation slots. This can translate into an
enormous space savings on the disk for large data sets (when compared to the
WORM) since old revisions of B+Tree nodes and leaves may be recycled
efficiently.
RWStrategy
supports access to historical commit states in
combination with the history retention policy of the
ITransactionService
.
RWStore
uses a distinct binary layout on the disk based which is
not directly compatible with the WORM binary storage layer. The WORM and the
RWStore
uses the same file header and root blocks. However, the
RWStore
defines some fields in the root blocks which are not used by
the WORM store such as the metabits info. In addition, some of the root block
fields defined by the WORM store are not used by the RWStore
.RWStore.Options
NULL
Modifier and Type | Method and Description |
---|---|
void |
abort()
Calls through to store and then to WriteCacheService.reset
|
void |
abortContext(IAllocationContext context)
Indicates that the allocation context will no longer be used and that the
allocations made within the context should be discarded.
|
int |
checkDeferredFrees(AbstractJournal journal)
Lifted to provide a direct interface from the Journal so that the
CommitRecordIndex can be pruned prior to store commit.
|
void |
close()
Close the store immediately.
|
void |
closeForWrites()
Operation is not supported.
|
void |
commit()
A method that removes assumptions of how a specific strategy commits
data.
|
void |
computeDigest(Object snapshot,
MessageDigest digest)
Compute the digest of the entire backing store (including the magic, file
version, root blocks, etc).
|
void |
delete(long addr)
The default implementation is a NOP.
|
void |
delete(long addr,
IAllocationContext context)
Must check whether there are existing transactions which may access
this data, and if not free immediately, otherwise defer.
|
void |
deleteResources()
Deletes the backing file(s) (if any).
|
void |
destroy()
Closes the store immediately (if open) and deletes its persistent
resources.
|
void |
detachContext(IAllocationContext context)
Indicates that the allocation context will no longer be used, but that
the allocations made within the context should be preserved.
|
void |
force(boolean metadata)
Force the data to stable storage.
|
IAddressManager |
getAddressManager()
Return the delegate object that provides the
IAddressManager
implementation for this IRawStore . |
long |
getBlockSequence()
Return the #of
WriteCache blocks that were written out for the
last write set. |
BufferMode |
getBufferMode()
The buffer mode supported by the implementation
|
int |
getByteCount(long addr)
The length of the datum in bytes.
|
Lock |
getCommitLock()
Optionally return a
Lock that must be used (when non-
null ) to make the IBufferStrategy.commit() /
IRWStrategy.postCommit() strategy atomic. |
CounterSet |
getCounters()
Return the performance counter hierarchy.
|
long |
getCurrentBlockSequence()
Return the then-current write cache block sequence.
|
long |
getExtent()
The current size of the journal in bytes.
|
File |
getFile()
The backing file -or-
null if there is no backing file
for the store. |
int |
getHeaderSize()
The size of the journal header, including MAGIC, version, and both root
blocks.
|
long |
getInitialExtent()
The initial extent.
|
InputStream |
getInputStream(long addr)
Return an input stream from which a previously written stream may be read
back.
|
long |
getLastReleaseTime()
If history is retained this returns the time for which data was most
recently released.
|
long |
getMaximumExtent()
The maximum extent allowable before a buffer overflow operation will be
rejected.
|
int |
getMaxRecordSize() |
long |
getMetaBitsAddr()
The RWStrategy requires meta allocation info in the root block, this
method is the hook to enable access.
|
long |
getMetaStartAddr()
The RWStrategy requires meta allocation info in the root block, this
method is the hook to enable access.
|
long |
getNextOffset()
The next offset at which a data item would be written on the store as an
offset into the user extent.
|
long |
getOffset(long addr)
The offset on the store at which the datum is stored.
|
int |
getOffsetBits()
Although the RW Store uses a latched addressing strategy it is not
meaningful to make this available in this interface.
|
IPSOutputStream |
getOutputStream()
Return an output stream which can be used to write on the backing store.
|
IPSOutputStream |
getOutputStream(IAllocationContext context)
Return an output stream which can be used to write on the backing store
within the given allocation context.
|
long |
getPhysicalAddress(long addr)
Determine the unencoded physical address
|
IResourceMetadata |
getResourceMetadata()
Not supported - this is available on the
AbstractJournal . |
RWStore |
getStore()
Return the backing
IStore . |
StoreState |
getStoreState()
A StoreState object references critical transient data that can be used
to determine a degree of consistency between stores, specifically for an
HA context.
|
long |
getUserExtent()
The size of the user data extent in bytes.
|
UUID |
getUUID()
|
WriteCacheService |
getWriteCacheService()
Return the
WriteCacheService (mainly for debugging). |
boolean |
inWriteCache(long addr)
Return
true iff the address was in the write cache as of the
moment the write cache was inspected. |
boolean |
isCommitted(long addr)
Return true if the address is marked as committed in the
RWStore
in memory bit maps. |
boolean |
isDirty()
Supports protocol in BigdataSailConnection to check for modifications
prior to calling rollback().
|
boolean |
isFullyBuffered()
True iff the store is fully buffered (all reads are against memory).
|
boolean |
isOpen()
true iff the store is open. |
boolean |
isReadOnly()
true iff the store does not allow writes. |
boolean |
isStable()
True iff backed by stable storage.
|
boolean |
lockAddress(long addr)
An assert oriented method that allows a finite number of addresses
to be monitored to ensure it is not freed.
|
IAllocationContext |
newAllocationContext(boolean isolated)
Creates a context to be used to isolate updates to within the context until it
is released to the parent environment.
|
IRawTx |
newTx()
A hook used to support session protection by incrementing and
decrementing a transaction counter within the
IStore . |
void |
postCommit()
Called post commit to dispose any transient commit state retained to
support reset/rollback.
|
void |
postHACommit(IRootBlockView rootBlock)
Provides a trigger for synchronization of transient state after a commit.
|
ByteBuffer |
read(long addr)
Read the data (unisolated).
|
ByteBuffer |
readFromLocalStore(long addr)
Read from the local store in support of failover reads on nodes in a
highly available
Quorum . |
ByteBuffer |
readRaw(long position,
ByteBuffer transfer)
Used to support the rebuild protocol.
|
ByteBuffer |
readRootBlock(boolean rootBlock0)
Read the specified root block from the backing file.
|
void |
registerExternalCache(ConcurrentWeakValueCache<Long,ICommitter> historicalIndexCache,
int byteCount)
Call made from AbstractJournal to register the cache used.
|
boolean |
requiresCommit(IRootBlockView block)
A method that removes assumptions of how a specific strategy determines
whether a transaction commit is required.
|
void |
resetFromHARootBlock(IRootBlockView rootBlock)
Reload from the current root block - CAUTION : THIS IS NOT A
RESET / ABORT.
|
long |
saveDeferrals()
Saves the current list of delete blocks, returning the address allocated.
|
Future<Void> |
sendHALogBuffer(IHALogRequest req,
IHAWriteMessage msg,
IBufferAccess b)
Send an
IHAWriteMessage and the associated raw buffer through the
write pipeline. |
Future<Void> |
sendRawBuffer(IHARebuildRequest req,
long sequence,
long quorumToken,
long fileExtent,
long offset,
int nbytes,
ByteBuffer b)
Send an
IHAWriteMessage and the associated raw buffer through the
write pipeline. |
void |
setExtentForLocalStore(long extent)
Called from HAGlue.receiveAndReplicate to ensure the correct file extent
prior to any writes.
|
long |
size()
The #of application data bytes written on the store (does not count any
headers or root blocks that may exist for the store).
|
Object |
snapshotAllocators()
Snapshot the allocators in preparation for computing a digest of the
committed allocations.
|
long |
toAddr(int nbytes,
long offset)
Converts a byte count and offset into a long integer.
|
String |
toString(long addr)
A human readable representation of the address.
|
long |
transferTo(RandomAccessFile out)
Operation is not supported.
|
void |
truncate(long extent)
Either truncates or extends the journal.
|
boolean |
useChecksums()
Return
true if the store uses per-record checksums. |
long |
write(ByteBuffer data)
Write the data (unisolated).
|
long |
write(ByteBuffer data,
IAllocationContext context)
Overridden to integrate with the shadow allocator support of the
RWStore . |
void |
writeOnStream(OutputStream os,
AbstractJournal.ISnapshotData snapshotData,
Quorum<HAGlue,QuorumService<HAGlue>> quorum,
long token)
Write a consistent snapshot of the committed state of the backing store.
|
void |
writeRawBuffer(HARebuildRequest req,
IHAWriteMessage msg,
ByteBuffer transfer)
Used to support the rebuild protocol
|
void |
writeRawBuffer(IHAWriteMessage msg,
IBufferAccess b)
Write a buffer containing data replicated from the master onto the local
persistence store.
|
void |
writeRootBlock(IRootBlockView rootBlock,
ForceEnum forceOnCommit)
Write the root block onto stable storage (ie, flush it through to disk).
|
public ByteBuffer readRootBlock(boolean rootBlock0)
IBufferStrategy
readRootBlock
in interface IBufferStrategy
public ByteBuffer read(long addr)
IRawStore
read
in interface IRawStore
addr
- A long integer that encodes both the offset from which the
data will be read and the #of bytes to be read. See
IAddressManager.toAddr(int, long)
.public long write(ByteBuffer data)
IRawStore
write
in interface IRawStore
data
- The data. The bytes from the current
Buffer.position()
to the
Buffer.limit()
will be written and the
Buffer.position()
will be advanced to the
Buffer.limit()
. The caller may subsequently
modify the contents of the buffer without changing the state
of the store (i.e., the data are copied into the store).IAddressManager
.public long write(ByteBuffer data, IAllocationContext context)
RWStore
. Shadow allocators may be used to isolate allocation
changes (both allocating slots and releasing slots) across different
processes.write
in interface IAllocationManagerStore
data
- The data.context
- The allocation context.public void delete(long addr)
AbstractRawStore
delete
in interface IRawStore
delete
in class AbstractRawStore
addr
- A long integer formed using Addr
that encodes both the
offset at which the data was written and the #of bytes that
were written.public void delete(long addr, IAllocationContext context)
delete
in interface IAllocationManagerStore
addr
- The address whose allocation is to be deleted.context
- The allocation context.public void detachContext(IAllocationContext context)
IAllocationManager
IStore
is the top-level parent of
allocation contexts. The allocators associated with the allocation
context are return to the global list of available allocators.detachContext
in interface IAllocationManager
context
- The application object which serves as the allocation context.public void abortContext(IAllocationContext context)
IAllocationManager
abortContext
in interface IAllocationManager
context
- The application object which serves as the allocation context.public void closeForWrites()
closeForWrites
in interface IBufferStrategy
UnsupportedOperationException
- always.public BufferMode getBufferMode()
IBufferStrategy
getBufferMode
in interface IBufferStrategy
public CounterSet getCounters()
IBufferStrategy
getCounters
in interface ICounterSetAccess
getCounters
in interface IBufferStrategy
public long getExtent()
IBufferStrategy
Options.INITIAL_EXTENT
.getExtent
in interface IBufferStrategy
public int getHeaderSize()
IBufferStrategy
getHeaderSize
in interface IBufferStrategy
public long getInitialExtent()
IBufferStrategy
getInitialExtent
in interface IBufferStrategy
public long getMaximumExtent()
IBufferStrategy
Note: The semantics here differ from those defined by
Options.MAXIMUM_EXTENT
. The latter specifies the threshold at
which a journal will overflow (onto another journal) while this specifies
the maximum size to which a buffer is allowed to grow.
Note: This is normally zero (0L), which basically means that
the maximum extent is ignored by the IBufferStrategy
but
respected by the AbstractJournal
, resulting in a soft limit
on journal overflow.
getMaximumExtent
in interface IBufferStrategy
0L
iff no limit is imposed.public boolean useChecksums()
IBufferStrategy
true
if the store uses per-record checksums. When
true
, an additional 4 bytes are written after the record on
the disk. Those bytes contain the checksum of the record.useChecksums
in interface IBufferStrategy
public long getNextOffset()
IBufferStrategy
getNextOffset
in interface IBufferStrategy
public long getUserExtent()
IBufferStrategy
Note: The size of the user extent is always generally smaller than the
value reported by IBufferStrategy.getExtent()
since the latter also reports the
space allocated to the journal header and root blocks.
getUserExtent
in interface IBufferStrategy
public long transferTo(RandomAccessFile out) throws IOException
transferTo
in interface IBufferStrategy
out
- The file to which the buffer contents will be transferred.UnsupportedOperationException
- always.IOException
public void truncate(long extent)
IBufferStrategy
Note: Implementations of this method MUST be synchronized so that the operation is atomic with respect to concurrent writers.
truncate
in interface IBufferStrategy
extent
- The new extent of the journal. This value represent the total
extent of the journal, including any root blocks together with
the user extent.public void writeRootBlock(IRootBlockView rootBlock, ForceEnum forceOnCommit)
IBufferStrategy
writeRootBlock
in interface IBufferStrategy
rootBlock
- The root block. Which root block is indicated by
IRootBlockView.isRootBlock0()
.public void close()
IRawStore
public void deleteResources()
IRawStore
deleteResources
in interface IRawStore
public void destroy()
IRawStore
IllegalStateException
if the store
is already closed, but still deletes the backing resources.destroy
in interface IRawStore
IRawStore.deleteResources()
public void commit()
IBufferStrategy
commit
in interface IBufferStrategy
public void abort()
abort
in interface IBufferStrategy
public void force(boolean metadata)
IRawStore
public File getFile()
IRawStore
null
if there is no backing file
for the store.public IResourceMetadata getResourceMetadata()
AbstractJournal
.getResourceMetadata
in interface IRawStore
UnsupportedOperationException
- alwayspublic boolean isFullyBuffered()
IRawStore
Note: This does not guarantee that the OS will not swap the buffer onto disk.
isFullyBuffered
in interface IRawStore
public boolean isOpen()
IRawStore
true
iff the store is open.public boolean isReadOnly()
IRawStore
true
iff the store does not allow writes.isReadOnly
in interface IRawStore
public boolean isStable()
IRawStore
public long size()
Overridden to return the #of bytes in the file rather than the user
bytes. This is because the RWStore
does not know the #of bytes of
user data in each allocation slot. Therefore it is not able to keep
accurrate track of the user bytes as allocation slots are cycled.
public IAddressManager getAddressManager()
AbstractRawStore
IAddressManager
implementation for this IRawStore
.
Note: LRUNexus
depends on the delegation model to retain
references to the IAddressManager
without causing the
IRawStore
to be retained. It uses the IAddressManager
to
decode the address in order to track the bytesOnDisk for the buffered
records.
getAddressManager
in interface IBufferStrategy
getAddressManager
in class AbstractRawStore
public int getByteCount(long addr)
IAddressManager
IRawStore
.getByteCount
in interface IAddressManager
addr
- The opaque identifier that is the within store locator for
some datum.public long getOffset(long addr)
IAddressManager
getOffset
in interface IAddressManager
addr
- The opaque identifier that is the within store locator for
some datum.public long toAddr(int nbytes, long offset)
IAddressManager
toAddr
in interface IAddressManager
nbytes
- The byte count.offset
- The byte offset.public String toString(long addr)
IAddressManager
toString
in interface IAddressManager
addr
- The opaque identifier that is the within store locator for
some datum.public boolean requiresCommit(IRootBlockView block)
The state of the provided block is not relevant since it does not hold
information on recent allocations (the meta allocations will only effect
the root block after a commit). This is passed through to the
RWStore
which examines its internal state.
requiresCommit
in interface IBufferStrategy
block
- The root block held by the client, can be checked against the
state of the Buffer Strategypublic boolean isDirty()
isDirty
in interface IBufferStrategy
public long getMetaBitsAddr()
IBufferStrategy
getMetaBitsAddr
in interface IBufferStrategy
public long getMetaStartAddr()
IBufferStrategy
getMetaStartAddr
in interface IBufferStrategy
public int getMaxRecordSize()
getMaxRecordSize
in interface IBufferStrategy
public int getOffsetBits()
getOffsetBits
in interface IBufferStrategy
public RWStore getStore()
IRWStrategy
IStore
.getStore
in interface IRWStrategy
public long getPhysicalAddress(long addr)
IAddressManager
getPhysicalAddress
in interface IAddressManager
addr
- The encoded addresspublic void writeRawBuffer(IHAWriteMessage msg, IBufferAccess b) throws IOException, InterruptedException
IHABufferStrategy
writeRawBuffer
in interface IHABufferStrategy
IOException
InterruptedException
public Future<Void> sendHALogBuffer(IHALogRequest req, IHAWriteMessage msg, IBufferAccess b) throws IOException, InterruptedException
IHABufferStrategy
IHAWriteMessage
and the associated raw buffer through the
write pipeline.sendHALogBuffer
in interface IHABufferStrategy
req
- The IHALogRequest
for some HALog file.msg
- The IHAWriteMessage
.b
- The raw buffer. Bytes from position to limit will be sent.
remaining() must equal IHAWriteMessageBase.getSize()
.Future
for that request.IOException
InterruptedException
public Future<Void> sendRawBuffer(IHARebuildRequest req, long sequence, long quorumToken, long fileExtent, long offset, int nbytes, ByteBuffer b) throws IOException, InterruptedException
IHABufferStrategy
IHAWriteMessage
and the associated raw buffer through the
write pipeline.sendRawBuffer
in interface IHABufferStrategy
req
- The IHARebuildRequest
to replicate the backing file to
the requesting service.sequence
- The sequence of this IHAWriteMessage
(origin ZERO
(0)).quorumToken
- The quorum token of the leader, which must remain valid across
the rebuild protocol.fileExtent
- The file extent as of the moment that the leader begins to
replicate the existing backing file.offset
- The starting offset (relative to the root blocks).nbytes
- The #of bytes to be sent.b
- The raw buffer. The buffer will be cleared and filled with the
specified data, then sent down the write pipeline.Future
for that request.IOException
InterruptedException
public void writeOnStream(OutputStream os, AbstractJournal.ISnapshotData snapshotData, Quorum<HAGlue,QuorumService<HAGlue>> quorum, long token) throws IOException, QuorumException
IHABufferStrategy
Note: The caller is able to obtain both root blocks atomically, while the strategy may not be aware of the root blocks or may not be able to coordinate their atomic capture.
Note: The caller must ensure that the resulting snapshot will be consistent either by ensuring that no writes occur or by taking a read-lock that will prevent overwrites of committed state during this operation.
writeOnStream
in interface IHABufferStrategy
os
- Where to write the data.quorum
- The Quorum
.token
- The token that must remain valid during this operation.IOException
QuorumException
- if the service is not joined with the met quorum for that
token at any point during the operation.public Object snapshotAllocators()
IHABufferStrategy
snapshotAllocators
in interface IHABufferStrategy
null
if the snapshot is a NOP for the
IBufferStrategy
(e.g., for the WORM).public void computeDigest(Object snapshot, MessageDigest digest) throws DigestException, IOException
IHABufferStrategy
Note: The digest is not reliable unless you either use a snapshot or suspend writes (on the quorum) while it is computed.
computeDigest
in interface IHABufferStrategy
snapshot
- The allocator snapshot (optional). When given, the digest is
computed only for the snapshot. When null
it is
computed for the entire file.DigestException
IOException
public ByteBuffer readFromLocalStore(long addr) throws InterruptedException
IHABufferStrategy
Quorum
.readFromLocalStore
in interface IHABufferStrategy
InterruptedException
public void setExtentForLocalStore(long extent) throws IOException, InterruptedException
setExtentForLocalStore
in interface IHABufferStrategy
InterruptedException
IOException
IHABufferStrategy.setExtentForLocalStore(long)
public boolean lockAddress(long addr)
addr
- - address to be lockedpublic long getLastReleaseTime()
IHistoryManager
getLastReleaseTime
in interface IHistoryManager
public int checkDeferredFrees(AbstractJournal journal)
checkDeferredFrees
in interface IHistoryManager
AbstractJournal#commitNow()
public boolean isCommitted(long addr)
RWStore
in memory bit maps.isCommitted
in interface IRWStrategy
addr
- The address.true
iff the address is currently committed.public boolean inWriteCache(long addr)
true
iff the address was in the write cache as of the
moment the write cache was inspected.addr
- The address.public IRawTx newTx()
IHistoryManager
IStore
. As long as
a transaction is active we can not release data which is currently marked
as freed but was committed at the point the session started.newTx
in interface IHistoryManager
public void registerExternalCache(ConcurrentWeakValueCache<Long,ICommitter> historicalIndexCache, int byteCount)
IHistoryManager
Note: It is not safe to clear at the point of the delete request since the data could still be loaded if the data is retained for a period due to a non-zero retention period or session protection.
registerExternalCache
in interface IHistoryManager
public long saveDeferrals()
IHistoryManager
Writes the content of currentTxnFreeList to the store.
These are the current buffered frees that have yet been saved into a block referenced from the deferredFreeList
saveDeferrals
in interface IHistoryManager
DeleteBlockCommitter
public InputStream getInputStream(long addr)
IStreamStore
getInputStream
in interface IStreamStore
getInputStream
in class AbstractRawStore
addr
- The address at which the stream was written.public IPSOutputStream getOutputStream()
IStreamStore
IPSOutputStream
.getOutputStream
in interface IStreamStore
getOutputStream
in class AbstractRawStore
public IPSOutputStream getOutputStream(IAllocationContext context)
IAllocationManagerStore
IPSOutputStream
.getOutputStream
in interface IAllocationManagerStore
context
- The context within which any allocations are made by the
returned IPSOutputStream
.public void resetFromHARootBlock(IRootBlockView rootBlock)
IHABufferStrategy
Note: This method is used when the root blocks of the leader are
installed onto a follower. This can change the UUID
for the
backing store file. The IHABufferStrategy
implementation MUST
update any cached value for that UUID
.
Use IHABufferStrategy.postHACommit(IRootBlockView)
rather than this method in the
2-phase commit on the follower.
resetFromHARootBlock
in interface IHABufferStrategy
public long getBlockSequence()
IHABufferStrategy
WriteCache
blocks that were written out for the
last write set. This is used to communicate the #of write cache blocks in
the commit point back to AbstractJournal.commitNow(long)
. It is
part of the commit protocol.
Note: This DOES NOT reflect the current value of the block sequence
counter for ongoing writes. That counter is owned by the
WriteCacheService
.
getBlockSequence
in interface IHABufferStrategy
WriteCacheService.resetSequence()
,
FIXME I would prefer to expose the {@link WriteCacheService} to the
{@link AbstractJournal} and let it directly invoke
{@link WriteCacheService#resetSequence()}. The current pattern
requires the {@link IHABufferStrategy} implementations to track the
lastBlockSequence and is messy.
public long getCurrentBlockSequence()
IHABufferStrategy
getCurrentBlockSequence
in interface IHABufferStrategy
IHABufferStrategy.getBlockSequence()
public ByteBuffer readRaw(long position, ByteBuffer transfer)
IHABufferStrategy
readRaw
in interface IHABufferStrategy
position
- - absolute file offsettransfer
- - target buffer for readpublic void writeRawBuffer(HARebuildRequest req, IHAWriteMessage msg, ByteBuffer transfer) throws IOException
IHABufferStrategy
writeRawBuffer
in interface IHABufferStrategy
IOException
public Lock getCommitLock()
IRWStrategy
Lock
that must be used (when non-
null
) to make the IBufferStrategy.commit()
/
IRWStrategy.postCommit()
strategy atomic.getCommitLock
in interface IRWStrategy
public void postCommit()
IRWStrategy
Note: It is the responsibility of the commit protocol layers to wind up
calling IBufferStrategy.abort()
if there is a failure during the
commit protocol.
postCommit
in interface IRWStrategy
public void postHACommit(IRootBlockView rootBlock)
IHABufferStrategy
For the RWStore this is used to resynchronize the allocators during the 2-phase commit on the follower with the delta in the allocators from the write set associated with that commit.
postHACommit
in interface IHABufferStrategy
rootBlock
- The newly installed root block.public WriteCacheService getWriteCacheService()
IHABufferStrategy
WriteCacheService
(mainly for debugging).getWriteCacheService
in interface IHABufferStrategy
public StoreState getStoreState()
IHABufferStrategy
getStoreState
in interface IHABufferStrategy
public IAllocationContext newAllocationContext(boolean isolated)
IAllocationManager
newAllocationContext
in interface IAllocationManager
Copyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.