com.bigdata.journal

Class AbstractBufferStrategy

java.lang.Object

com.bigdata.rawstore.AbstractRawStore

com.bigdata.rawstore.AbstractRawWormStore

com.bigdata.journal.AbstractBufferStrategy

All Implemented Interfaces:

ICounterSetAccess, IBufferStrategy, IAddressManager, IMRMW, IMROW, IRawStore, IStreamStore, IWORM

Direct Known Subclasses:

BasicBufferStrategy, DiskOnlyStrategy, WORMStrategy

public abstract class AbstractBufferStrategy
extends AbstractRawWormStore
implements IBufferStrategy

Abstract base class for IBufferStrategy implementation.

Version:

$Id$

Author:

Bryan Thompson

Field Summary

Fields

Modifier and Type	Field and Description
protected BufferMode	bufferMode The buffer strategy implemented by this class.
protected AtomicLong	commitOffset The WORM address of the last committed allocation.
static String	ERR_ADDRESS_IS_NULL Text of the error message used when a ZERO (0L) is passed as an address to IRawStore.read(long) or similar methods.
static String	ERR_ADDRESS_NOT_WRITTEN Text of the error message used when an address is given has never been written.
static String	ERR_BAD_RECORD_SIZE Error message used when the record size is invalid (e.g., negative).
static String	ERR_BUFFER_EMPTY Text of the error message used when a ByteBuffer with zero bytes Buffer.remaining() is passed to IRawStore.write(ByteBuffer).
static String	ERR_BUFFER_NULL Text of the error message used when a null reference is provided for a ByteBuffer.
static String	ERR_BUFFER_OVERRUN Error message used when an operation would write more data than would be permitted onto a buffer.
static String	ERR_MAX_EXTENT Text of the error message used when a write operation would exceed the maximum extent for a backing store.
static String	ERR_NOT_OPEN Error message used when the store is closed but the operation requires that the store is open.
static String	ERR_OPEN Error message used when the store is open by the operation requires that the store is closed.
static String	ERR_READ_ONLY Error message used when the writes are not allowed.
static String	ERR_RECORD_LENGTH_ZERO Text of the error message used when an address provided to IRawStore.read(long) or a similar method encodes a record length of zero (0).
static String	ERR_TRUNCATE Text of the error message used when IBufferStrategy.truncate(long) would truncate data that has already been written.
protected long	initialExtent
protected static org.apache.log4j.Logger	log Log for buffer operations.
protected long	maximumExtent
protected AtomicLong	nextOffset The next offset at which a data item would be written on the store as an offset into the user extent (offset zero(0) addresses the first byte after the root blocks).
protected static boolean	WARN

Fields inherited from class com.bigdata.rawstore.AbstractRawWormStore

am

Fields inherited from interface com.bigdata.rawstore.IAddressManager

NULL

Method Summary

Methods

Modifier and Type	Method and Description
void	abort() A method that requires the implementation to discard its buffered write set (if any).
protected void	assertOpen()
void	close() Manages the open flag state.
void	closeForWrites() Sets the readOnly flag.
void	commit() A method that removes assumptions of how a specific strategy commits data.
void	delete(long addr) The default is a NOP.
void	destroy() Closes the store immediately (if open) and deletes its persistent resources.
BufferMode	getBufferMode() The buffer mode supported by the implementation
long	getInitialExtent() The initial extent.
long	getMaximumExtent() The maximum extent allowable before a buffer overflow operation will be rejected.
int	getMaxRecordSize() The maximum size of a record for the address manager less 4 bytes iff checksums are enabled.
long	getMetaBitsAddr() The RWStrategy requires meta allocation info in the root block, this method is the hook to enable access.
long	getMetaStartAddr() The RWStrategy requires meta allocation info in the root block, this method is the hook to enable access.
protected long	getMinimumExtension() The minimum amount to extend the backing storage when it overflows.
long	getNextOffset() The next offset at which a data item would be written on the store as an offset into the user extent.
IResourceMetadata	getResourceMetadata() Not supported - this is available on the AbstractJournal.
UUID	getUUID() Not supported - this is available on the AbstractJournal.
boolean	isDirty() Return true if the store has been modified since the last IBufferStrategy.commit() or IBufferStrategy.abort().
boolean	isOpen() true iff the store is open.
boolean	isReadOnly() true iff the store does not allow writes.
boolean	overflow(long needed) Invoked if the store would exceed its current extent by IRawStore.write(ByteBuffer).
boolean	requiresCommit(IRootBlockView block) A method that removes assumptions of how a specific strategy determines whether a transaction commit is required.
long	size() The #of application data bytes written on the store (does not count any headers or root blocks that may exist for the store).
protected static long	transferFromDiskTo(IDiskBasedStrategy src, RandomAccessFile out) Helper method used by DiskBackedBufferStrategy and DiskOnlyStrategy to implement IBufferStrategy.transferTo(RandomAccessFile) using a FileChannel to FileChannel transfer.
boolean	useChecksums() false by default since these were added for HA with the WORMStrategy and the RWStrategy.

Methods inherited from class com.bigdata.rawstore.AbstractRawWormStore

getAddressManager, getByteCount, getOffset, getOffsetBits, getPhysicalAddress, toAddr, toString

Methods inherited from class com.bigdata.rawstore.AbstractRawStore

getInputStream, getOutputStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.journal.IBufferStrategy

getAddressManager, getCounters, getExtent, getHeaderSize, getOffsetBits, getUserExtent, readRootBlock, transferTo, truncate, writeRootBlock

Methods inherited from interface com.bigdata.rawstore.IRawStore

deleteResources, force, getFile, isFullyBuffered, isStable, read, write

Methods inherited from interface com.bigdata.rawstore.IAddressManager

getByteCount, getOffset, getPhysicalAddress, toAddr, toString

Methods inherited from interface com.bigdata.rawstore.IStreamStore

getInputStream, getOutputStream

Field Detail

log

protected static final org.apache.log4j.Logger log

Log for buffer operations.

WARN

protected static final boolean WARN

ERR_BUFFER_EMPTY

public static final String ERR_BUFFER_EMPTY

Text of the error message used when a ByteBuffer with zero bytes Buffer.remaining() is passed to IRawStore.write(ByteBuffer).

See Also:

Constant Field Values

ERR_BUFFER_NULL

public static final String ERR_BUFFER_NULL

Text of the error message used when a null reference is provided for a ByteBuffer.

See Also:

Constant Field Values

ERR_ADDRESS_NOT_WRITTEN

public static final String ERR_ADDRESS_NOT_WRITTEN

Text of the error message used when an address is given has never been written. Since the journal is an append-only store, an address whose offset plus record length exceeds the nextOffset on which data would be written may be easily detected.

See Also:

Constant Field Values

ERR_ADDRESS_IS_NULL

public static final String ERR_ADDRESS_IS_NULL

Text of the error message used when a ZERO (0L) is passed as an address to IRawStore.read(long) or similar methods. This value 0L is reserved to indicate a persistent null reference and may never be read.

See Also:

Constant Field Values

ERR_RECORD_LENGTH_ZERO

public static final String ERR_RECORD_LENGTH_ZERO

Text of the error message used when an address provided to IRawStore.read(long) or a similar method encodes a record length of zero (0). Empty records are not permitted on write and addresses with a zero length are rejected on read.

See Also:

Constant Field Values

ERR_MAX_EXTENT

public static final String ERR_MAX_EXTENT

Text of the error message used when a write operation would exceed the maximum extent for a backing store.

See Also:

Constant Field Values

ERR_TRUNCATE

public static final String ERR_TRUNCATE

Text of the error message used when IBufferStrategy.truncate(long) would truncate data that has already been written.

See Also:

Constant Field Values

ERR_READ_ONLY

public static final String ERR_READ_ONLY

Error message used when the writes are not allowed.

See Also:

Constant Field Values

ERR_BAD_RECORD_SIZE

public static final String ERR_BAD_RECORD_SIZE

Error message used when the record size is invalid (e.g., negative).

See Also:

Constant Field Values

ERR_NOT_OPEN

public static final String ERR_NOT_OPEN

Error message used when the store is closed but the operation requires that the store is open.

See Also:

Constant Field Values

ERR_OPEN

public static final String ERR_OPEN

Error message used when the store is open by the operation requires that the store is closed.

See Also:

Constant Field Values

ERR_BUFFER_OVERRUN

public static final String ERR_BUFFER_OVERRUN

Error message used when an operation would write more data than would be permitted onto a buffer.

See Also:

Constant Field Values

initialExtent

protected final long initialExtent

maximumExtent

protected final long maximumExtent

bufferMode

protected final BufferMode bufferMode

The buffer strategy implemented by this class.

nextOffset

protected final AtomicLong nextOffset

The next offset at which a data item would be written on the store as an offset into the user extent (offset zero(0) addresses the first byte after the root blocks). This is updated each time a new record is written on the store. On restart, the value is initialized from the current root block. The current value is written as part of the new root block during each commit.

Note: It is NOT safe to reload the current root block and therefore reset this to an earlier offset unless all transactions are discarded. The reason is that transactions may use objects (btrees) to provide isolation. Those objects write on the store but do not register as ICommitters and therefore never make themselves restart safe. However, you can not discard the writes of those objects unless the entire store is being restarted, e.g., after a shutdown or a crash.

Note: An AtomicLong is used to provide an object on which we can lock when assigning the next record's address and synchronously updating the counter value. It also ensures that threads can not see a stale value for the counter.

commitOffset

protected final AtomicLong commitOffset

The WORM address of the last committed allocation.

Method Detail

getInitialExtent

public final long getInitialExtent()

Description copied from interface: IBufferStrategy

The initial extent.

Specified by:

getInitialExtent in interface IBufferStrategy

getMaximumExtent

public final long getMaximumExtent()

Description copied from interface: IBufferStrategy

The maximum extent allowable before a buffer overflow operation will be rejected.

Note: The semantics here differ from those defined by Options.MAXIMUM_EXTENT. The latter specifies the threshold at which a journal will overflow (onto another journal) while this specifies the maximum size to which a buffer is allowed to grow.

Note: This is normally zero (0L), which basically means that the maximum extent is ignored by the IBufferStrategy but respected by the AbstractJournal, resulting in a soft limit on journal overflow.

Specified by:

getMaximumExtent in interface IBufferStrategy

Returns:

The maximum extent permitted for the buffer -or- 0L iff no limit is imposed.

getMinimumExtension

protected long getMinimumExtension()

The minimum amount to extend the backing storage when it overflows.

getBufferMode

public final BufferMode getBufferMode()

Description copied from interface: IBufferStrategy

The buffer mode supported by the implementation

Specified by:

getBufferMode in interface IBufferStrategy

Returns:

The implemented buffer mode.

getNextOffset

public final long getNextOffset()

Description copied from interface: IBufferStrategy

The next offset at which a data item would be written on the store as an offset into the user extent.

Specified by:

getNextOffset in interface IBufferStrategy

size

public final long size()

Description copied from interface: IRawStore

The #of application data bytes written on the store (does not count any headers or root blocks that may exist for the store).

Specified by:

size in interface IRawStore

assertOpen

protected final void assertOpen()

isOpen

public boolean isOpen()

Description copied from interface: IRawStore

true iff the store is open.

Specified by:

isOpen in interface IRawStore

Returns:

true iff the store is open.

isReadOnly

public boolean isReadOnly()

Description copied from interface: IRawStore

true iff the store does not allow writes.

Specified by:

isReadOnly in interface IRawStore

close

public void close()

Manages the open flag state.

Specified by:

close in interface IRawStore

destroy

public final void destroy()

Description copied from interface: IRawStore

Closes the store immediately (if open) and deletes its persistent resources. Does NOT throw an IllegalStateException if the store is already closed, but still deletes the backing resources.

Specified by:

destroy in interface IRawStore

See Also:

IRawStore.deleteResources()

overflow

public final boolean overflow(long needed)

Invoked if the store would exceed its current extent by IRawStore.write(ByteBuffer). The default behavior extends the capacity of the buffer by the at least the requested amount and a maximum of 32M or the Options.INITIAL_EXTENT.

If the data are fully buffered, then the maximum store size is limited to int32 bytes which is the maximum #of bytes that can be addressed in RAM (the pragmatic maximum is slightly less than 2G due to the limits of the JVM to address system memory).

Returns:

true if the capacity of the store was extended and the write operation should be retried.

transferFromDiskTo

protected static long transferFromDiskTo(IDiskBasedStrategy src,
                      RandomAccessFile out)
                                  throws IOException

Helper method used by DiskBackedBufferStrategy and DiskOnlyStrategy to implement IBufferStrategy.transferTo(RandomAccessFile) using a FileChannel to FileChannel transfer.

Parameters:

src - The source.

out - The output file.

Returns:

The #of bytes transferred.

Throws:

IOException

getUUID

public UUID getUUID()

Not supported - this is available on the AbstractJournal.

Specified by:

getUUID in interface IRawStore

Throws:

UnsupportedOperationException - always

getResourceMetadata

public IResourceMetadata getResourceMetadata()

Not supported - this is available on the AbstractJournal.

Specified by:

getResourceMetadata in interface IRawStore

Throws:

UnsupportedOperationException - always

closeForWrites

public void closeForWrites()

Sets the readOnly flag.

Note: This method SHOULD be extended to release write caches, etc.

Specified by:

closeForWrites in interface IBufferStrategy

delete

public void delete(long addr)

The default is a NOP.

Specified by:

delete in interface IRawStore

Overrides:

delete in class AbstractRawStore

Parameters:

addr - A long integer formed using Addr that encodes both the offset at which the data was written and the #of bytes that were written.

isDirty

public boolean isDirty()

Return true if the store has been modified since the last IBufferStrategy.commit() or IBufferStrategy.abort().

This implementation checks the current allocation offset with that in the rootBlock

Specified by:

isDirty in interface IBufferStrategy

Returns:

true if store has been modified since last commit()

commit

public void commit()

Description copied from interface: IBufferStrategy

A method that removes assumptions of how a specific strategy commits data. For most strategies the action is void since the client WORM DISK strategy writes data as allocated. For the Read Write Strategy more data must be managed as part of the protocol outside of the RootBlock, and this is the method that triggers that management. The caller MUST provide appropriate synchronization.

Specified by:

commit in interface IBufferStrategy

abort

public void abort()

Description copied from interface: IBufferStrategy

A method that requires the implementation to discard its buffered write set (if any). The caller is responsible for any necessary synchronization as part of the abort protocol.

Specified by:

abort in interface IBufferStrategy

getMetaBitsAddr

public long getMetaBitsAddr()

Description copied from interface: IBufferStrategy

The RWStrategy requires meta allocation info in the root block, this method is the hook to enable access. The metaBitsAddr is the address in the file where the metaBits that control the allocation of the allocation blocks themselves is stored.

Specified by:

getMetaBitsAddr in interface IBufferStrategy

Returns:

the metaBitsAddr for the root block if any

getMetaStartAddr

public long getMetaStartAddr()

Description copied from interface: IBufferStrategy

The RWStrategy requires meta allocation info in the root block, this method is the hook to enable access. The metaStartAddr is the address in the file where the allocation blocks are stored.

Specified by:

getMetaStartAddr in interface IBufferStrategy

Returns:

the metaStartAddr for the root block if any

requiresCommit

public boolean requiresCommit(IRootBlockView block)

Description copied from interface: IBufferStrategy

A method that removes assumptions of how a specific strategy determines whether a transaction commit is required.

Specified by:

requiresCommit in interface IBufferStrategy

Parameters:

block - The root block held by the client, can be checked against the state of the Buffer Strategy

Returns:

whether any modification has occurred.

getMaxRecordSize

public int getMaxRecordSize()

The maximum size of a record for the address manager less 4 bytes iff checksums are enabled.

Specified by:

getMaxRecordSize in interface IBufferStrategy

Returns:

the maximum record size supported by this strategy

useChecksums

public boolean useChecksums()

false by default since these were added for HA with the WORMStrategy and the RWStrategy.

Specified by:

useChecksums in interface IBufferStrategy