com.bigdata.rwstore.sector

Interface IMemoryManager

All Superinterfaces:

IAllocationManager, IAllocationManagerStore, ICounterSetAccess, IHistoryManager, IStore, IStreamStore

All Known Implementing Classes:

AllocationContext, MemoryManager

public interface IMemoryManager
extends IStore, ICounterSetAccess, IAllocationManagerStore

Abstraction for managing data in ByteBuffers. Typically those buffers will be allocated on the native process heap.

CAUTION: The memory manager helps you manage direct storage. However, it does not prevent you from doing something stupid with it. The most likely error is one in which you get(long) an address and hold onto the returned ByteBuffer after the data at that address has been freed. This can leave you in a position where you are reading on (or writing on!) someone else's data on the JVM native heap. The memory manager works with DirectBufferPools. This class provides the efficient reuse of direct allocations, but does not release them back to the JVM. For this reason, if you do stomp on someone's memory, it will not be memory in use by the JVM but only memory in use by another part of your application using the same DirectBufferPool instance.

Author:

martyncutcher

Method Summary

Methods

Modifier and Type	Method and Description
long	allocate(ByteBuffer data) Version of allocate(ByteBuffer, boolean) which is either blocking or non-blocking depending on whether or not the memory manager is set in a blocking mode.
long	allocate(ByteBuffer data, boolean blocks) Allocates space on the backing resource and copies the provided data.
long	allocate(ByteBuffer data, IAllocationContext context)
long	allocate(int nbytes) Return the address of a new allocation sufficient to store the specified number of bytes of application data.
long	allocate(int nbytes, boolean blocks) Return the address of a new allocation sufficient to store the specified number of bytes of application data.
int	allocationSize(long addr) Return the size of the application data for the allocation with the given address.
void	clear() Clears all current allocations.
IMemoryManager	createAllocationContext() Create a child allocation context within which the caller may make and release allocations.
void	free(long addr) Frees the address and makes available for recycling
void	free(long addr, IAllocationContext context)
ByteBuffer[]	get(long addr) Return an array of ByteBuffers providing an updatable view onto the backing allocation.
long	getAllocationCount() The #of allocation spanned by this allocation context (including any any child allocation contexts).
int	getMaxSectors() The maximum number of backing buffers which may be allocated by the IMemoryManager and Integer.MAX_VALUE if there is no effective limit on the #of backing buffers which may be allocated.
long	getPhysicalAddress(long addr) Determine the unencoded physical address
int	getSectorCount() The #of backing buffers in use.
int	getSectorSize() The size of a backing buffer in bytes.
long	getSlotBytes() Return the #of bytes of consumed by allocation slots allocated against this IMemoryManager (including any child allocation contexts).
long	getUserBytes() Return the #of bytes of application data allocated against this IMemoryManager (including any child allocation contexts).
boolean	isCommitted(long addr) Return true iff the allocation having that address is flagged as committed.
byte[]	read(long addr) Return a copy of the data stored at that address.

Methods inherited from interface com.bigdata.rwstore.IStore

alloc, close, commit, free, getAssociatedSlotSize, getCommitLock, getData, getStoreFile, postCommit

Methods inherited from interface com.bigdata.rwstore.IAllocationManager

abortContext, detachContext, newAllocationContext

Methods inherited from interface com.bigdata.rwstore.IHistoryManager

checkDeferredFrees, getLastReleaseTime, newTx, registerExternalCache, saveDeferrals

Methods inherited from interface com.bigdata.counters.ICounterSetAccess

getCounters

Methods inherited from interface com.bigdata.rawstore.IAllocationManagerStore

delete, getOutputStream, write

Methods inherited from interface com.bigdata.rawstore.IStreamStore

getInputStream, getOutputStream

Method Detail

allocate

long allocate(ByteBuffer data,
            boolean blocks)

Allocates space on the backing resource and copies the provided data.

Parameters:

data - The data will be copied to the backing resource. For each buffer in this array, the position will be advanced to the limit.

blocks - When true the request will block until the memory is available for the allocation.

Returns:

the address to be passed to the get method to retrieve the data.

Throws:

IllegalArgumentException - if data is null.

IllegalArgumentException - if Buffer.remaining() is ZERO (0).

MemoryManagerResourceError - If the memory is not available for the allocation and blocks:=false. Whether or not this exception can be thrown depends on whether the backing buffer pool has a bounded capacity and whether the IMemoryManager using that pool has a bounded capacity.

allocate

long allocate(ByteBuffer data)

Version of allocate(ByteBuffer, boolean) which is either blocking or non-blocking depending on whether or not the memory manager is set in a blocking mode.

Parameters:

data - The data will be copied to the backing resource. For each buffer in this array, the position will be advanced to the limit.

Throws:

IllegalArgumentException - if data is null.

IllegalArgumentException - if Buffer.remaining() is ZERO (0).

MemoryManagerResourceError - If the memory is not available for the allocation and blocks:=false. Whether or not this exception can be thrown depends on whether the backing buffer pool has a bounded capacity and whether the IMemoryManager using that pool has a bounded capacity.

allocate

long allocate(int nbytes,
            boolean blocks)

Return the address of a new allocation sufficient to store the specified number of bytes of application data.

Parameters:

nbytes - The size of the allocation request.

blocks - When true the method will block until the allocation request can be satisfied. When false a MemoryManagerOutOfMemory will be thrown.

Returns:

The address of the allocation.

Throws:

IllegalArgumentException - if nbytes is non-positive.

MemoryManagerResourceError - If the memory is not available for the allocation and blocks:=false. Whether or not this exception can be thrown depends on whether the backing buffer pool has a bounded capacity and whether the IMemoryManager using that pool has a bounded capacity.

allocate

long allocate(int nbytes)

Return the address of a new allocation sufficient to store the specified number of bytes of application data. This is the blocking version of allocate(int).

Parameters:

nbytes - The size of the allocation request.

blocks - When true the method will block until the allocation request can be satisfied. When false a MemoryManagerOutOfMemory will be thrown.

Returns:

The address of the allocation.

Throws:

IllegalArgumentException - if nbytes is non-positive.

MemoryManagerResourceError - If the memory is not available for the allocation and blocks:=false. Whether or not this exception can be thrown depends on whether the backing buffer pool has a bounded capacity and whether the IMemoryManager using that pool has a bounded capacity.

allocate

long allocate(ByteBuffer data,
            IAllocationContext context)

get

ByteBuffer[] get(long addr)

Return an array of ByteBuffers providing an updatable view onto the backing allocation.

The ByteBuffer[] return enables the handling of blobs that span more than a single slot, without the need to create an intermediate ByteBuffer. This method is designed for use with zero-copy NIO. Furthermore, since the ByteBuffers in the returned array are not read-only, they can be updated directly. In this way the allocate(int) can be used in conjunction with get to provide more flexibility when storing data.

Using ByteBuffer:put the returned array can be efficiently copied to another ByteBuffer:

 ByteBuffer mybb;
 ByteBuffer[] bufs = get(addr);
 for (ByteBuffer b : bufs) {
        mybb.put(b);
 }
 

CAUTION: Do not hold onto the ByteBuffer longer than is necessary. If the allocation is released by free(long) or clear(), then the memory backing the ByteBuffer could be reallocated by another DirectBufferPool consumer.

Parameters:

addr - An previously allocated address.

Returns:

array of ByteBuffers

read

byte[] read(long addr)

Return a copy of the data stored at that address. This method is intended for use with patterns where the IMemoryManager is treated as a persistence store.

Parameters:

addr - The address.

Returns:

A copy of the data stored at that address.

free

void free(long addr)

Frees the address and makes available for recycling

Parameters:

addr - to be freed

Throws:

IllegalArgumentException - If the address is known to be invalid (never written or deleted). Note that the address 0L is always invalid as is any address which encodes a 0 byte length.

free

void free(long addr,
        IAllocationContext context)

clear

void clear()

Clears all current allocations. Clearing an allocation context makes the backing heap storage available to immediate reallocation. Clearing the allocation context of the top-level IMemoryManager will release any direct ByteBuffers back to the pool from which they were allocated.

CAUTION: Do not clear an allocation context until you know that all threads with access to that allocation context have either been terminated or released their reference to that allocation context.

createAllocationContext

IMemoryManager createAllocationContext()

Create a child allocation context within which the caller may make and release allocations. TODO Why is the return not an IAllocationContext as well (that is, why does IMemoryManager not extend IAllocationContext ). Also, note that IStore.commit() is similar to IAllocationManager.detachContext(IAllocationContext).

allocationSize

int allocationSize(long addr)

Return the size of the application data for the allocation with the given address.

Parameters:

addr - The address.

Returns:

The #of bytes of in the applications allocation request.

getAllocationCount

long getAllocationCount()

The #of allocation spanned by this allocation context (including any any child allocation contexts).

getUserBytes

long getUserBytes()

Return the #of bytes of application data allocated against this IMemoryManager (including any child allocation contexts). Due to the overhead of the storage allocation scheme, this value may be smaller than getSlotBytes().

getSlotBytes

long getSlotBytes()

Return the #of bytes of consumed by allocation slots allocated against this IMemoryManager (including any child allocation contexts).

getSectorSize

int getSectorSize()

The size of a backing buffer in bytes.

getSectorCount

int getSectorCount()

The #of backing buffers in use.

getMaxSectors

int getMaxSectors()

The maximum number of backing buffers which may be allocated by the IMemoryManager and Integer.MAX_VALUE if there is no effective limit on the #of backing buffers which may be allocated.

isCommitted

boolean isCommitted(long addr)

Return true iff the allocation having that address is flagged as committed. The caller must be holding the allocation lock in order for the result to remain valid outside of the method call.

Parameters:

addr - The address.

Returns:

true iff the address is currently committed.

getPhysicalAddress

long getPhysicalAddress(long addr)

Determine the unencoded physical address

Parameters:

addr - The encoded address

Returns:

an unencoded address offset