public class MemoryManager extends Object implements IMemoryManager, ISectorManager
ByteBuffer. It uses the
new SectorAllocator to allocate slots within the address range.
The interface is designed to support efficient transfer between NIO buffers.
The most complex aspect of the implementation is the BLOB representation,
requiring a mapping across multiple allocation slots. This is managed using
recursive calls in the main three methods: allocate, free and get.| Constructor and Description |
|---|
MemoryManager(DirectBufferPool pool)
Create a new
MemoryManager. |
MemoryManager(DirectBufferPool pool,
int sectors)
Create a new
MemoryManager. |
MemoryManager(DirectBufferPool pool,
int sectors,
boolean blocks,
Properties properties)
Create a new
MemoryManager. |
| Modifier and Type | Method and Description |
|---|---|
void |
abortContext(IAllocationContext context)
Indicates that the allocation context will no longer be used and that the
allocations made within the context should be discarded.
|
void |
addToFreeList(SectorAllocator sector)
When sufficient allocations have been freed for recycling that a
threshold of availability of reached for all block sizes, then the
allocator calls back to the SectorManager to signal it is available to be
returned to the free list.
|
long |
alloc(byte[] buf,
int size,
IAllocationContext context)
Writes data on the store.
|
long |
allocate(ByteBuffer data)
Version of
IMemoryManager.allocate(ByteBuffer, boolean) which is either
blocking or non-blocking depending on whether or not the memory
manager is set in a blocking mode. |
long |
allocate(ByteBuffer data,
boolean blocks)
Allocates space on the backing resource and copies the provided data.
|
long |
allocate(ByteBuffer data,
IAllocationContext context) |
long |
allocate(int nbytes)
Return the address of a new allocation sufficient to store the specified
number of bytes of application data.
|
long |
allocate(int nbytes,
boolean blocks)
Return the address of a new allocation sufficient to store the specified
number of bytes of application data.
|
int |
allocationSize(long addr)
Return the size of the application data for the allocation with the given
address.
|
int |
checkDeferredFrees(AbstractJournal journal)
This method is invoked during the commit protocol and gives the backing
store an opportunity to check whether storage associated with deferred
frees can now be released.
|
void |
clear()
Clears all current allocations.
|
void |
close()
Close the backing storage.
|
void |
commit()
Global commit on the backing store.
|
IMemoryManager |
createAllocationContext()
Maintain allocationContext to check for session protection
|
void |
deferFree(int rwaddr,
int sze) |
void |
delete(long addr,
IAllocationContext context)
Delete the data associated with the address within the allocation
context.
|
void |
detachContext(IAllocationContext context)
Indicates that the allocation context will no longer be used, but that
the allocations made within the context should be preserved.
|
protected void |
finalize() |
void |
free(long addr)
Frees the address and makes available for recycling
|
void |
free(long addr,
IAllocationContext context) |
void |
free(long addr,
int size)
Frees allocated storage (clears the bit to enable recycling after
the next commit).
|
ByteBuffer[] |
get(long addr)
Return an array of
ByteBuffers providing an updatable view onto
the backing allocation. |
long |
getAllocationCount()
The #of allocation spanned by this allocation context (including any
any child allocation contexts).
|
int |
getAssociatedSlotSize(int addr) |
long |
getCapacity()
The maximum #of bytes which are available to the memory manager.
|
Lock |
getCommitLock()
Optionally return a
Lock that must be used (when non-
null) to make the IStore.commit() / IStore.postCommit()
strategy atomic. |
CounterSet |
getCounters()
Return performance counters.
|
void |
getData(long l,
byte[] buf)
Read data of a known size from the store.
|
long |
getExtent()
The total #of bytes in the backing buffers currently attached to the
MemoryManager. |
InputStream |
getInputStream(long addr)
Return an input stream from which a previously written stream may be read
back.
|
long |
getLastReleaseTime()
If history is retained this returns the time for which data was most
recently released.
|
long |
getMaxMemoryCapacity()
The maximum capacity in bytes of the managed memory.
|
int |
getMaxSectors()
Return the maximum #of sectors which may be allocated.
|
IPSOutputStream |
getOutputStream()
Return an output stream which can be used to write on the backing store.
|
IPSOutputStream |
getOutputStream(IAllocationContext context)
Return an output stream which can be used to write on the backing store
within the given allocation context.
|
long |
getPhysicalAddress(long addr)
Determine the unencoded physical address
|
int |
getSectorCount()
Return the #of sectors which are currently in use.
|
int |
getSectorSize()
The size in bytes of the backing sector.
|
long |
getSlotBytes()
Return the #of bytes of consumed by allocation slots allocated against
this
IMemoryManager (including any child allocation contexts). |
File |
getStoreFile()
Retrieves store file.
|
long |
getUserBytes()
Return the #of bytes of application data allocated against this
IMemoryManager (including any child allocation contexts). |
boolean |
isBlocking()
Return
true iff the default policy of this
MemoryManager instance is to block if an allocation can not be
made (due to exhaustion of the maximum number of backing buffers for the
MemoryManager instance). |
boolean |
isCommitted(long addr)
Return
true iff the allocation having that address is
flagged as committed. |
IAllocationContext |
newAllocationContext(boolean isolated)
Creates a context to be used to isolate updates to within the context until it
is released to the parent environment.
|
IRawTx |
newTx()
A hook used to support session protection by incrementing and
decrementing a transaction counter within the
IStore. |
void |
postCommit()
Hook that supports synchronization with an external commit before which
a rollback to "pre-commit" state is supported.
|
byte[] |
read(long addr)
Return a copy of the data stored at that address.
|
void |
registerExternalCache(ConcurrentWeakValueCache<Long,ICommitter> externalCache,
int dataSize)
Call made from AbstractJournal to register the cache used.
|
void |
removeFromFreeList(SectorAllocator sector)
This request is made when the sectorAllocator no longer has a full set of
block allocations available.
|
long |
saveDeferrals()
Saves the current list of delete blocks, returning the address allocated.
|
String |
toString() |
void |
trimSector(long trim,
SectorAllocator sector)
When a sector is first created, it will remain at the head of the free
list until one of two conditions has been reached:
The allocation has been saturated.
The bit space has been filled.
|
long |
write(ByteBuffer data,
IAllocationContext context)
Write the data within the allocation context.
|
public MemoryManager(DirectBufferPool pool)
MemoryManager.
The backing DirectBufferPool may be either bounded or
(effectively) unbounded. The MemoryManager will be (effectively)
unbounded. If either the pool is bounded, then blocking
allocation requests may block, otherwise allocation requests will be
non-blocking.
The garbage collection of direct ByteBuffers depends on a full GC
pass. In an application which managers its heap pressure well, full GC
passes are rare. Therefore, the best practice is to share an unbounded
pool across multiple purposes. Since there are typically multiple users
of the pool, the demand can not always be predicated and deadlocks can
arise with a bounded pool.
Individual buffers will be allocated as necessary and released if they become empty. However, since allocation patterns may cause the in use data to be scattered across the allocated buffers, the backing buffers may not be returned to the backing pool until the top-level allocation context is cleared.
Any storage allocated by this instance will be released no later than
when the instance is finalized. Storage may be
returned to the pool within the life cycle of the MemoryManager
using clear(). Nested allocation contexts may be created and
managed using createAllocationContext().
pool - The pool from which the MemoryManager will allocate
its buffers (each "sector" is one buffer).IllegalArgumentException - if pool is null.public MemoryManager(DirectBufferPool pool, int sectors)
MemoryManager.
The backing DirectBufferPool may be either bounded or
(effectively) unbounded. The MemoryManager may also be bounded or
(effectively) unbounded. If either the pool or the memory manager is
bounded, then blocking allocation requests may block. Neither
non-blocking allocation requests nor allocation requests made against an
unbounded memory manager backed by an unbounded pool will block. The
preferred method for bounding the memory manager is to specify a maximum
#of buffers which it may consume from the pool.
The garbage collection of direct ByteBuffers depends on a full GC
pass. In an application which managers its heap pressure well, full GC
passes are rare. Therefore, the best practice is to share an unbounded
pool across multiple purposes. Since there are typically multiple users
of the pool, the demand can not always be predicated and deadlocks can
arise with a bounded pool.
Individual buffers will be allocated as necessary and released if they become empty. However, since allocation patterns may cause the in use data to be scattered across the allocated buffers, the backing buffers may not be returned to the backing pool until the top-level allocation context is cleared.
Any storage allocated by this instance will be released no later than
when the instance is finalized. Storage may be
returned to the pool within the life cycle of the MemoryManager
using clear(). Nested allocation contexts may be created and
managed using createAllocationContext().
pool - The pool from which the MemoryManager will allocate
its buffers (each "sector" is one buffer).sectors - The maximum #of buffers which the MemoryManager will
allocate from that pool (each "sector" is one buffer). This
may be Integer.MAX_VALUE for an effectively unbounded
capacity.IllegalArgumentException - if pool is null.IllegalArgumentException - if sectors is non-positive.public MemoryManager(DirectBufferPool pool, int sectors, boolean blocks, Properties properties)
MemoryManager.
The backing DirectBufferPool may be either bounded or
(effectively) unbounded. The MemoryManager may also be bounded or
(effectively) unbounded. If either the pool or the memory manager is
bounded, then blocking allocation requests may block. Neither
non-blocking allocation requests nor allocation requests made against an
unbounded memory manager backed by an unbounded pool will block. The
preferred method for bounding the memory manager is to specify a maximum
#of buffers which it may consume from the pool.
The garbage collection of direct ByteBuffers depends on a full GC
pass. In an application which managers its heap pressure well, full GC
passes are rare. Therefore, the best practice is to share an unbounded
pool across multiple purposes. Since there are typically multiple users
of the pool, the demand can not always be predicated and deadlocks can
arise with a bounded pool.
Individual buffers will be allocated as necessary and released if they become empty. However, since allocation patterns may cause the in use data to be scattered across the allocated buffers, the backing buffers may not be returned to the backing pool until the top-level allocation context is cleared.
Any storage allocated by this instance will be released no later than
when the instance is finalized. Storage may be
returned to the pool within the life cycle of the MemoryManager
using clear(). Nested allocation contexts may be created and
managed using createAllocationContext().
pool - The pool from which the MemoryManager will allocate
its buffers (each "sector" is one buffer).sectors - The maximum #of buffers which the MemoryManager will
allocate from that pool (each "sector" is one buffer). This
may be Integer.MAX_VALUE for an effectively unbounded
capacity.blocks - When true an allocation request
will block until it can be satisfied. When false
and allocation request that can not be satisfied immediately
will result in a MemoryManagerOutOfMemory.properties - Used to communicate various configuration properties,
including
AbstractTransactionService.Options#MIN_RELEASE_AGE
(optional).IllegalArgumentException - if pool is null.IllegalArgumentException - if sectors is non-positive.protected void finalize()
throws Throwable
public int getMaxSectors()
getMaxSectors in interface IMemoryManagerpublic boolean isBlocking()
true iff the default policy of this
MemoryManager instance is to block if an allocation can not be
made (due to exhaustion of the maximum number of backing buffers for the
MemoryManager instance). Return false iff the
allocation will throw a MemoryManagerOutOfMemory exception if the
backing memory pool is exhausted.public int getSectorCount()
getSectorCount in interface IMemoryManagerpublic int getSectorSize()
getSectorSize in interface IMemoryManagerpublic long getMaxMemoryCapacity()
public long allocate(ByteBuffer data)
IMemoryManagerIMemoryManager.allocate(ByteBuffer, boolean) which is either
blocking or non-blocking depending on whether or not the memory
manager is set in a blocking mode.allocate in interface IMemoryManagerdata - The data will be copied to the backing resource. For each
buffer in this array, the position will be advanced to the
limit.public long allocate(ByteBuffer data, boolean blocks)
IMemoryManagerallocate in interface IMemoryManagerdata - The data will be copied to the backing resource. For each
buffer in this array, the position will be advanced to the
limit.blocks - When true the request will block until the memory
is available for the allocation.public long allocate(int nbytes)
IMemoryManagerIMemoryManager.allocate(int).allocate in interface IMemoryManagernbytes - The size of the allocation request.public long allocate(int nbytes,
boolean blocks)
IMemoryManagerallocate in interface IMemoryManagernbytes - The size of the allocation request.blocks - When true the method will block until the
allocation request can be satisfied. When false a
MemoryManagerOutOfMemory will be thrown.public ByteBuffer[] get(long addr)
IMemoryManagerByteBuffers providing an updatable view onto
the backing allocation.
The ByteBuffer[] return enables the handling of blobs that span more than
a single slot, without the need to create an intermediate ByteBuffer.
This method is designed for use with zero-copy NIO. Furthermore, since
the ByteBuffers in the returned array are not read-only, they can
be updated directly. In this way the IMemoryManager.allocate(int) can be used
in conjunction with get to provide more flexibility when storing data.
Using ByteBuffer:put the returned array can be efficiently copied to another ByteBuffer:
ByteBuffer mybb;
ByteBuffer[] bufs = get(addr);
for (ByteBuffer b : bufs) {
mybb.put(b);
}
CAUTION: Do not hold onto the ByteBuffer longer than is
necessary. If the allocation is released by IMemoryManager.free(long)
or IMemoryManager.clear(), then the memory backing the ByteBuffer could
be reallocated by another DirectBufferPool consumer.get in interface IMemoryManageraddr - An previously allocated address.public byte[] read(long addr)
IMemoryManagerIMemoryManager is treated as a
persistence store.read in interface IMemoryManageraddr - The address.public void free(long addr)
IMemoryManagerfree in interface IMemoryManageraddr - to be freedpublic long getPhysicalAddress(long addr)
IMemoryManagergetPhysicalAddress in interface IMemoryManageraddr - The encoded addresspublic void clear()
IMemoryManagerIMemoryManager will release
any direct ByteBuffers back to the pool from which they were
allocated.
CAUTION: Do not clear an allocation context until you know that all threads with access to that allocation context have either been terminated or released their reference to that allocation context.
clear in interface IMemoryManagerpublic void addToFreeList(SectorAllocator sector)
ISectorManageraddToFreeList in interface ISectorManagersector - to be addedpublic void removeFromFreeList(SectorAllocator sector)
ISectorManagerremoveFromFreeList in interface ISectorManagersector - to be removedpublic void trimSector(long trim,
SectorAllocator sector)
ISectorManagertrimSector in interface ISectorManagertrim - - the amount by which the sector allocation can be reducedpublic IMemoryManager createAllocationContext()
createAllocationContext in interface IMemoryManagerpublic int allocationSize(long addr)
IMemoryManagerallocationSize in interface IMemoryManageraddr - The address.public long getCapacity()
public long getExtent()
MemoryManager.public long getAllocationCount()
IMemoryManagergetAllocationCount in interface IMemoryManagerpublic long getSlotBytes()
IMemoryManagerIMemoryManager (including any child allocation contexts).getSlotBytes in interface IMemoryManagerpublic long getUserBytes()
IMemoryManagerIMemoryManager (including any child allocation contexts). Due to
the overhead of the storage allocation scheme, this value may be smaller
than IMemoryManager.getSlotBytes().getUserBytes in interface IMemoryManagerpublic CounterSet getCounters()
ICounterSetAccessgetCounters in interface ICounterSetAccesspublic long alloc(byte[] buf,
int size,
IAllocationContext context)
IStorepublic void close()
IStorepublic void free(long addr,
int size)
IStorepublic int getAssociatedSlotSize(int addr)
getAssociatedSlotSize in interface IStoreaddr - - the addresspublic void getData(long l,
byte[] buf)
IStorepublic File getStoreFile()
IStoregetStoreFile in interface IStorepublic IPSOutputStream getOutputStream()
IStreamStoreIPSOutputStream.getOutputStream in interface IStreamStorepublic IPSOutputStream getOutputStream(IAllocationContext context)
IAllocationManagerStoreIPSOutputStream.getOutputStream in interface IAllocationManagerStorecontext - The context within which any allocations are made by the
returned IPSOutputStream.public InputStream getInputStream(long addr)
IStreamStoregetInputStream in interface IStreamStoreaddr - The address at which the stream was written.public void commit()
IStoreIStore.free(long, int) is now available for recycling.
However, recycling can not occur if session protection is active.public Lock getCommitLock()
IStoreLock that must be used (when non-
null) to make the IStore.commit() / IStore.postCommit()
strategy atomic.getCommitLock in interface IStorepublic void postCommit()
IStorepostCommit in interface IStorepublic void registerExternalCache(ConcurrentWeakValueCache<Long,ICommitter> externalCache, int dataSize)
IHistoryManagerNote: It is not safe to clear at the point of the delete request since the data could still be loaded if the data is retained for a period due to a non-zero retention period or session protection.
registerExternalCache in interface IHistoryManagerpublic long saveDeferrals()
IHistoryManagerWrites the content of currentTxnFreeList to the store.
These are the current buffered frees that have yet been saved into a block referenced from the deferredFreeList
saveDeferrals in interface IHistoryManagerDeleteBlockCommitterpublic void deferFree(int rwaddr,
int sze)
public int checkDeferredFrees(AbstractJournal journal)
IHistoryManagercheckDeferredFrees in interface IHistoryManagerAbstractJournal#commitNow()public IRawTx newTx()
IHistoryManagerIStore. As long as
a transaction is active we can not release data which is currently marked
as freed but was committed at the point the session started.newTx in interface IHistoryManagerpublic long getLastReleaseTime()
IHistoryManagergetLastReleaseTime in interface IHistoryManagerpublic void abortContext(IAllocationContext context)
IAllocationManagerabortContext in interface IAllocationManagercontext - The application object which serves as the allocation context.public void detachContext(IAllocationContext context)
IAllocationManagerIStore is the top-level parent of
allocation contexts. The allocators associated with the allocation
context are return to the global list of available allocators.detachContext in interface IAllocationManagercontext - The application object which serves as the allocation context.public boolean isCommitted(long addr)
IMemoryManagertrue iff the allocation having that address is
flagged as committed. The caller must be holding the allocation lock in
order for the result to remain valid outside of the method call.isCommitted in interface IMemoryManageraddr - The address.true iff the address is currently committed.public long allocate(ByteBuffer data, IAllocationContext context)
allocate in interface IMemoryManagerpublic long write(ByteBuffer data, IAllocationContext context)
IAllocationManagerStorewrite in interface IAllocationManagerStoredata - The data.context - The allocation context.public void free(long addr,
IAllocationContext context)
free in interface IMemoryManagerpublic void delete(long addr,
IAllocationContext context)
IAllocationManagerStoredelete in interface IAllocationManagerStoreaddr - The address whose allocation is to be deleted.context - The allocation context.public IAllocationContext newAllocationContext(boolean isolated)
IAllocationManagernewAllocationContext in interface IAllocationManagerCopyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.