public class MemoryManager extends Object implements IMemoryManager, ISectorManager
ByteBuffer
. It uses the
new SectorAllocator to allocate slots within the address range.
The interface is designed to support efficient transfer between NIO buffers.
The most complex aspect of the implementation is the BLOB representation,
requiring a mapping across multiple allocation slots. This is managed using
recursive calls in the main three methods: allocate, free and get.Constructor and Description |
---|
MemoryManager(DirectBufferPool pool)
Create a new
MemoryManager . |
MemoryManager(DirectBufferPool pool,
int sectors)
Create a new
MemoryManager . |
MemoryManager(DirectBufferPool pool,
int sectors,
boolean blocks,
Properties properties)
Create a new
MemoryManager . |
Modifier and Type | Method and Description |
---|---|
void |
abortContext(IAllocationContext context)
Indicates that the allocation context will no longer be used and that the
allocations made within the context should be discarded.
|
void |
addToFreeList(SectorAllocator sector)
When sufficient allocations have been freed for recycling that a
threshold of availability of reached for all block sizes, then the
allocator calls back to the SectorManager to signal it is available to be
returned to the free list.
|
long |
alloc(byte[] buf,
int size,
IAllocationContext context)
Writes data on the store.
|
long |
allocate(ByteBuffer data)
Version of
IMemoryManager.allocate(ByteBuffer, boolean) which is either
blocking or non-blocking depending on whether or not the memory
manager is set in a blocking mode. |
long |
allocate(ByteBuffer data,
boolean blocks)
Allocates space on the backing resource and copies the provided data.
|
long |
allocate(ByteBuffer data,
IAllocationContext context) |
long |
allocate(int nbytes)
Return the address of a new allocation sufficient to store the specified
number of bytes of application data.
|
long |
allocate(int nbytes,
boolean blocks)
Return the address of a new allocation sufficient to store the specified
number of bytes of application data.
|
int |
allocationSize(long addr)
Return the size of the application data for the allocation with the given
address.
|
int |
checkDeferredFrees(AbstractJournal journal)
This method is invoked during the commit protocol and gives the backing
store an opportunity to check whether storage associated with deferred
frees can now be released.
|
void |
clear()
Clears all current allocations.
|
void |
close()
Close the backing storage.
|
void |
commit()
Global commit on the backing store.
|
IMemoryManager |
createAllocationContext()
Maintain allocationContext to check for session protection
|
void |
deferFree(int rwaddr,
int sze) |
void |
delete(long addr,
IAllocationContext context)
Delete the data associated with the address within the allocation
context.
|
void |
detachContext(IAllocationContext context)
Indicates that the allocation context will no longer be used, but that
the allocations made within the context should be preserved.
|
protected void |
finalize() |
void |
free(long addr)
Frees the address and makes available for recycling
|
void |
free(long addr,
IAllocationContext context) |
void |
free(long addr,
int size)
Frees allocated storage (clears the bit to enable recycling after
the next commit).
|
ByteBuffer[] |
get(long addr)
Return an array of
ByteBuffer s providing an updatable view onto
the backing allocation. |
long |
getAllocationCount()
The #of allocation spanned by this allocation context (including any
any child allocation contexts).
|
int |
getAssociatedSlotSize(int addr) |
long |
getCapacity()
The maximum #of bytes which are available to the memory manager.
|
Lock |
getCommitLock()
Optionally return a
Lock that must be used (when non-
null ) to make the IStore.commit() / IStore.postCommit()
strategy atomic. |
CounterSet |
getCounters()
Return performance counters.
|
void |
getData(long l,
byte[] buf)
Read data of a known size from the store.
|
long |
getExtent()
The total #of bytes in the backing buffers currently attached to the
MemoryManager . |
InputStream |
getInputStream(long addr)
Return an input stream from which a previously written stream may be read
back.
|
long |
getLastReleaseTime()
If history is retained this returns the time for which data was most
recently released.
|
long |
getMaxMemoryCapacity()
The maximum capacity in bytes of the managed memory.
|
int |
getMaxSectors()
Return the maximum #of sectors which may be allocated.
|
IPSOutputStream |
getOutputStream()
Return an output stream which can be used to write on the backing store.
|
IPSOutputStream |
getOutputStream(IAllocationContext context)
Return an output stream which can be used to write on the backing store
within the given allocation context.
|
long |
getPhysicalAddress(long addr)
Determine the unencoded physical address
|
int |
getSectorCount()
Return the #of sectors which are currently in use.
|
int |
getSectorSize()
The size in bytes of the backing sector.
|
long |
getSlotBytes()
Return the #of bytes of consumed by allocation slots allocated against
this
IMemoryManager (including any child allocation contexts). |
File |
getStoreFile()
Retrieves store file.
|
long |
getUserBytes()
Return the #of bytes of application data allocated against this
IMemoryManager (including any child allocation contexts). |
boolean |
isBlocking()
Return
true iff the default policy of this
MemoryManager instance is to block if an allocation can not be
made (due to exhaustion of the maximum number of backing buffers for the
MemoryManager instance). |
boolean |
isCommitted(long addr)
Return
true iff the allocation having that address is
flagged as committed. |
IAllocationContext |
newAllocationContext(boolean isolated)
Creates a context to be used to isolate updates to within the context until it
is released to the parent environment.
|
IRawTx |
newTx()
A hook used to support session protection by incrementing and
decrementing a transaction counter within the
IStore . |
void |
postCommit()
Hook that supports synchronization with an external commit before which
a rollback to "pre-commit" state is supported.
|
byte[] |
read(long addr)
Return a copy of the data stored at that address.
|
void |
registerExternalCache(ConcurrentWeakValueCache<Long,ICommitter> externalCache,
int dataSize)
Call made from AbstractJournal to register the cache used.
|
void |
removeFromFreeList(SectorAllocator sector)
This request is made when the sectorAllocator no longer has a full set of
block allocations available.
|
long |
saveDeferrals()
Saves the current list of delete blocks, returning the address allocated.
|
String |
toString() |
void |
trimSector(long trim,
SectorAllocator sector)
When a sector is first created, it will remain at the head of the free
list until one of two conditions has been reached:
The allocation has been saturated.
The bit space has been filled.
|
long |
write(ByteBuffer data,
IAllocationContext context)
Write the data within the allocation context.
|
public MemoryManager(DirectBufferPool pool)
MemoryManager
.
The backing DirectBufferPool
may be either bounded or
(effectively) unbounded. The MemoryManager
will be (effectively)
unbounded. If either the pool is bounded, then blocking
allocation requests may block, otherwise allocation requests will be
non-blocking.
The garbage collection of direct ByteBuffer
s depends on a full GC
pass. In an application which managers its heap pressure well, full GC
passes are rare. Therefore, the best practice is to share an unbounded
pool across multiple purposes. Since there are typically multiple users
of the pool, the demand can not always be predicated and deadlocks can
arise with a bounded pool.
Individual buffers will be allocated as necessary and released if they become empty. However, since allocation patterns may cause the in use data to be scattered across the allocated buffers, the backing buffers may not be returned to the backing pool until the top-level allocation context is cleared.
Any storage allocated by this instance will be released no later than
when the instance is finalized
. Storage may be
returned to the pool within the life cycle of the MemoryManager
using clear()
. Nested allocation contexts may be created and
managed using createAllocationContext()
.
pool
- The pool from which the MemoryManager
will allocate
its buffers (each "sector" is one buffer).IllegalArgumentException
- if pool is null
.public MemoryManager(DirectBufferPool pool, int sectors)
MemoryManager
.
The backing DirectBufferPool
may be either bounded or
(effectively) unbounded. The MemoryManager
may also be bounded or
(effectively) unbounded. If either the pool or the memory manager is
bounded, then blocking allocation requests may block. Neither
non-blocking allocation requests nor allocation requests made against an
unbounded memory manager backed by an unbounded pool will block. The
preferred method for bounding the memory manager is to specify a maximum
#of buffers which it may consume from the pool.
The garbage collection of direct ByteBuffer
s depends on a full GC
pass. In an application which managers its heap pressure well, full GC
passes are rare. Therefore, the best practice is to share an unbounded
pool across multiple purposes. Since there are typically multiple users
of the pool, the demand can not always be predicated and deadlocks can
arise with a bounded pool.
Individual buffers will be allocated as necessary and released if they become empty. However, since allocation patterns may cause the in use data to be scattered across the allocated buffers, the backing buffers may not be returned to the backing pool until the top-level allocation context is cleared.
Any storage allocated by this instance will be released no later than
when the instance is finalized
. Storage may be
returned to the pool within the life cycle of the MemoryManager
using clear()
. Nested allocation contexts may be created and
managed using createAllocationContext()
.
pool
- The pool from which the MemoryManager
will allocate
its buffers (each "sector" is one buffer).sectors
- The maximum #of buffers which the MemoryManager
will
allocate from that pool (each "sector" is one buffer). This
may be Integer.MAX_VALUE
for an effectively unbounded
capacity.IllegalArgumentException
- if pool is null
.IllegalArgumentException
- if sectors is non-positive.public MemoryManager(DirectBufferPool pool, int sectors, boolean blocks, Properties properties)
MemoryManager
.
The backing DirectBufferPool
may be either bounded or
(effectively) unbounded. The MemoryManager
may also be bounded or
(effectively) unbounded. If either the pool or the memory manager is
bounded, then blocking allocation requests may block. Neither
non-blocking allocation requests nor allocation requests made against an
unbounded memory manager backed by an unbounded pool will block. The
preferred method for bounding the memory manager is to specify a maximum
#of buffers which it may consume from the pool.
The garbage collection of direct ByteBuffer
s depends on a full GC
pass. In an application which managers its heap pressure well, full GC
passes are rare. Therefore, the best practice is to share an unbounded
pool across multiple purposes. Since there are typically multiple users
of the pool, the demand can not always be predicated and deadlocks can
arise with a bounded pool.
Individual buffers will be allocated as necessary and released if they become empty. However, since allocation patterns may cause the in use data to be scattered across the allocated buffers, the backing buffers may not be returned to the backing pool until the top-level allocation context is cleared.
Any storage allocated by this instance will be released no later than
when the instance is finalized
. Storage may be
returned to the pool within the life cycle of the MemoryManager
using clear()
. Nested allocation contexts may be created and
managed using createAllocationContext()
.
pool
- The pool from which the MemoryManager
will allocate
its buffers (each "sector" is one buffer).sectors
- The maximum #of buffers which the MemoryManager
will
allocate from that pool (each "sector" is one buffer). This
may be Integer.MAX_VALUE
for an effectively unbounded
capacity.blocks
- When true
an allocation request
will block until it can be satisfied. When false
and allocation request that can not be satisfied immediately
will result in a MemoryManagerOutOfMemory
.properties
- Used to communicate various configuration properties,
including
AbstractTransactionService.Options#MIN_RELEASE_AGE
(optional).IllegalArgumentException
- if pool is null
.IllegalArgumentException
- if sectors is non-positive.protected void finalize() throws Throwable
public int getMaxSectors()
getMaxSectors
in interface IMemoryManager
public boolean isBlocking()
true
iff the default policy of this
MemoryManager
instance is to block if an allocation can not be
made (due to exhaustion of the maximum number of backing buffers for the
MemoryManager
instance). Return false
iff the
allocation will throw a MemoryManagerOutOfMemory
exception if the
backing memory pool is exhausted.public int getSectorCount()
getSectorCount
in interface IMemoryManager
public int getSectorSize()
getSectorSize
in interface IMemoryManager
public long getMaxMemoryCapacity()
public long allocate(ByteBuffer data)
IMemoryManager
IMemoryManager.allocate(ByteBuffer, boolean)
which is either
blocking or non-blocking depending on whether or not the memory
manager is set in a blocking mode.allocate
in interface IMemoryManager
data
- The data will be copied to the backing resource. For each
buffer in this array, the position will be advanced to the
limit.public long allocate(ByteBuffer data, boolean blocks)
IMemoryManager
allocate
in interface IMemoryManager
data
- The data will be copied to the backing resource. For each
buffer in this array, the position will be advanced to the
limit.blocks
- When true
the request will block until the memory
is available for the allocation.public long allocate(int nbytes)
IMemoryManager
IMemoryManager.allocate(int)
.allocate
in interface IMemoryManager
nbytes
- The size of the allocation request.public long allocate(int nbytes, boolean blocks)
IMemoryManager
allocate
in interface IMemoryManager
nbytes
- The size of the allocation request.blocks
- When true
the method will block until the
allocation request can be satisfied. When false
a
MemoryManagerOutOfMemory
will be thrown.public ByteBuffer[] get(long addr)
IMemoryManager
ByteBuffer
s providing an updatable view onto
the backing allocation.
The ByteBuffer[] return enables the handling of blobs that span more than
a single slot, without the need to create an intermediate ByteBuffer.
This method is designed for use with zero-copy NIO. Furthermore, since
the ByteBuffer
s in the returned array are not read-only, they can
be updated directly. In this way the IMemoryManager.allocate(int)
can be used
in conjunction with get to provide more flexibility when storing data.
Using ByteBuffer:put the returned array can be efficiently copied to another ByteBuffer:
ByteBuffer mybb; ByteBuffer[] bufs = get(addr); for (ByteBuffer b : bufs) { mybb.put(b); }CAUTION: Do not hold onto the
ByteBuffer
longer than is
necessary. If the allocation is released by IMemoryManager.free(long)
or IMemoryManager.clear()
, then the memory backing the ByteBuffer
could
be reallocated by another DirectBufferPool
consumer.get
in interface IMemoryManager
addr
- An previously allocated address.public byte[] read(long addr)
IMemoryManager
IMemoryManager
is treated as a
persistence store.read
in interface IMemoryManager
addr
- The address.public void free(long addr)
IMemoryManager
free
in interface IMemoryManager
addr
- to be freedpublic long getPhysicalAddress(long addr)
IMemoryManager
getPhysicalAddress
in interface IMemoryManager
addr
- The encoded addresspublic void clear()
IMemoryManager
IMemoryManager
will release
any direct ByteBuffer
s back to the pool from which they were
allocated.
CAUTION: Do not clear an allocation context until you know that all threads with access to that allocation context have either been terminated or released their reference to that allocation context.
clear
in interface IMemoryManager
public void addToFreeList(SectorAllocator sector)
ISectorManager
addToFreeList
in interface ISectorManager
sector
- to be addedpublic void removeFromFreeList(SectorAllocator sector)
ISectorManager
removeFromFreeList
in interface ISectorManager
sector
- to be removedpublic void trimSector(long trim, SectorAllocator sector)
ISectorManager
trimSector
in interface ISectorManager
trim
- - the amount by which the sector allocation can be reducedpublic IMemoryManager createAllocationContext()
createAllocationContext
in interface IMemoryManager
public int allocationSize(long addr)
IMemoryManager
allocationSize
in interface IMemoryManager
addr
- The address.public long getCapacity()
public long getExtent()
MemoryManager
.public long getAllocationCount()
IMemoryManager
getAllocationCount
in interface IMemoryManager
public long getSlotBytes()
IMemoryManager
IMemoryManager
(including any child allocation contexts).getSlotBytes
in interface IMemoryManager
public long getUserBytes()
IMemoryManager
IMemoryManager
(including any child allocation contexts). Due to
the overhead of the storage allocation scheme, this value may be smaller
than IMemoryManager.getSlotBytes()
.getUserBytes
in interface IMemoryManager
public CounterSet getCounters()
ICounterSetAccess
getCounters
in interface ICounterSetAccess
public long alloc(byte[] buf, int size, IAllocationContext context)
IStore
public void close()
IStore
public void free(long addr, int size)
IStore
public int getAssociatedSlotSize(int addr)
getAssociatedSlotSize
in interface IStore
addr
- - the addresspublic void getData(long l, byte[] buf)
IStore
public File getStoreFile()
IStore
getStoreFile
in interface IStore
public IPSOutputStream getOutputStream()
IStreamStore
IPSOutputStream
.getOutputStream
in interface IStreamStore
public IPSOutputStream getOutputStream(IAllocationContext context)
IAllocationManagerStore
IPSOutputStream
.getOutputStream
in interface IAllocationManagerStore
context
- The context within which any allocations are made by the
returned IPSOutputStream
.public InputStream getInputStream(long addr)
IStreamStore
getInputStream
in interface IStreamStore
addr
- The address at which the stream was written.public void commit()
IStore
IStore.free(long, int)
is now available for recycling.
However, recycling can not occur if session protection is active.public Lock getCommitLock()
IStore
Lock
that must be used (when non-
null
) to make the IStore.commit()
/ IStore.postCommit()
strategy atomic.getCommitLock
in interface IStore
public void postCommit()
IStore
postCommit
in interface IStore
public void registerExternalCache(ConcurrentWeakValueCache<Long,ICommitter> externalCache, int dataSize)
IHistoryManager
Note: It is not safe to clear at the point of the delete request since the data could still be loaded if the data is retained for a period due to a non-zero retention period or session protection.
registerExternalCache
in interface IHistoryManager
public long saveDeferrals()
IHistoryManager
Writes the content of currentTxnFreeList to the store.
These are the current buffered frees that have yet been saved into a block referenced from the deferredFreeList
saveDeferrals
in interface IHistoryManager
DeleteBlockCommitter
public void deferFree(int rwaddr, int sze)
public int checkDeferredFrees(AbstractJournal journal)
IHistoryManager
checkDeferredFrees
in interface IHistoryManager
AbstractJournal#commitNow()
public IRawTx newTx()
IHistoryManager
IStore
. As long as
a transaction is active we can not release data which is currently marked
as freed but was committed at the point the session started.newTx
in interface IHistoryManager
public long getLastReleaseTime()
IHistoryManager
getLastReleaseTime
in interface IHistoryManager
public void abortContext(IAllocationContext context)
IAllocationManager
abortContext
in interface IAllocationManager
context
- The application object which serves as the allocation context.public void detachContext(IAllocationContext context)
IAllocationManager
IStore
is the top-level parent of
allocation contexts. The allocators associated with the allocation
context are return to the global list of available allocators.detachContext
in interface IAllocationManager
context
- The application object which serves as the allocation context.public boolean isCommitted(long addr)
IMemoryManager
true
iff the allocation having that address is
flagged as committed. The caller must be holding the allocation lock in
order for the result to remain valid outside of the method call.isCommitted
in interface IMemoryManager
addr
- The address.true
iff the address is currently committed.public long allocate(ByteBuffer data, IAllocationContext context)
allocate
in interface IMemoryManager
public long write(ByteBuffer data, IAllocationContext context)
IAllocationManagerStore
write
in interface IAllocationManagerStore
data
- The data.context
- The allocation context.public void free(long addr, IAllocationContext context)
free
in interface IMemoryManager
public void delete(long addr, IAllocationContext context)
IAllocationManagerStore
delete
in interface IAllocationManagerStore
addr
- The address whose allocation is to be deleted.context
- The allocation context.public IAllocationContext newAllocationContext(boolean isolated)
IAllocationManager
newAllocationContext
in interface IAllocationManager
Copyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.