SectorAllocator (Blazegraph Database Platform 2.1.5 API)

java.lang.Object
- com.bigdata.rwstore.sector.SectorAllocator

All Implemented Interfaces:

Comparable<SectorAllocator>
```
public class SectorAllocator
extends Object
implements Comparable<SectorAllocator>
```
The SectorAllocator is designed as an alternative the the standard RWStore FixedAllocators. The idea of the SectorAllocator is to efficiently contain within a single region as dense a usage as possible. Since a SectorAllocator is able to allocate a full range of slot sizes, it should be able to service several thousand allocations and maximise disk locality on write. Furthermore, it presents an option to be synced with the backing store - similarly to a MappedFile, in which case a single write for the entire sector could be made for update. What we do not want is to run out of bits and to leave significant unused space in the sector. This could happen if we primarily allocated small slots - say on average 512 bytes. In this case, the maximum 1636 entries would map 1636 * 32 * 512 bytes => only just over 26M, so a 64M sector is massively wasteful. The solution is to increment the sector reserve as required for each tab, say by a minimum of 256K while ensuring always less than 64M. Bit waste in the allocator - where the sector memory is allocated with far fewer bits than available is less of an issue, although it does impact on teh total number of allocations available. The absolute maximum size of an RWStore is derived from the maximum sector size * the number of sectors. An 8K sector allows for 32K bits, which, which only requires 15 unsigned bits, leaving a signed17bits for the sector index = 64K of sectors. Implying a maximum addressable store file of 64M * 64K, = 4TB of full sectors. If the average sector only requires 32M, then the total store would be reduced appropriately. The maximum theoretical storage is yielded by MAX_INT * AVG_SLOT_SIZE, so 2GB * 2K (avg) would equate to the optimal maximum addressable allocations and file size. An AVG of > 2K yields fewer allocations and an AVG of < 2K a reduced file size. TODO: add parameterisation of META_SIZE for exploitation by MemoryManager. TODO: cache block starts in m_addresses to simplify/optimise bit2Offset When a new SectorAllocator is at the head of the free list, a store such as the RWSectorStore can use an in-memory buffer to write the data to - sized to the full size of the sector. This can be written in a single write to the WriteCacheService.

Author:

Martyn Cutcher

Field Summary

Fields
Modifier and Type Field and Description

static int[] ALLOC_SIZES

static int BLOB_SIZE

Fields
Modifier and Type	Field and Description
`static int[]`	`ALLOC_SIZES`
`static int`	`BLOB_SIZE`

Constructor Summary

Constructors
Constructor and Description

SectorAllocator(ISectorManager store, IWriteCacheManager writes)

Constructors
Constructor and Description
`SectorAllocator(ISectorManager store, IWriteCacheManager writes)`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`addAddresses(ArrayList<Long> addrs)`
`int`	`addressSize(int offset)`
`int`	`alloc(int size)` Must find tag type that size fits in (or BLOB) and then find block of type into which an allocation can be made.
`int`	`bit2tag(int bit)` Since we know that all allocations are 32 bits each, there is no need to scan through the array.
`void`	`commit()` Called from MemoryManager to commit bits
`int`	`compareTo(SectorAllocator other)`
`boolean`	`free(int bit)`
`static int`	`getBlobBlockCount(int size)`
`static int`	`getBlockForSize(int size)`
`int`	`getIndex()`
`long`	`getPhysicalAddress(int offset)`
`int`	`getPhysicalSize(int offset)`
`static int`	`getSectorIndex(int rwaddr)`
`static int`	`getSectorOffset(int rwaddr)`
`long`	`getStartAddr()`
`String`	`getStats()`
`boolean`	`hasFree()` Checks
`boolean`	`hasFree(int threshold)`
`boolean`	`isCommitted(int offset)`
`boolean`	`isGettable(int offset)`
`static int`	`makeAddr(int index, int bit)`
`void`	`preserveSessionData()`
`void`	`releaseSession(IWriteCacheManager cache)`
`void`	`setIndex(int index)`
`void`	`setSectorAddress(long sectorAddress, int maxsize)` As well as setting the address, this is the point when the allocator can pre-allocate the first set of tags.
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

BLOB_SIZE
```
public static final int BLOB_SIZE
```
See Also:
Constant Field Values

ALLOC_SIZES
```
public static final int[] ALLOC_SIZES
```

Constructor Detail

SectorAllocator

public SectorAllocator(ISectorManager store,
               IWriteCacheManager writes)

Method Detail
- alloc
```
public int alloc(int size)
```
  Must find tag type that size fits in (or BLOB) and then find block of type into which an allocation can be made.
- makeAddr
```
public static int makeAddr(int index,
           int bit)
```
- free
```
public boolean free(int bit)
```
  Parameters:
  bit -
  
  Returns:
- bit2tag
```
public int bit2tag(int bit)
```
  Since we know that all allocations are 32 bits each, there is no need to scan through the array.
  
  Parameters:
  bit -
  
  Returns:
  the tag of the bit
- getPhysicalAddress
```
public long getPhysicalAddress(int offset)
```
- getPhysicalSize
```
public int getPhysicalSize(int offset)
```
- getStartAddr
```
public long getStartAddr()
```
- getStats
```
public String getStats()
```
- hasFree
```
public boolean hasFree(int threshold)
```
  Parameters:
  threshold - the minimum number of bits free per 32 bit block
  
  Returns:
  whether there are sufficient free for all block sizes
- hasFree
```
public boolean hasFree()
```
  Checks
  
  Returns:
  if there is a positive free count for all tags
- preserveSessionData
```
public void preserveSessionData()
```
- addressSize
```
public int addressSize(int offset)
```
- setIndex
```
public void setIndex(int index)
```
- addAddresses
```
public void addAddresses(ArrayList<Long> addrs)
```
- setSectorAddress
```
public void setSectorAddress(long sectorAddress,
                    int maxsize)
```
  As well as setting the address, this is the point when the allocator can pre-allocate the first set of tags.
  
  Parameters:
  sectorAddress - managed by this Allocator
- getSectorIndex
```
public static int getSectorIndex(int rwaddr)
```
- getSectorOffset
```
public static int getSectorOffset(int rwaddr)
```
- getBlobBlockCount
```
public static int getBlobBlockCount(int size)
```
- getBlockForSize
```
public static int getBlockForSize(int size)
```
- compareTo
```
public int compareTo(SectorAllocator other)
```
  Specified by:
  
  compareTo in interface Comparable<SectorAllocator>
- getIndex
```
public int getIndex()
```
- releaseSession
```
public void releaseSession(IWriteCacheManager cache)
```
- toString
```
public String toString()
```
  Overrides:
  
  toString in class Object
- commit
```
public void commit()
```
  Called from MemoryManager to commit bits
- isCommitted
```
public boolean isCommitted(int offset)
```
- isGettable
```
public boolean isGettable(int offset)
```

Class SectorAllocator

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

BLOB_SIZE

ALLOC_SIZES

Constructor Detail

SectorAllocator

Method Detail

alloc

makeAddr

free

bit2tag

getPhysicalAddress

getPhysicalSize

getStartAddr

getStats

hasFree

hasFree

preserveSessionData

addressSize

setIndex

addAddresses

setSectorAddress

getSectorIndex

getSectorOffset

getBlobBlockCount

getBlockForSize

compareTo

getIndex

releaseSession

toString

commit

isCommitted

isGettable