com.bigdata.rwstore

Class StorageStats

java.lang.Object

com.bigdata.rwstore.StorageStats

public class StorageStats
extends Object

Maintains stats on the RWStore allocations, useful for tuning Allocator sizes and tracking store efficiency. It can also track reallocation patterns that are lost in static snapshots of current usage. Stats are retained on external requests and also on internal allocator use. totalSlots represents the total reserved slots in all the AllocationBlocks for the associated FixedAllocators. slotAllocations is the total of all allocations made slotDeletes is the total of all slot deletions Therefore the total of currently allocated slots is slotAllocations - slotDeletes sizeAllocations is the total in bytes of the actual data stored in the slots sizeDeletes is the total in bytes of the actual data that has been deleted Therefore the size of the total in bytes stored in currently allocated slots is sizeAllocations - sizeDeletes

Author:

Martyn Cutcher

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	StorageStats.BlobBucket
class	StorageStats.Bucket

Constructor Summary

Constructors

Constructor and Description
StorageStats(DataInputStream instr)
StorageStats(int[] buckets)

Method Summary

Methods

Modifier and Type	Method and Description
void	allocateBlob(int sze)
void	commit()
void	deleteBlob(int sze)
StorageStats.Bucket	findBucket(int sze)
Iterator<StorageStats.Bucket>	getBuckets()
byte[]	getData()
void	register(FixedAllocator alloc)
void	register(FixedAllocator alloc, boolean init)
void	register(SectorAllocator allocator, boolean init)
void	reset()
void	showStats(StringBuilder str) Collected statistics are against each Allocation Block size: AllocatorSizeThe #of bytes in the allocated slots issued by this allocator. AllocatorCountThe #of fixed allocators for that slot size. SlotsAllocatedCumulative allocation of slots to date in this slot size (regardless of the transaction outcome). %SlotsAllocatedSlotsAllocated/(Sum of SlotsAllocated across all slot sizes). SlotsRecycledCumulative recycled slots to date in this slot size (regardless of the transaction outcome). SlotsChurnHow frequently slots of this size are re-allocated (SlotsAllocated/SlotsReserved). SlotsInUseSlotsAllocated - SlotsRecycled (net slots in use for this slot size). %SlotsInUseSlotsInUse / (total SlotsInUse across all slots sizes). MeanAllocation((Total application bytes used across all allocations for this slot size) / SlotsAllocated). SlotsReservedThe #of slots in this slot size which have had storage reserved for them. %SlotsUnusedThe percentage of slots of this size which are not in use (1-(SlotsInUse/SlotsReserved)). BytesReservedThe space reserved on the backing file for those allocation slots (AllocatorSlots * SlotsReserved). UsedStoreThe #of bytes in the allocated slots. %SlotWasteHow well the application data fits in the slots (BytesAppData/(SlotsInUse*AllocatorSize)). %AppDataHow much of your data is stored by each allocator (BytesAppData/Sum(BytesAppData)). %StoreFileHow much of the backing file is reserved for each allocator (BytesReserved/Sum(BytesReserved)). TODO TotalWaste %StoreWasteHow much of the total waste on the store is waste for this allocator size ((BytesReserved-BytesAppData)/(Sum(BytesReserved)-Sum(BytesAppData))).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StorageStats

public StorageStats(int[] buckets)

Parameters:

buckets - - the slot sizes used by the FixedAllocators

StorageStats

public StorageStats(DataInputStream instr)
             throws IOException

Parameters:

instr - restore from reopen

Throws:

IOException

Method Detail

getData

public byte[] getData()
               throws IOException

Throws:

IOException

allocateBlob

public void allocateBlob(int sze)

deleteBlob

public void deleteBlob(int sze)

findBucket

public StorageStats.Bucket findBucket(int sze)

register

public void register(FixedAllocator alloc,
            boolean init)

register

public void register(FixedAllocator alloc)

showStats

public void showStats(StringBuilder str)

Collected statistics are against each Allocation Block size:

AllocatorSize

The #of bytes in the allocated slots issued by this allocator.

AllocatorCount

The #of fixed allocators for that slot size.

SlotsAllocated

Cumulative allocation of slots to date in this slot size (regardless of the transaction outcome).

%SlotsAllocated

SlotsAllocated/(Sum of SlotsAllocated across all slot sizes).

SlotsRecycled

Cumulative recycled slots to date in this slot size (regardless of the transaction outcome).

SlotsChurn

How frequently slots of this size are re-allocated (SlotsAllocated/SlotsReserved).

SlotsInUse

SlotsAllocated - SlotsRecycled (net slots in use for this slot size).

%SlotsInUse

SlotsInUse / (total SlotsInUse across all slots sizes).

MeanAllocation

((Total application bytes used across all allocations for this slot size) / SlotsAllocated).

SlotsReserved

The #of slots in this slot size which have had storage reserved for them.

%SlotsUnused

The percentage of slots of this size which are not in use (1-(SlotsInUse/SlotsReserved)).

BytesReserved

The space reserved on the backing file for those allocation slots (AllocatorSlots * SlotsReserved).

UsedStore

The #of bytes in the allocated slots.

%SlotWaste

How well the application data fits in the slots (BytesAppData/(SlotsInUse*AllocatorSize)).

%AppData

How much of your data is stored by each allocator (BytesAppData/Sum(BytesAppData)).

%StoreFile

How much of the backing file is reserved for each allocator (BytesReserved/Sum(BytesReserved)).

TODO TotalWaste

%StoreWaste

How much of the total waste on the store is waste for this allocator size ((BytesReserved-BytesAppData)/(Sum(BytesReserved)-Sum(BytesAppData))).

Parameters:

str - The allocator statistics will be appended to the caller's buffer.

See Also:

BytesAppData counter can not be tracked accurately and should be removed

register

public void register(SectorAllocator allocator,
            boolean init)

commit

public void commit()

reset

public void reset()

getBuckets

public Iterator<StorageStats.Bucket> getBuckets()