com.bigdata.journal

Class BenchmarkJournalWriteRate

java.lang.Object

junit.framework.Assert

junit.framework.TestCase

junit.framework.TestCase2

com.bigdata.journal.BenchmarkJournalWriteRate

All Implemented Interfaces:

junit.framework.Test

Direct Known Subclasses:

BenchmarkJournalWriteRate.BenchmarkDirectJournal, BenchmarkJournalWriteRate.BenchmarkDiskJournal, BenchmarkJournalWriteRate.BenchmarkDiskRWJournal, BenchmarkJournalWriteRate.BenchmarkMappedJournal, BenchmarkJournalWriteRate.BenchmarkTransientJournal

public abstract class BenchmarkJournalWriteRate
extends TestCase2

A benchmark that computes the raw platform write rate for pure sequential IO, the journal's write rate, and both unisolated and isolated index write rates. The overhead of the journal can be estimated by comparing the pure sequential writes on the file system with the write rates when using each of the journal modes. Likewise, the overhead of the index can be estimated in comparison to the journal write rate, and the overehead of (large) transactions can be estimated in comparison to the unisolated index write rate (this test does not estimate the overhead for small transactions for a variety of reasons).

The results from this benchmark can be used to compare the performance of the different IBufferMode implementations. The theoretical maximum for a platform is the sustained write rate for the disk subsystem on which the journal file is located - this can be obtained with a bit of research on your disk drives, e.g., using storagereview.com . It is generally achieved by BenchmarkJournalWriteRate.BenchmarkBlockBasedOptimium

Note: you should run these tests multiple times to make sure that you are getting valid numbers for your platform. You should also compare the data with the expected disk maximum write rate for your platform. You can monitor your platform using "perfmon" on Windows or "vmstat" on Un*x. It is important that your system has not swapped out parts of the JVM or the benchmark will run poorly (this can be a problem with a memory-limited Windows platform).

Analysis: The Transient mode achieves 30x the raw write rate when compared to any of the disk-backed modes (1,282 MB/sec vs 40 MB/sec). However, the index write rates are essentially constant across the buffer modes (roughly 20MB/sec for unisolated writes, which is ~50% of the journal write rate when backed by disk, and 9MB/sec for isolated writes, or ~25% of the journal write rate when backed by disk). The limiting factor for index writes is the btree code itself (it tends to be key search). The limiting factor for the isolated index writes is that the transaction write set overflows onto disk, so we wind up doing much more IO for a large transaction (however small transactions have very little overhead when compared to unisolated index writes). The disk-only buffer mode does a little better than the fully-buffered modes for the isolated writes - presumably since (a) the monotonically increasing keys defeat the index node and leaf cache; and (b) the disk-only mode is able to make more RAM available to the JVM since it does not maintain the large in memory buffer.

Version:

$Id$

Author:

Bryan Thompson

See Also:

src/architecture/performance.xls.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BenchmarkJournalWriteRate.AbstractBenchmarkOptimium Does N writes of M size data blocks on a pre-extended file using pure sequential IO.
static class	BenchmarkJournalWriteRate.BenchmarkBlockBasedOptimium Writes the same amount of data using large blocks on a pre-extended file using pure sequential IO.
static class	BenchmarkJournalWriteRate.BenchmarkDirectJournal
static class	BenchmarkJournalWriteRate.BenchmarkDiskJournal
static class	BenchmarkJournalWriteRate.BenchmarkDiskRWJournal
static class	BenchmarkJournalWriteRate.BenchmarkMappedJournal
static class	BenchmarkJournalWriteRate.BenchmarkSmallRecordOptimium Writes the same amount of data, using 128 byte records on a pre-extended file using pure sequential IO.
static class	BenchmarkJournalWriteRate.BenchmarkSustainedTransferOptimium Writes the same amount of data using a single nio "write buffer" operation on a pre-extended file.
static class	BenchmarkJournalWriteRate.BenchmarkTransientJournal

Nested classes/interfaces inherited from class junit.framework.TestCase2

TestCase2.MyProperties, TestCase2.RandomType

Field Summary

Fields inherited from class junit.framework.TestCase2

_randomType, log

Constructor Summary

Constructors

Constructor and Description
BenchmarkJournalWriteRate()
BenchmarkJournalWriteRate(String name)

Method Summary

Methods

Modifier and Type	Method and Description
long	doIndexWriteRateTest(String name, long tx, int valueSize) Writes N records of the given size such that the journal will be filled to "near" capacity using either an isolated or unisolated BTree to absorb the writes.
long	doRawRecordWriteRateTest(int writeSize) Writes N records of the given size such that the journal will be filled to capacity using the IRawStore interface (unisolated raw writes not using an index).
protected int	getBranchingFactor() The branching factor used by the unisolated btree on the journal and by the isolated btree iff a transaction is used to isolated the write set.
protected abstract BufferMode	getBufferMode()
protected String	getFilename()
protected long	getInitialExtent() The tests are performed with a 100M journal.
Properties	getProperties() Sets the initial extent for the test.
protected int	getRecordSize() The tests are performed with a record size of 4k, but you can vary the "record size" to be larger or smaller.
static void	main(String[] args) Main routine can be used for running the test under a performance analyzer.
void	setUp()
static junit.framework.Test	suite() Runs the tests that have not been commented out :-)
void	tearDown()
void	testIsolatedIndexWriteRate() Test the index write rate for a fully isolated transaction using 32 bit integer keys and 128 byte values for the index entries.
void	testNonIsolatableIndexWriteRate() Test the index write rate using an index that does NOT support transactional isolation using 32 bit integer keys and 128 byte values for the index entries.
void	testRawRecordWriteRate()
void	testUnisolatedIndexWriteRate() Test the index write rate using an index that supports transactional isolation but without transactional isolation using 32 bit integer keys and 128 byte values for the index entries.

Methods inherited from class junit.framework.TestCase2

assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEqualsWithinUlps, assertSameArray, assertSameArray, assertSameBigDecimal, assertSameBigDecimal, assertSameBigInteger, assertSameBigInteger, assertSameIterator, assertSameIterator, assertSameIteratorAnyOrder, assertSameIteratorAnyOrder, assertSameValue, assertSameValue, assertZeroUlps, assertZeroUlps, fail, getInnerCause, getNormalInt, getProjectBuildPath, getRandomObject, getRandomObject, getRandomOrder, getRandomString, getTestInputStream, getTestResource, getTestResource, getUlps, getUlps, isDEBUG, isDEBUG, isINFO, isINFO, isInnerCause, logProperties

Methods inherited from class junit.framework.TestCase

assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertFalse, assertFalse, assertNotNull, assertNotNull, assertNotSame, assertNotSame, assertNull, assertNull, assertSame, assertSame, assertTrue, assertTrue, countTestCases, createResult, fail, fail, failNotEquals, failNotSame, failSame, format, getName, run, run, runBare, runTest, setName, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

BenchmarkJournalWriteRate

public BenchmarkJournalWriteRate()

BenchmarkJournalWriteRate

public BenchmarkJournalWriteRate(String name)

Parameters:

name -

Method Detail

getProperties

public Properties getProperties()

Sets the initial extent for the test.

Overrides:

getProperties in class TestCase2

Returns:

A Properties object that supplies bindings for property names according to the described hierarchy among property resources. The returned Properties is NOT cached. TODO This does not handle the recursive truncation of the class name to search for "test.properties" yet.

getRecordSize

protected int getRecordSize()

The tests are performed with a record size of 4k, but you can vary the "record size" to be larger or smaller. 4k was choosen as being representative of the expected size of a node or leaf of a btree since that is the primary kind of object that we write on the journal.

getInitialExtent

protected long getInitialExtent()

The tests are performed with a 100M journal.

getBufferMode

protected abstract BufferMode getBufferMode()

getFilename

protected String getFilename()

getBranchingFactor

protected int getBranchingFactor()

The branching factor used by the unisolated btree on the journal and by the isolated btree iff a transaction is used to isolated the write set.

Note: A higher branching factor can be choosen for this test since the btree writes use monotoically increasing keys.

setUp

public void setUp()
           throws IOException

Overrides:

setUp in class junit.framework.TestCase

Throws:

IOException

tearDown

public void tearDown()
              throws IOException

Overrides:

tearDown in class junit.framework.TestCase

Throws:

IOException

testRawRecordWriteRate

public void testRawRecordWriteRate()
                            throws IOException

Throws:

IOException

testNonIsolatableIndexWriteRate

public void testNonIsolatableIndexWriteRate()
                                     throws IOException

Test the index write rate using an index that does NOT support transactional isolation using 32 bit integer keys and 128 byte values for the index entries.

Throws:

IOException

testUnisolatedIndexWriteRate

public void testUnisolatedIndexWriteRate()
                                  throws IOException

Test the index write rate using an index that supports transactional isolation but without transactional isolation using 32 bit integer keys and 128 byte values for the index entries.

Throws:

IOException

testIsolatedIndexWriteRate

public void testIsolatedIndexWriteRate()
                                throws IOException

Test the index write rate for a fully isolated transaction using 32 bit integer keys and 128 byte values for the index entries.

Throws:

IOException

doRawRecordWriteRateTest

public long doRawRecordWriteRateTest(int writeSize)

Writes N records of the given size such that the journal will be filled to capacity using the IRawStore interface (unisolated raw writes not using an index).

Parameters:

writeSize - The size of the object to be written.

Returns:

The elapsed time for the test.

doIndexWriteRateTest

public long doIndexWriteRateTest(String name,
                        long tx,
                        int valueSize)

Writes N records of the given size such that the journal will be filled to "near" capacity using either an isolated or unisolated BTree to absorb the writes. The records are written in key order, so this is the best cast for sequential key writes. The test ends before the journal would overflow in order to measure only the cost of writes without buffer extension handling.

Note that for transactional writes, the writes are buffered in memory and then on disk, validated against the buffered writes, and finally transferred to the unisolated index on the journal. Short transactions are therefore very fast, but large transactions will be significantly slower than the corresponding unisolated writes since there is several times more IO for large transactions (write on tx buffer, read tx buffer and validate against the unisolated index, read tx buffer and write on the unisolated index). However, there is also logic to defeat validation when no concurrent writes have occurred, so the worst case will not be demonstrated by a single write process.

Parameters:

name - The name of the index on which the writes will be performed. The named index MUST have been registered by the caller and that registration MUST have been committed.

tx - The transaction identifier -or- 0L if the writes will not be isolated by a transaction.

valueSize - The size in bytes of the value to be written under each key.

Returns:

The elapsed time for the test.

suite

public static junit.framework.Test suite()

Runs the tests that have not been commented out :-)

Note: Running all benchmarks together can challange the VM by running low on heap, native memory given over to direct buffers - and things can actually slow down with more memory.

main

public static void main(String[] args)
                 throws Exception

Main routine can be used for running the test under a performance analyzer.

Parameters:

args - Not used.

Throws:

Exception