LoadBalancerService (Blazegraph Database Platform 2.1.5 API)

java.lang.Object
- com.bigdata.service.AbstractService
- - com.bigdata.service.LoadBalancerService

All Implemented Interfaces:

IEventReceivingService, IEventReportingService, ILoadBalancerService, IService, IServiceShutdown, Remote

Direct Known Subclasses:

AbstractEmbeddedLoadBalancerService
```
public abstract class LoadBalancerService
extends AbstractService
implements ILoadBalancerService, IServiceShutdown, IEventReportingService
```
The LoadBalancerService collects a variety of performance counters from hosts and services, identifies over- and under- utilized hosts and services based on the collected data and reports those to DataService s so that they can auto-balance, and acts as a clearing house for WARN and URGENT alerts for hosts and services.
While the LoadBalancerService MAY observe service start/stop events, it does NOT get directly informed of actions that change the load distribution, such as index partition moves or reading from a failover service. Instead, DataServices determine whether or not they are overloaded and, if so, query the LoadBalancerService for the identity of under-utilized services. If under-utilized DataServices are reported by the LoadBalancerService then the DataService will self-identify index partitions to be shed and move them onto the identified under-utilized DataServices. The LoadBalancerService learns of these actions solely through their effect on host and service load as self- reported by various services.
Note: utilization should be defined in terms of transient system resources : CPU, IO (DISK and NET), RAM. DISK exhaustion on the other hand is the basis for WARN or URGENT alerts since it can lead to immediate failure of all services on the same host.
Note: When new services are made available, either on new hosts or on the existing hardware, and service utilization discrepancies should become rapidly apparent (within a few minutes). Once we have collected performance counters for the new hosts / services, a subsequent overflow event(s) on existing DataService(s) will cause index partition moves to be nominated targeting the new hosts and services. The amount of time that it takes to re-balance the load on the services will depend in part on the write rate, since writes drive overflow events and index partition splits, both of which lead to pre-conditions for index partition moves.
Note: If a host is suffering high IOWAIT then it is probably "hot for read" (writes are heavily buffered and purely sequential and therefore unlikely to cause high IOWAIT where as reads are typically random on journals even through a key range scan is sequential on index segments). Therefore a "hot for read" condition should be addressed by increasing the replication count for those service(s) which are being swamped by read requests on a host suffering from high IOWAIT.

See Also:
IRequiredHostCounters, A core set of variables to support decision-making., http://www.google.com/search?hl=en&q=load+balancing+jini

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static interface`	`LoadBalancerService.Options` Options understood by the `LoadBalancerService`.
`protected class`	`LoadBalancerService.RoundRobinServiceLoadHelper` Integration with the `LoadBalancerService`.
`protected class`	`LoadBalancerService.ServiceLoadHelperWithoutScores` Integration with the `LoadBalancerService`.
`protected class`	`LoadBalancerService.ServiceLoadHelperWithScores` Integration with the `LoadBalancerService`.
`protected class`	`LoadBalancerService.UpdateTask` Computes and updates the `ServiceScore`s based on an examination of aggregated performance counters.

Field Summary

Fields
Modifier and Type	Field and Description
`protected ConcurrentHashMap<UUID,ServiceScore>`	`activeDataServices` The set of active services.
`protected ConcurrentHashMap<String,HostScore>`	`activeHosts` The active hosts (one or more services).
`protected EventReceiver`	`eventReceiver`
`protected Journal`	`eventStore` Used to persist the logged events.
`protected int`	`historyMinutes` The #of minutes of history that will be smoothed into an average when `LoadBalancerService.UpdateTask` updates the `HostScore`s and the `ServiceScore`s.
`protected AtomicReference<HostScore[]>`	`hostScores` Scores for the hosts in ascending order (least utilized to most utilized).
`protected long`	`initialRoundRobinUpdateCount` The #of updates during which `getUnderUtilizedDataServices(int, int, UUID)` will apply a round robin policy.
`protected boolean`	`isTransient` `true` iff the LBS will refrain from writing state on the disk.
`protected Condition`	`joined` Used to await a service join when there are no services.
`protected ReentrantLock`	`lock` Lock is used to control access to data structures that are not thread-safe.
`protected static org.apache.log4j.Logger`	`log`
`protected File`	`logDir` The directory in which the service will log the `CounterSet`s and `Event`s.
`protected long`	`nupdates` The #of `LoadBalancerService.UpdateTask`s which have run so far.
`protected String`	`ps`
`protected long`	`serviceJoinTimeout` Service join timeout in milliseconds - used when we need to wait for a service to join before we can recommend an under-utilized service.
`protected AtomicReference<ServiceScore[]>`	`serviceScores` Scores for the services in ascending order (least utilized to most utilized).
`protected ScheduledExecutorService`	`updateService` Runs a periodic `LoadBalancerService.UpdateTask`.

Constructor Summary

Constructors
Constructor and Description
`LoadBalancerService(Properties properties)` Note: The load balancer MUST NOT collect host statistics unless it is the only service running on that host.

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`destroy()` Destroy the service.
`protected void`	`finalized()`
`protected abstract String`	`getClientHostname()` Return the canonical hostname of the client in the context of a RMI request.
`Properties`	`getProperties()` An object wrapping the properties provided to the constructor.
`Class`	`getServiceIface()` Returns `ILoadBalancerService`.
`UUID`	`getUnderUtilizedDataService()` Return the `UUID` of an under-utilized data service.
`UUID[]`	`getUnderUtilizedDataServices(int minCount, int maxCount, UUID exclude)` Return up to limit `IDataService` `UUID`s that are currently under-utilized.
`protected boolean`	`isHighlyUtilizedDataService(ServiceScore score, ServiceScore[] scores)`
`boolean`	`isHighlyUtilizedDataService(UUID serviceUUID)` Return `true` if the service is considered to be "highly utilized".
`boolean`	`isOpen()` Return `true` iff the service is running.
`protected boolean`	`isUnderUtilizedDataService(ServiceScore score, ServiceScore[] scores)`
`boolean`	`isUnderUtilizedDataService(UUID serviceUUID)` Return `true` if the service is considered to be "under-utilized".
`void`	`join(UUID serviceUUID, Class serviceIface, String hostname)` Notify the `LoadBalancerService` that a new service is available.
`void`	`leave(UUID serviceUUID)` Notify the `LoadBalancerService` that a service is no longer available.
`void`	`logCounters()` Logs the counters on a file created using `File.createTempFile(String, String, File)` in the log directory.
`protected void`	`logCounters(File file)` Writes the counters on a file.
`protected void`	`logCounters(String basename)` Writes the counters on a file.
`void`	`notify(UUID serviceUUID, byte[] data)` Send performance counters.
`void`	`notifyEvent(Event e)` Accepts the event, either updates the existing event with the same `UUID` or adds the event to the set of recent events, and then prunes the set of recent events so that all completed events older than `#eventHistoryMillis` are discarded.
`long`	`rangeCount(long fromTime, long toTime)` Reports the #of completed events that start in the given interval.
`Iterator<Event>`	`rangeIterator(long fromTime, long toTime)` Visits completed events that start in the given interval in order by their start time.
`protected void`	`setHostScores(HostScore[] a)` Normalizes the `ServiceScore`s and set them in place.
`protected void`	`setServiceScores(ServiceScore[] a)` Normalizes the `ServiceScore`s and set them in place.
`void`	`shutdown()` The service will no longer accept new requests, but existing requests will be processed (sychronous).
`void`	`shutdownNow()` The service will no longer accept new requests and will make a best effort attempt to terminate all existing requests and return ASAP.
`void`	`sighup()` Logs counters to a temp file.
`LoadBalancerService`	`start()` Starts the `AbstractService`.
`void`	`urgent(String msg, UUID serviceUUID)` An urgent warning issued the caller is in immediate danger of depleting its resources with a consequence of immediate service and/or host failure(s).
`void`	`warn(String msg, UUID serviceUUID)` A warning issued by a client when it is in danger of depleting its resources.

Methods inherited from class com.bigdata.service.AbstractService
clearLoggingContext, getFederation, getHostname, getServiceName, getServiceUUID, setServiceUUID, setupLoggingContext

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.service.IService
getHostname, getServiceName, getServiceUUID

- Field Detail
  - log
```
protected static final org.apache.log4j.Logger log
```
  - ps
```
protected final String ps
```
    See Also:
    Constant Field Values
  - serviceJoinTimeout
```
protected final long serviceJoinTimeout
```
    Service join timeout in milliseconds - used when we need to wait for a service to join before we can recommend an under-utilized service.
    
    See Also:
    LoadBalancerService.Options.SERVICE_JOIN_TIMEOUT
  - lock
```
protected final ReentrantLock lock
```
    Lock is used to control access to data structures that are not thread-safe.
  - joined
```
protected final Condition joined
```
    Used to await a service join when there are no services.
  - activeHosts
```
protected ConcurrentHashMap<String,HostScore> activeHosts
```
    The active hosts (one or more services).
  - activeDataServices
```
protected ConcurrentHashMap<UUID,ServiceScore> activeDataServices
```
    The set of active services.
  - hostScores
```
protected AtomicReference<HostScore[]> hostScores
```
    Scores for the hosts in ascending order (least utilized to most utilized).
    This array is initially null and gets updated periodically by the LoadBalancerService.UpdateTask. The main consumer of this information is the logic in LoadBalancerService.UpdateTask that computes the service utilization.
  - serviceScores
```
protected AtomicReference<ServiceScore[]> serviceScores
```
    Scores for the services in ascending order (least utilized to most utilized).
    This array is initially null and gets updated periodically by the LoadBalancerService.UpdateTask. The methods that report service utilization and under-utilized services are all based on the data in this array. Since services can leave at any time, that logic MUST also test for existence of the service in activeDataServices before assuming that the service is still live.
  - nupdates
```
protected long nupdates
```
    The #of LoadBalancerService.UpdateTasks which have run so far.
    
    See Also:
    LoadBalancerService.Options.INITIAL_ROUND_ROBIN_UPDATE_COUNT, getUnderUtilizedDataServices(int, int, UUID)
  - initialRoundRobinUpdateCount
```
protected final long initialRoundRobinUpdateCount
```
    The #of updates during which getUnderUtilizedDataServices(int, int, UUID) will apply a round robin policy.
    
    See Also:
    LoadBalancerService.Options.INITIAL_ROUND_ROBIN_UPDATE_COUNT
  - logDir
```
protected final File logDir
```
    The directory in which the service will log the CounterSets and Events.
    
    See Also:
    LoadBalancerService.Options.LOG_DIR
  - isTransient
```
protected final boolean isTransient
```
    true iff the LBS will refrain from writing state on the disk. This option causes the LBS to use an in memory eventStore. In addition, it will refuse to write counter snapshots when this option is specified.
    
    See Also:
    LoadBalancerService.Options.TRANSIENT
  - updateService
```
protected final ScheduledExecutorService updateService
```
    Runs a periodic LoadBalancerService.UpdateTask.
  - historyMinutes
```
protected final int historyMinutes
```
    The #of minutes of history that will be smoothed into an average when LoadBalancerService.UpdateTask updates the HostScores and the ServiceScores.
    
    See Also:
    LoadBalancerService.Options.HISTORY_MINUTES
  - eventStore
```
protected final Journal eventStore
```
    Used to persist the logged events.
  - eventReceiver
```
protected final EventReceiver eventReceiver
```
- Constructor Detail
  - LoadBalancerService
```
public LoadBalancerService(Properties properties)
```
    Note: The load balancer MUST NOT collect host statistics unless it is the only service running on that host. Normally it relies on another service running on the same host to collect statistics for that host and those statistics are then reported to the load balancer and aggregated along with the rest of the performance counters reported by the other services in the federation. However, if the load balancer itself collects host statistics then it will only know about and report the current (last 60 seconds) statistics for the host rather than having the historical data for the host.
    
    Parameters:
    properties - See LoadBalancerService.Options
- Method Detail
  - getProperties
```
public Properties getProperties()
```
    An object wrapping the properties provided to the constructor.
  - getClientHostname
```
protected abstract String getClientHostname()
```
    Return the canonical hostname of the client in the context of a RMI request. If the request is not remote then return the canonical hostname for this host.
  - start
```
public LoadBalancerService start()
```
    Description copied from class: AbstractService
    
    Starts the AbstractService.
    Note: A AbstractService.start() is required in order to give subclasses an opportunity to be fully initialized before they are required to begin operations. It is impossible to encapsulate the startup logic cleanly without this ctor() + start() pattern. Those familiar with Objective-C will recognized this.
    
    Specified by:
    
    start in class AbstractService
    
    Returns:
    this (the return type should be strengthened by the concrete implementation to return the actual type).
  - isOpen
```
public boolean isOpen()
```
    Description copied from interface: IServiceShutdown
    
    Return true iff the service is running.
    
    Specified by:
    
    isOpen in interface IServiceShutdown
  - finalized
```
protected void finalized()
                  throws Throwable
```
    Throws:
    
    Throwable
  - shutdown
```
public void shutdown()
```
    Description copied from interface: IServiceShutdown
    
    The service will no longer accept new requests, but existing requests will be processed (sychronous). This method should await the termination of pending requests, but no longer than the timeout specified by IServiceShutdown.Options.SHUTDOWN_TIMEOUT. Implementations SHOULD be synchronized. If the service is aleady shutdown, then this method should be a NOP.
    
    Specified by:
    
    shutdown in interface IServiceShutdown
    
    Overrides:
    
    shutdown in class AbstractService
  - shutdownNow
```
public void shutdownNow()
```
    Description copied from interface: IServiceShutdown
    
    The service will no longer accept new requests and will make a best effort attempt to terminate all existing requests and return ASAP. This method should terminate any asynchronous processing, release all resources and return immediately. Implementations SHOULD be synchronized. If the service is aleady shutdown, then this method should be a NOP.
    
    Specified by:
    
    shutdownNow in interface IServiceShutdown
    
    Overrides:
    
    shutdownNow in class AbstractService
  - destroy
```
public void destroy()
```
    Description copied from interface: IService
    
    Destroy the service. If the service is running, it is shutdown immediately and then destroyed. This method has the same signature as DestroyAdmin#destroy().
    
    Specified by:
    
    destroy in interface IService
    
    Overrides:
    
    destroy in class AbstractService
  - getServiceIface
```
public final Class getServiceIface()
```
    Returns ILoadBalancerService.
    
    Specified by:
    
    getServiceIface in interface IService
    
    Specified by:
    
    getServiceIface in class AbstractService
  - setHostScores
```
protected void setHostScores(HostScore[] a)
```
    Normalizes the ServiceScores and set them in place.
    
    Parameters:
    a - The new service scores.
  - setServiceScores
```
protected void setServiceScores(ServiceScore[] a)
```
    Normalizes the ServiceScores and set them in place.
    
    Parameters:
    a - The new service scores.
  - logCounters
```
protected void logCounters(String basename)
```
    Writes the counters on a file.
    
    Parameters:
    basename - The basename of the file. The file will be written in the logDir.
  - logCounters
```
protected void logCounters(File file)
```
    Writes the counters on a file.
    
    Parameters:
    file - The file. If the file exists it will be overwritten.
  - logCounters
```
public void logCounters()
                 throws IOException
```
    Logs the counters on a file created using File.createTempFile(String, String, File) in the log directory.
    
    Throws:
    
    IOException
  - sighup
```
public void sighup()
            throws IOException
```
    Description copied from interface: ILoadBalancerService
    
    Logs counters to a temp file. Replacement for sighup mechanism.
    
    Specified by:
    
    sighup in interface ILoadBalancerService
    
    Throws:
    
    IOException
  - join
```
public void join(UUID serviceUUID,
        Class serviceIface,
        String hostname)
```
    Notify the LoadBalancerService that a new service is available.
    Note: Embedded services must invoke this method directly when they start up.
    Note: Distributed services implementations MUST discover services using a framework, such as jini, and invoke this method the first time a given service is discovered.
    
    Parameters:
    serviceUUID -
    serviceIface -
    hostname -
    See Also:
    IFederationDelegate.serviceJoin(IService, UUID), #leave(String, UUID)
  - leave
```
public void leave(UUID serviceUUID)
```
    Notify the LoadBalancerService that a service is no longer available.
    Note: Embedded services must invoke this method directly when they shut down.
    
    Note: Distributed services implementations MUST discover services using a framework, such as jini, and invoke this method when a service is no longer registered.
    
    Parameters:
    serviceUUID - The service UUID.
    See Also:
    IFederationDelegate.serviceLeave(UUID), join(UUID, Class, String)
  - notifyEvent
```
public void notifyEvent(Event e)
                 throws IOException
```
    Accepts the event, either updates the existing event with the same UUID or adds the event to the set of recent events, and then prunes the set of recent events so that all completed events older than #eventHistoryMillis are discarded.
    
    Specified by:
    
    notifyEvent in interface IEventReceivingService
    
    Throws:
    
    IOException
    See Also:
    EventReceiver
  - rangeIterator
```
public Iterator<Event> rangeIterator(long fromTime,
                            long toTime)
```
    Visits completed events that start in the given interval in order by their start time.
    
    Specified by:
    
    rangeIterator in interface IEventReportingService
  - rangeCount
```
public long rangeCount(long fromTime,
              long toTime)
```
    Reports the #of completed events that start in the given interval.
    
    Specified by:
    
    rangeCount in interface IEventReportingService
    
    Parameters:
    fromTime - The first start time to be included.
    toTime - The first start time to be excluded.
    
    Returns:
    The #of events whose start time is in that interval.
  - notify
```
public void notify(UUID serviceUUID,
          byte[] data)
```
    Description copied from interface: ILoadBalancerService
    
    Send performance counters. Clients SHOULD invoke this method no less than once every 60 seconds.
    
    Specified by:
    
    notify in interface ILoadBalancerService
    
    Parameters:
    serviceUUID - The service UUID that is self-reporting.
    data - The serialized performance counter data.
  - warn
```
public void warn(String msg,
        UUID serviceUUID)
```
    Description copied from interface: ILoadBalancerService
    
    A warning issued by a client when it is in danger of depleting its resources.
    
    Specified by:
    
    warn in interface ILoadBalancerService
    
    Parameters:
    msg - A message.
    serviceUUID - The service UUID that is self-reporting.
  - urgent
```
public void urgent(String msg,
          UUID serviceUUID)
```
    Description copied from interface: ILoadBalancerService
    
    An urgent warning issued the caller is in immediate danger of depleting its resources with a consequence of immediate service and/or host failure(s).
    
    Specified by:
    
    urgent in interface ILoadBalancerService
    
    Parameters:
    msg - A message.
    serviceUUID - The service UUID that is self-reporting.
  - isHighlyUtilizedDataService
```
public boolean isHighlyUtilizedDataService(UUID serviceUUID)
                                    throws IOException
```
    Description copied from interface: ILoadBalancerService
    
    Return true if the service is considered to be "highly utilized".
    Note: This is used mainly to decide when a service should attempt to shed index partitions. This implementation SHOULD reflect the relative rank of the service among all services as well as its absolute load.
    
    Specified by:
    
    isHighlyUtilizedDataService in interface ILoadBalancerService
    
    Parameters:
    serviceUUID - The service UUID.
    
    Returns:
    true if the service is considered to be "highly utilized".
    
    Throws:
    
    IOException
  - isUnderUtilizedDataService
```
public boolean isUnderUtilizedDataService(UUID serviceUUID)
                                   throws IOException
```
    Description copied from interface: ILoadBalancerService
    
    Return true if the service is considered to be "under-utilized".
    
    Specified by:
    
    isUnderUtilizedDataService in interface ILoadBalancerService
    
    Parameters:
    serviceUUID - The service UUID.
    
    Returns:
    true if the service is considered to be "under-utilized".
    
    Throws:
    
    IOException
  - isHighlyUtilizedDataService
```
protected boolean isHighlyUtilizedDataService(ServiceScore score,
                                  ServiceScore[] scores)
```
  - isUnderUtilizedDataService
```
protected boolean isUnderUtilizedDataService(ServiceScore score,
                                 ServiceScore[] scores)
```
  - getUnderUtilizedDataService
```
public UUID getUnderUtilizedDataService()
                                 throws IOException,
                                        TimeoutException,
                                        InterruptedException
```
    Description copied from interface: ILoadBalancerService
    
    Return the UUID of an under-utilized data service. If there is no under-utilized service, then return the UUID of the service with the least load.
    
    Specified by:
    
    getUnderUtilizedDataService in interface ILoadBalancerService
    
    Throws:
    
    TimeoutException - if there are no data services and a timeout occurs while awaiting a service join.
    
    InterruptedException - if the request is interrupted.
    
    IOException
  - getUnderUtilizedDataServices
```
public UUID[] getUnderUtilizedDataServices(int minCount,
                                  int maxCount,
                                  UUID exclude)
                                    throws IOException,
                                           TimeoutException,
                                           InterruptedException
```
    Description copied from interface: ILoadBalancerService
    
    Return up to limit IDataService UUIDs that are currently under-utilized.
    When minCount is positive, this method will always return at least minCount service UUIDs, however the UUIDs returned MAY contain duplicates if the LoadBalancerService has a strong preference for allocating load to some services (or for NOT allocating load to other services). Further, the LoadBalancerService MAY choose (or be forced to choose) to return UUIDs for services that are within a nominal utilization range, or even UUIDs for services that are highly-utilized if it could otherwise not satisify the request.
    
    Specified by:
    
    getUnderUtilizedDataServices in interface ILoadBalancerService
    
    Parameters:
    minCount - The minimum #of services UUIDs to return -or- zero (0) if there is no minimum limit.
    maxCount - The maximum #of services UUIDs to return -or- zero (0) if there is no maximum limit.
    exclude - The optional UUID of a data service to be excluded from the returned set.
    
    Returns:
    Up to maxCount under-utilized services -or- null IFF no services are recommended at this time as needing additional load.
    
    Throws:
    
    TimeoutException - if there are no data services, or if there is only a single data service and it is excluded by the request, and a timeout occurs while awaiting a service join.
    
    InterruptedException - if the request is interrupted.
    
    IOException

Class LoadBalancerService

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class com.bigdata.service.AbstractService

Methods inherited from class java.lang.Object

Methods inherited from interface com.bigdata.service.IService

Field Detail

log

ps

serviceJoinTimeout

lock

joined

activeHosts

activeDataServices

hostScores

serviceScores

nupdates

initialRoundRobinUpdateCount

logDir

isTransient

updateService

historyMinutes

eventStore

eventReceiver

Constructor Detail

LoadBalancerService

Method Detail

getProperties

getClientHostname

start

isOpen

finalized

shutdown

shutdownNow

destroy

getServiceIface

setHostScores

setServiceScores

logCounters

logCounters

logCounters

sighup

join

leave

notifyEvent

rangeIterator

rangeCount

notify

warn

urgent

isHighlyUtilizedDataService

isUnderUtilizedDataService

isHighlyUtilizedDataService

isUnderUtilizedDataService

getUnderUtilizedDataService

getUnderUtilizedDataServices