com.bigdata.service

Class LoadBalancerService.UpdateTask

java.lang.Object

com.bigdata.service.LoadBalancerService.UpdateTask

All Implemented Interfaces:

Runnable

Enclosing class:

LoadBalancerService

protected class LoadBalancerService.UpdateTask
extends Object
implements Runnable

Computes and updates the ServiceScores based on an examination of aggregated performance counters.

Version:

$Id$

Author:

Bryan Thompson

See Also:

QueryUtil.getRequiredPerformanceCountersFilter()

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.log4j.Logger	log Note: The logger is named for this class, but since it is an inner class the name uses a "$" delimiter (vs a ".") between the outer and the inner class names.

Constructor Summary

Constructors

Constructor and Description
LoadBalancerService.UpdateTask()

Method Summary

Methods

Modifier and Type	Method and Description
protected ServiceScore	computeScore(HostScore hostScore, UUID serviceUUID, ICounterSet hostCounterSet, ICounterSet serviceCounterSet) Compute the score for a service.
protected HostScore	computeScore(String hostname, ICounterSet hostCounterSet) Compute the score for a host.
protected double	getAverageValueForMinutes(ICounterSet counterSet, String path, double defaultValue, int minutes) Return the average of the counter having the given path over the last minutes minutes.
protected double	getCurrentValue(ICounterSet counterSet, String path, double defaultValue)
protected void	logCounters() Writes the counters on a file.
void	run() Note: Don't throw anything here since we don't want to have the task suppressed!
protected void	setupCounters() Sets up reporting for the computed per-host and per-service scores.
protected void	updateHostScores() (Re-)compute the utilization score for each active host.
protected void	updateServiceScores() (Re-)compute the utilization score for each active service.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected final transient org.apache.log4j.Logger log

Note: The logger is named for this class, but since it is an inner class the name uses a "$" delimiter (vs a ".") between the outer and the inner class names.

Constructor Detail

LoadBalancerService.UpdateTask

public LoadBalancerService.UpdateTask()

Method Detail

run

public void run()

Note: Don't throw anything here since we don't want to have the task suppressed!

Specified by:

run in interface Runnable

updateHostScores

protected void updateHostScores()

(Re-)compute the utilization score for each active host.

updateServiceScores

protected void updateServiceScores()

(Re-)compute the utilization score for each active service.

Note: There is a dependency on AbstractFederation.getServiceCounterPathPrefix(UUID, Class, String). This method assumes that the service UUID is found in a specific place in the constructed path.

computeScore

protected HostScore computeScore(String hostname,
                     ICounterSet hostCounterSet)

Compute the score for a host.

The host scores MUST reflect critical resource exhaustion, especially DISK free space, which can take down all services on the host, and SWAPPING, which can bring the effective throughput of the host to a halt. All other resources fail soft, by causing the response time to increase.

Note: DISK exhaustion can lead to immediate failure of all services on the same host. A host that is nearing DISK exhaustion SHOULD get heavily dinged and an admin SHOULD be alerted.

The correct response for heavy swapping is to alert an admin to shutdown one or more processes on that host. If you do not have failover provisioned for your data services then DO NOT shutdown data services or you WILL loose data!

Note: If we are not getting critical counters for some host then we are assuming a reasonable values for the missing data and computing the utilization based on those assumptions. Note that a value of zero (0) may be interpreted as either critically high utilization or no utilization depending on the performance counter involved and that the impact of the different counters can vary depending on the formula used to compute the utilization score.

Parameters:

hostname - The fully qualified hostname.

hostCounterSet - The performance counters for that host.

serviceCounterSet - The performance counters for that service.

Returns:

The computed host score.

computeScore

protected ServiceScore computeScore(HostScore hostScore,
                        UUID serviceUUID,
                        ICounterSet hostCounterSet,
                        ICounterSet serviceCounterSet)

Compute the score for a service.

Note: utilization is defined in terms of transient system resources : CPU, IO (DISK and NET), RAM. A host with enough CPU/RAM/IO/DISK can support more than one data service. Therefore it is important to look at not just host utilization but also at process utilization.

Parameters:

hostScore - The pre-computed score for the host on which the service is running.

serviceUUID - The service UUID.

hostCounterSet - The performance counters for that host (in case you need anything that is not already in the HostScore).

serviceCounterSet - The performance counters for that service.

Returns:

The computed score for that service.

getCurrentValue

protected double getCurrentValue(ICounterSet counterSet,
                     String path,
                     double defaultValue)

getAverageValueForMinutes

protected double getAverageValueForMinutes(ICounterSet counterSet,
                               String path,
                               double defaultValue,
                               int minutes)

Return the average of the counter having the given path over the last minutes minutes.

Parameters:

counterSet -

path -

defaultValue -

minutes -

Returns:

FIXME should be a weighted average, right?

setupCounters

protected void setupCounters()

Sets up reporting for the computed per-host and per-service scores. These counters are reported under the service UUID for the LoadBalancerService itself. This makes it easy to consult the scores for the various hosts and services.

Note: The host and service scores will not appear until the LoadBalancerService.UpdateTask has executed and those scores have been computed.

See Also:

LoadBalancerService.Options.UPDATE_DELAY

logCounters

protected void logCounters()

Writes the counters on a file.

See Also:

LoadBalancerService.Options