org.apache.spark.ml.regression

Class GBTRegressor

Object

org.apache.spark.ml.PipelineStage

org.apache.spark.ml.Estimator<M>

org.apache.spark.ml.Predictor<Vector,GBTRegressor,GBTRegressionModel>

org.apache.spark.ml.regression.GBTRegressor

All Implemented Interfaces:

java.io.Serializable, Logging, Params, HasCheckpointInterval, HasFeaturesCol, HasLabelCol, HasMaxIter, HasPredictionCol, HasSeed, HasStepSize, HasValidationIndicatorCol, PredictorParams, DecisionTreeParams, GBTParams, GBTRegressorParams, TreeEnsembleParams, TreeRegressorParams, DefaultParamsWritable, Identifiable, MLWritable

public class GBTRegressor
extends Predictor<Vector,GBTRegressor,GBTRegressionModel>
implements GBTRegressorParams, DefaultParamsWritable, Logging

Gradient-Boosted Trees (GBTs) learning algorithm for regression. It supports both continuous and categorical features.

The implementation is based upon: J.H. Friedman. "Stochastic Gradient Boosting." 1999.

Notes on Gradient Boosting vs. TreeBoost: - This implementation is for Stochastic Gradient Boosting, not for TreeBoost. - Both algorithms learn tree ensembles by minimizing loss functions. - TreeBoost (Friedman, 1999) additionally modifies the outputs at tree leaf nodes based on the loss function, whereas the original gradient boosting method does not. - When the loss is SquaredError, these methods give the same result, but they could differ for other loss functions. - We expect to implement TreeBoost in the future: [https://issues.apache.org/jira/browse/SPARK-4240]

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
GBTRegressor()
GBTRegressor(String uid)

Method Summary

Modifier and Type	Method and Description
GBTRegressor	copy(ParamMap extra) Creates a copy of this instance with the same UID and some extra params.
static GBTRegressor	load(String path)
static MLReader<T>	read()
GBTRegressor	setCacheNodeIds(boolean value)
GBTRegressor	setCheckpointInterval(int value) Specifies how often to checkpoint the cached node IDs.
GBTRegressor	setFeatureSubsetStrategy(String value)
GBTRegressor	setImpurity(String value) The impurity setting is ignored for GBT models.
GBTRegressor	setLossType(String value)
GBTRegressor	setMaxBins(int value)
GBTRegressor	setMaxDepth(int value)
GBTRegressor	setMaxIter(int value)
GBTRegressor	setMaxMemoryInMB(int value)
GBTRegressor	setMinInfoGain(double value)
GBTRegressor	setMinInstancesPerNode(int value)
GBTRegressor	setSeed(long value)
GBTRegressor	setStepSize(double value)
GBTRegressor	setSubsamplingRate(double value)
GBTRegressor	setValidationIndicatorCol(String value)
static String[]	supportedLossTypes() Accessor for supported loss settings: squared (L2), absolute (L1)
String	uid() An immutable unique ID for the object and its derivatives.

Methods inherited from class org.apache.spark.ml.Predictor

fit, setFeaturesCol, setLabelCol, setPredictionCol, transformSchema

Methods inherited from class org.apache.spark.ml.Estimator

fit, fit, fit, fit

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.tree.GBTRegressorParams

convertToOldLossType, getLossType, getOldLossType, lossType

Methods inherited from interface org.apache.spark.ml.tree.GBTParams

getOldBoostingStrategy, getValidationTol, stepSize, validationTol

Methods inherited from interface org.apache.spark.ml.tree.TreeEnsembleParams

featureSubsetStrategy, getFeatureSubsetStrategy, getOldStrategy, getSubsamplingRate, subsamplingRate

Methods inherited from interface org.apache.spark.ml.tree.DecisionTreeParams

cacheNodeIds, getCacheNodeIds, getMaxBins, getMaxDepth, getMaxMemoryInMB, getMinInfoGain, getMinInstancesPerNode, getOldStrategy, maxBins, maxDepth, maxMemoryInMB, minInfoGain, minInstancesPerNode

Methods inherited from interface org.apache.spark.ml.PredictorParams

validateAndTransformSchema

Methods inherited from interface org.apache.spark.ml.param.shared.HasLabelCol

getLabelCol, labelCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasFeaturesCol

featuresCol, getFeaturesCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasPredictionCol

getPredictionCol, predictionCol

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Methods inherited from interface org.apache.spark.ml.util.Identifiable

toString

Methods inherited from interface org.apache.spark.ml.param.shared.HasCheckpointInterval

checkpointInterval, getCheckpointInterval

Methods inherited from interface org.apache.spark.ml.param.shared.HasSeed

getSeed, seed

Methods inherited from interface org.apache.spark.ml.param.shared.HasMaxIter

getMaxIter, maxIter

Methods inherited from interface org.apache.spark.ml.param.shared.HasStepSize

getStepSize

Methods inherited from interface org.apache.spark.ml.param.shared.HasValidationIndicatorCol

getValidationIndicatorCol, validationIndicatorCol

Methods inherited from interface org.apache.spark.ml.tree.TreeRegressorParams

getImpurity, getOldImpurity, impurity

Methods inherited from interface org.apache.spark.ml.util.DefaultParamsWritable

write

Methods inherited from interface org.apache.spark.ml.util.MLWritable

save

Methods inherited from interface org.apache.spark.internal.Logging

initializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

GBTRegressor

public GBTRegressor(String uid)

GBTRegressor

public GBTRegressor()

Method Detail

supportedLossTypes

public static final String[] supportedLossTypes()

Accessor for supported loss settings: squared (L2), absolute (L1)

load

public static GBTRegressor load(String path)

read

public static MLReader<T> read()

uid

public String uid()

Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Specified by:

uid in interface Identifiable

Returns:

(undocumented)

setMaxDepth

public GBTRegressor setMaxDepth(int value)

Specified by:

setMaxDepth in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMaxBins

public GBTRegressor setMaxBins(int value)

Specified by:

setMaxBins in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMinInstancesPerNode

public GBTRegressor setMinInstancesPerNode(int value)

Specified by:

setMinInstancesPerNode in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMinInfoGain

public GBTRegressor setMinInfoGain(double value)

Specified by:

setMinInfoGain in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMaxMemoryInMB

public GBTRegressor setMaxMemoryInMB(int value)

Specified by:

setMaxMemoryInMB in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setCacheNodeIds

public GBTRegressor setCacheNodeIds(boolean value)

Specified by:

setCacheNodeIds in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setCheckpointInterval

public GBTRegressor setCheckpointInterval(int value)

Specifies how often to checkpoint the cached node IDs. E.g. 10 means that the cache will get checkpointed every 10 iterations. This is only used if cacheNodeIds is true and if the checkpoint directory is set in SparkContext. Must be at least 1. (default = 10)

Specified by:

setCheckpointInterval in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setImpurity

public GBTRegressor setImpurity(String value)

The impurity setting is ignored for GBT models. Individual trees are built using impurity "Variance."

Specified by:

setImpurity in interface TreeRegressorParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setSubsamplingRate

public GBTRegressor setSubsamplingRate(double value)

Specified by:

setSubsamplingRate in interface TreeEnsembleParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setSeed

public GBTRegressor setSeed(long value)

Specified by:

setSeed in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMaxIter

public GBTRegressor setMaxIter(int value)

Specified by:

setMaxIter in interface GBTParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setStepSize

public GBTRegressor setStepSize(double value)

Specified by:

setStepSize in interface GBTParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setLossType

public GBTRegressor setLossType(String value)

setFeatureSubsetStrategy

public GBTRegressor setFeatureSubsetStrategy(String value)

Specified by:

setFeatureSubsetStrategy in interface TreeEnsembleParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setValidationIndicatorCol

public GBTRegressor setValidationIndicatorCol(String value)

copy

public GBTRegressor copy(ParamMap extra)

Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().

Specified by:

copy in interface Params

Specified by:

copy in class Predictor<Vector,GBTRegressor,GBTRegressionModel>

Parameters:

extra - (undocumented)

Returns:

(undocumented)