package tensorflow.boosted_trees.learner

Get desktop application:
View/edit binary Protocol Buffers messages

When we have a sequence of trees 1, 2, 3 ... n, these essentially represent weights updates in functional space, and thus we can use averaging of weight updates to achieve better performance. For example, we can say that our final ensemble will be an average of ensembles of tree 1, and ensemble of tree 1 and tree 2 etc .. ensemble of all trees. Note that this averaging will apply ONLY DURING PREDICTION. The training stays the same.

Used in: LearnerConfig

• oneof config

  • float average_last_n_trees = 1

  • float average_last_percent_trees = 2

    Between 0 and 1. If set to 1.0, we are averaging ensembles of tree 1, ensemble of tree 1 and tree 2, etc ensemble of all trees. If set to 0.5, last half of the trees are averaged etc.

• uint32 num_classes = 1

  Number of classes.

• oneof feature_fraction

  Fraction of features to consider in each tree sampled randomly from all available features.

  • float feature_fraction_per_tree = 2

  • float feature_fraction_per_level = 3

• optional TreeRegularizationConfig regularization = 4

  Regularization.

• optional TreeConstraintsConfig constraints = 5

  Constraints.

• LearnerConfig.PruningMode pruning_mode = 8

  Pruning.

• LearnerConfig.GrowingMode growing_mode = 9

  Growing Mode.

• optional LearningRateConfig learning_rate_tuner = 6

  Learning rate.

• LearnerConfig.MultiClassStrategy multi_class_strategy = 10

  Multi-class strategy.

• optional AveragingConfig averaging_config = 11

  If you want to average the ensembles (for regularization), provide the config below.

Used in: LearnerConfig

• WHOLE_TREE = 0

• LAYER_BY_LAYER = 1

  Layer by layer is only supported by the batch learner.

Used in: LearnerConfig

• TREE_PER_CLASS = 0

• FULL_HESSIAN = 1

• DIAGONAL_HESSIAN = 2

Used in: LearnerConfig

• PRE_PRUNE = 0

• POST_PRUNE = 1

LearningRateConfig describes all supported learning rate tuners.

Used in: LearnerConfig

• oneof tuner

  • LearningRateFixedConfig fixed = 1

  • LearningRateDropoutDrivenConfig dropout = 2

  • LearningRateLineSearchConfig line_search = 3

Used in: LearningRateConfig

• float dropout_probability = 1

  Probability of dropping each tree in an existing so far ensemble.

• float probability_of_skipping_dropout = 2

  When trees are built after dropout happen, they don't "advance" to the optimal solution, they just rearrange the path. However you can still choose to skip dropout periodically, to allow a new tree that "advances" to be added. For example, if running for 200 steps with probability of dropout 1/100, you would expect the dropout to start happening for sure for all iterations after 100. However you can add probability_of_skipping_dropout of 0.1, this way iterations 100-200 will include approx 90 iterations of dropout and 10 iterations of normal steps.Set it to 0 if you want just keep building the refinement trees after dropout kicks in.

• float learning_rate = 3

  Between 0 and 1.

Config for a fixed learning rate.

Used in: LearningRateConfig

• float learning_rate = 1

Config for a tuned learning rate.

Used in: LearningRateConfig

• float max_learning_rate = 1

  Max learning rate. Must be strictly positive.

• int32 num_steps = 2

  Number of learning rate values to consider between [0, max_learning_rate).

Gathered information for a split node.

• optional trees.TreeNode split_node = 1

  The split node without the child nodes attached.

• optional trees.Leaf left_child = 2

  Left Leaf node.

• optional trees.Leaf right_child = 3

  Right Leaf node.

Tree constraints config.

Used in: LearnerConfig

• uint32 max_tree_depth = 1

  Maximum depth of the trees.

• float min_node_weight = 2

  Min hessian weight per node.

Tree regularization config.

Used in: LearnerConfig

• float l1 = 1

  Classic L1/L2.

• float l2 = 2

• float tree_complexity = 3

  Tree complexity penalizes overall model complexity effectively limiting how deep the tree can grow in regions with small gain.