org.apache.spark.sql.execution

Class SparkStrategies.HashJoin

Object

org.apache.spark.sql.catalyst.planning.GenericStrategy<SparkPlan>

org.apache.spark.sql.execution.SparkStrategies.HashJoin

All Implemented Interfaces:

Logging, org.apache.spark.sql.catalyst.expressions.PredicateHelper

Enclosing class:

SparkStrategies

public class SparkStrategies.HashJoin
extends org.apache.spark.sql.catalyst.planning.GenericStrategy<SparkPlan>
implements org.apache.spark.sql.catalyst.expressions.PredicateHelper

Constructor Summary

Constructors

Constructor and Description
SparkStrategies.HashJoin() Uses the ExtractEquiJoinKeys pattern to find joins where at least some of the predicates can be evaluated by matching hash keys.

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.Seq<SparkPlan>	apply(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)

Methods inherited from class org.apache.spark.sql.catalyst.planning.GenericStrategy

isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$Logging$$log__$eq, org$apache$spark$Logging$$log_

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.catalyst.expressions.PredicateHelper

canEvaluate, splitConjunctivePredicates

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, log_

Constructor Detail

SparkStrategies.HashJoin

public SparkStrategies.HashJoin()

Uses the ExtractEquiJoinKeys pattern to find joins where at least some of the predicates can be evaluated by matching hash keys.

This strategy applies a simple optimization based on the estimates of the physical sizes of the two join sides. When planning a BroadcastHashJoin, if one side has an estimated physical size smaller than the user-settable threshold org.apache.spark.sql.SQLConf.AUTO_BROADCASTJOIN_THRESHOLD, the planner would mark it as the ''build'' relation and mark the other relation as the ''stream'' side. The build table will be ''broadcasted'' to all of the executors involved in the join, as a Broadcast object. If both estimates exceed the threshold, they will instead be used to decide the build side in a ShuffledHashJoin.

Method Detail

apply

public scala.collection.Seq<SparkPlan> apply(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)

Specified by:

apply in class org.apache.spark.sql.catalyst.planning.GenericStrategy<SparkPlan>