@Override protected Optional<PlanNode> pushDownProjectOff(PlanNodeIdAllocator idAllocator, SemiJoinNode semiJoinNode, Set<Symbol> referencedOutputs) { if (!referencedOutputs.contains(semiJoinNode.getSemiJoinOutput())) { return Optional.of(semiJoinNode.getSource()); } Set<Symbol> requiredSourceInputs = Streams.concat( referencedOutputs.stream() .filter(symbol -> !symbol.equals(semiJoinNode.getSemiJoinOutput())), Stream.of(semiJoinNode.getSourceJoinSymbol()), semiJoinNode.getSourceHashSymbol().map(Stream::of).orElse(Stream.empty())) .collect(toImmutableSet()); return restrictOutputs(idAllocator, semiJoinNode.getSource(), requiredSourceInputs) .map(newSource -> semiJoinNode.replaceChildren(ImmutableList.of( newSource, semiJoinNode.getFilteringSource()))); } }
@Override protected Optional<PlanNode> pushDownProjectOff(PlanNodeIdAllocator idAllocator, SemiJoinNode semiJoinNode, Set<Symbol> referencedOutputs) { if (!referencedOutputs.contains(semiJoinNode.getSemiJoinOutput())) { return Optional.of(semiJoinNode.getSource()); } Set<Symbol> requiredSourceInputs = Streams.concat( referencedOutputs.stream() .filter(symbol -> !symbol.equals(semiJoinNode.getSemiJoinOutput())), Stream.of(semiJoinNode.getSourceJoinSymbol()), semiJoinNode.getSourceHashSymbol().map(Stream::of).orElse(Stream.empty())) .collect(toImmutableSet()); return restrictOutputs(idAllocator, semiJoinNode.getSource(), requiredSourceInputs) .map(newSource -> semiJoinNode.replaceChildren(ImmutableList.of( newSource, semiJoinNode.getFilteringSource()))); } }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<Expression> context) { Expression inheritedPredicate = context.get(); if (!extractConjuncts(inheritedPredicate).contains(node.getSemiJoinOutput().toSymbolReference())) { return visitNonFilteringSemiJoin(node, context); } return visitFilteringSemiJoin(node, context); }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<Expression> context) { Expression inheritedPredicate = context.get(); if (!extractConjuncts(inheritedPredicate).contains(node.getSemiJoinOutput().toSymbolReference())) { return visitNonFilteringSemiJoin(node, context); } return visitFilteringSemiJoin(node, context); }
@Override public Void visitSemiJoin(SemiJoinNode node, Set<Symbol> boundSymbols) { node.getSource().accept(this, boundSymbols); node.getFilteringSource().accept(this, boundSymbols); checkArgument(node.getSource().getOutputSymbols().contains(node.getSourceJoinSymbol()), "Symbol from semi join clause (%s) not in source (%s)", node.getSourceJoinSymbol(), node.getSource().getOutputSymbols()); checkArgument(node.getFilteringSource().getOutputSymbols().contains(node.getFilteringSourceJoinSymbol()), "Symbol from semi join clause (%s) not in filtering source (%s)", node.getSourceJoinSymbol(), node.getFilteringSource().getOutputSymbols()); Set<Symbol> outputs = createInputs(node, boundSymbols); checkArgument(outputs.containsAll(node.getSource().getOutputSymbols()), "Semi join output symbols (%s) must contain all of the source symbols (%s)", node.getOutputSymbols(), node.getSource().getOutputSymbols()); checkArgument(outputs.contains(node.getSemiJoinOutput()), "Semi join output symbols (%s) must contain join result (%s)", node.getOutputSymbols(), node.getSemiJoinOutput()); return null; }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<Set<Symbol>> context) { ImmutableSet.Builder<Symbol> sourceInputsBuilder = ImmutableSet.builder(); sourceInputsBuilder.addAll(context.get()).add(node.getSourceJoinSymbol()); if (node.getSourceHashSymbol().isPresent()) { sourceInputsBuilder.add(node.getSourceHashSymbol().get()); } Set<Symbol> sourceInputs = sourceInputsBuilder.build(); ImmutableSet.Builder<Symbol> filteringSourceInputBuilder = ImmutableSet.builder(); filteringSourceInputBuilder.add(node.getFilteringSourceJoinSymbol()); if (node.getFilteringSourceHashSymbol().isPresent()) { filteringSourceInputBuilder.add(node.getFilteringSourceHashSymbol().get()); } Set<Symbol> filteringSourceInputs = filteringSourceInputBuilder.build(); PlanNode source = context.rewrite(node.getSource(), sourceInputs); PlanNode filteringSource = context.rewrite(node.getFilteringSource(), filteringSourceInputs); return new SemiJoinNode(node.getId(), source, filteringSource, node.getSourceJoinSymbol(), node.getFilteringSourceJoinSymbol(), node.getSemiJoinOutput(), node.getSourceHashSymbol(), node.getFilteringSourceHashSymbol(), node.getDistributionType()); }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<Set<Symbol>> context) { ImmutableSet.Builder<Symbol> sourceInputsBuilder = ImmutableSet.builder(); sourceInputsBuilder.addAll(context.get()).add(node.getSourceJoinSymbol()); if (node.getSourceHashSymbol().isPresent()) { sourceInputsBuilder.add(node.getSourceHashSymbol().get()); } Set<Symbol> sourceInputs = sourceInputsBuilder.build(); ImmutableSet.Builder<Symbol> filteringSourceInputBuilder = ImmutableSet.builder(); filteringSourceInputBuilder.add(node.getFilteringSourceJoinSymbol()); if (node.getFilteringSourceHashSymbol().isPresent()) { filteringSourceInputBuilder.add(node.getFilteringSourceHashSymbol().get()); } Set<Symbol> filteringSourceInputs = filteringSourceInputBuilder.build(); PlanNode source = context.rewrite(node.getSource(), sourceInputs); PlanNode filteringSource = context.rewrite(node.getFilteringSource(), filteringSourceInputs); return new SemiJoinNode(node.getId(), source, filteringSource, node.getSourceJoinSymbol(), node.getFilteringSourceJoinSymbol(), node.getSemiJoinOutput(), node.getSourceHashSymbol(), node.getFilteringSourceHashSymbol(), node.getDistributionType()); }
@Override public Void visitSemiJoin(SemiJoinNode node, Set<Symbol> boundSymbols) { node.getSource().accept(this, boundSymbols); node.getFilteringSource().accept(this, boundSymbols); checkArgument(node.getSource().getOutputSymbols().contains(node.getSourceJoinSymbol()), "Symbol from semi join clause (%s) not in source (%s)", node.getSourceJoinSymbol(), node.getSource().getOutputSymbols()); checkArgument(node.getFilteringSource().getOutputSymbols().contains(node.getFilteringSourceJoinSymbol()), "Symbol from semi join clause (%s) not in filtering source (%s)", node.getSourceJoinSymbol(), node.getFilteringSource().getOutputSymbols()); Set<Symbol> outputs = createInputs(node, boundSymbols); checkArgument(outputs.containsAll(node.getSource().getOutputSymbols()), "Semi join output symbols (%s) must contain all of the source symbols (%s)", node.getOutputSymbols(), node.getSource().getOutputSymbols()); checkArgument(outputs.contains(node.getSemiJoinOutput()), "Semi join output symbols (%s) must contain join result (%s)", node.getOutputSymbols(), node.getSemiJoinOutput()); return null; }
private Optional<PlanNodeStatsEstimate> calculate(FilterNode filterNode, SemiJoinNode semiJoinNode, StatsProvider statsProvider, Session session, TypeProvider types) { PlanNodeStatsEstimate sourceStats = statsProvider.getStats(semiJoinNode.getSource()); PlanNodeStatsEstimate filteringSourceStats = statsProvider.getStats(semiJoinNode.getFilteringSource()); Symbol filteringSourceJoinSymbol = semiJoinNode.getFilteringSourceJoinSymbol(); Symbol sourceJoinSymbol = semiJoinNode.getSourceJoinSymbol(); Optional<SemiJoinOutputFilter> semiJoinOutputFilter = extractSemiJoinOutputFilter(filterNode.getPredicate(), semiJoinNode.getSemiJoinOutput()); if (!semiJoinOutputFilter.isPresent()) { return Optional.empty(); } PlanNodeStatsEstimate semiJoinStats; if (semiJoinOutputFilter.get().isNegated()) { semiJoinStats = computeAntiJoin(sourceStats, filteringSourceStats, sourceJoinSymbol, filteringSourceJoinSymbol); } else { semiJoinStats = computeSemiJoin(sourceStats, filteringSourceStats, sourceJoinSymbol, filteringSourceJoinSymbol); } if (semiJoinStats.isOutputRowCountUnknown()) { return Optional.of(PlanNodeStatsEstimate.unknown()); } // apply remaining predicate PlanNodeStatsEstimate filteredStats = filterStatsCalculator.filterStats(semiJoinStats, semiJoinOutputFilter.get().getRemainingPredicate(), session, types); if (filteredStats.isOutputRowCountUnknown()) { return Optional.of(semiJoinStats.mapOutputRowCount(rowCount -> rowCount * UNKNOWN_FILTER_COEFFICIENT)); } return Optional.of(filteredStats); }
private Optional<PlanNodeStatsEstimate> calculate(FilterNode filterNode, SemiJoinNode semiJoinNode, StatsProvider statsProvider, Session session, TypeProvider types) { PlanNodeStatsEstimate sourceStats = statsProvider.getStats(semiJoinNode.getSource()); PlanNodeStatsEstimate filteringSourceStats = statsProvider.getStats(semiJoinNode.getFilteringSource()); Symbol filteringSourceJoinSymbol = semiJoinNode.getFilteringSourceJoinSymbol(); Symbol sourceJoinSymbol = semiJoinNode.getSourceJoinSymbol(); Optional<SemiJoinOutputFilter> semiJoinOutputFilter = extractSemiJoinOutputFilter(filterNode.getPredicate(), semiJoinNode.getSemiJoinOutput()); if (!semiJoinOutputFilter.isPresent()) { return Optional.empty(); } PlanNodeStatsEstimate semiJoinStats; if (semiJoinOutputFilter.get().isNegated()) { semiJoinStats = computeAntiJoin(sourceStats, filteringSourceStats, sourceJoinSymbol, filteringSourceJoinSymbol); } else { semiJoinStats = computeSemiJoin(sourceStats, filteringSourceStats, sourceJoinSymbol, filteringSourceJoinSymbol); } if (semiJoinStats.isOutputRowCountUnknown()) { return Optional.of(PlanNodeStatsEstimate.unknown()); } // apply remaining predicate PlanNodeStatsEstimate filteredStats = filterStatsCalculator.filterStats(semiJoinStats, semiJoinOutputFilter.get().getRemainingPredicate(), session, types); if (filteredStats.isOutputRowCountUnknown()) { return Optional.of(semiJoinStats.mapOutputRowCount(rowCount -> rowCount * UNKNOWN_FILTER_COEFFICIENT)); } return Optional.of(filteredStats); }
@Override public PlanWithProperties visitSemiJoin(SemiJoinNode node, HashComputationSet parentPreference) { Optional<HashComputation> sourceHashComputation = computeHash(ImmutableList.of(node.getSourceJoinSymbol())); PlanWithProperties source = planAndEnforce( node.getSource(), new HashComputationSet(sourceHashComputation), true, new HashComputationSet(sourceHashComputation)); Symbol sourceHashSymbol = source.getRequiredHashSymbol(sourceHashComputation.get()); Optional<HashComputation> filterHashComputation = computeHash(ImmutableList.of(node.getFilteringSourceJoinSymbol())); HashComputationSet requiredHashes = new HashComputationSet(filterHashComputation); PlanWithProperties filteringSource = planAndEnforce(node.getFilteringSource(), requiredHashes, true, requiredHashes); Symbol filteringSourceHashSymbol = filteringSource.getRequiredHashSymbol(filterHashComputation.get()); return new PlanWithProperties( new SemiJoinNode( node.getId(), source.getNode(), filteringSource.getNode(), node.getSourceJoinSymbol(), node.getFilteringSourceJoinSymbol(), node.getSemiJoinOutput(), Optional.of(sourceHashSymbol), Optional.of(filteringSourceHashSymbol), node.getDistributionType()), source.getHashSymbols()); }
@Override public PlanWithProperties visitSemiJoin(SemiJoinNode node, HashComputationSet parentPreference) { Optional<HashComputation> sourceHashComputation = computeHash(ImmutableList.of(node.getSourceJoinSymbol())); PlanWithProperties source = planAndEnforce( node.getSource(), new HashComputationSet(sourceHashComputation), true, new HashComputationSet(sourceHashComputation)); Symbol sourceHashSymbol = source.getRequiredHashSymbol(sourceHashComputation.get()); Optional<HashComputation> filterHashComputation = computeHash(ImmutableList.of(node.getFilteringSourceJoinSymbol())); HashComputationSet requiredHashes = new HashComputationSet(filterHashComputation); PlanWithProperties filteringSource = planAndEnforce(node.getFilteringSource(), requiredHashes, true, requiredHashes); Symbol filteringSourceHashSymbol = filteringSource.getRequiredHashSymbol(filterHashComputation.get()); return new PlanWithProperties( new SemiJoinNode( node.getId(), source.getNode(), filteringSource.getNode(), node.getSourceJoinSymbol(), node.getFilteringSourceJoinSymbol(), node.getSemiJoinOutput(), Optional.of(sourceHashSymbol), Optional.of(filteringSourceHashSymbol), node.getDistributionType()), source.getHashSymbols()); }
@Override public MatchResult detailMatches(PlanNode node, StatsProvider stats, Session session, Metadata metadata, SymbolAliases symbolAliases) { checkState(shapeMatches(node), "Plan testing framework error: shapeMatches returned false in detailMatches in %s", this.getClass().getName()); SemiJoinNode semiJoinNode = (SemiJoinNode) node; if (!(symbolAliases.get(sourceSymbolAlias).equals(semiJoinNode.getSourceJoinSymbol().toSymbolReference()) && symbolAliases.get(filteringSymbolAlias).equals(semiJoinNode.getFilteringSourceJoinSymbol().toSymbolReference()))) { return NO_MATCH; } return match(outputAlias, semiJoinNode.getSemiJoinOutput().toSymbolReference()); }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<Void> context) { PlanNode sourceRewritten = context.rewrite(node.getSource(), context.get()); PlanNode filteringSourceRewritten = context.rewrite(node.getFilteringSource(), context.get()); SemiJoinNode.DistributionType targetJoinDistributionType = getTargetSemiJoinDistributionType(isDeleteQuery); return new SemiJoinNode( node.getId(), sourceRewritten, filteringSourceRewritten, node.getSourceJoinSymbol(), node.getFilteringSourceJoinSymbol(), node.getSemiJoinOutput(), node.getSourceHashSymbol(), node.getFilteringSourceHashSymbol(), Optional.of(targetJoinDistributionType)); }
@Override public MatchResult detailMatches(PlanNode node, StatsProvider stats, Session session, Metadata metadata, SymbolAliases symbolAliases) { checkState(shapeMatches(node), "Plan testing framework error: shapeMatches returned false in detailMatches in %s", this.getClass().getName()); SemiJoinNode semiJoinNode = (SemiJoinNode) node; if (!(symbolAliases.get(sourceSymbolAlias).equals(semiJoinNode.getSourceJoinSymbol().toSymbolReference()) && symbolAliases.get(filteringSymbolAlias).equals(semiJoinNode.getFilteringSourceJoinSymbol().toSymbolReference()))) { return NO_MATCH; } return match(outputAlias, semiJoinNode.getSemiJoinOutput().toSymbolReference()); }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<Void> context) { PlanNode sourceRewritten = context.rewrite(node.getSource(), context.get()); PlanNode filteringSourceRewritten = context.rewrite(node.getFilteringSource(), context.get()); SemiJoinNode.DistributionType targetJoinDistributionType = getTargetSemiJoinDistributionType(isDeleteQuery); return new SemiJoinNode( node.getId(), sourceRewritten, filteringSourceRewritten, node.getSourceJoinSymbol(), node.getFilteringSourceJoinSymbol(), node.getSemiJoinOutput(), node.getSourceHashSymbol(), node.getFilteringSourceHashSymbol(), Optional.of(targetJoinDistributionType)); }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<LimitContext> context) { PlanNode source = context.rewrite(node.getSource(), context.get()); if (source != node.getSource()) { return new SemiJoinNode( node.getId(), source, node.getFilteringSource(), node.getSourceJoinSymbol(), node.getFilteringSourceJoinSymbol(), node.getSemiJoinOutput(), node.getSourceHashSymbol(), node.getFilteringSourceHashSymbol(), node.getDistributionType()); } return node; } }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<LimitContext> context) { PlanNode source = context.rewrite(node.getSource(), context.get()); if (source != node.getSource()) { return new SemiJoinNode( node.getId(), source, node.getFilteringSource(), node.getSourceJoinSymbol(), node.getFilteringSourceJoinSymbol(), node.getSemiJoinOutput(), node.getSourceHashSymbol(), node.getFilteringSourceHashSymbol(), node.getDistributionType()); } return node; } }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<Void> context) { PlanNode source = context.rewrite(node.getSource()); PlanNode filteringSource = context.rewrite(node.getFilteringSource()); return new SemiJoinNode( node.getId(), source, filteringSource, canonicalize(node.getSourceJoinSymbol()), canonicalize(node.getFilteringSourceJoinSymbol()), canonicalize(node.getSemiJoinOutput()), canonicalize(node.getSourceHashSymbol()), canonicalize(node.getFilteringSourceHashSymbol()), node.getDistributionType()); }
@Override public PlanNode visitSemiJoin(SemiJoinNode node, RewriteContext<Void> context) { PlanNode source = context.rewrite(node.getSource()); PlanNode filteringSource = context.rewrite(node.getFilteringSource()); return new SemiJoinNode( node.getId(), source, filteringSource, canonicalize(node.getSourceJoinSymbol()), canonicalize(node.getFilteringSourceJoinSymbol()), canonicalize(node.getSemiJoinOutput()), canonicalize(node.getSourceHashSymbol()), canonicalize(node.getFilteringSourceHashSymbol()), node.getDistributionType()); }