.equalTo(new Tuple2KeyExtractor()) .window(TumblingEventTimeWindows.of(Time.of(3, TimeUnit.MILLISECONDS))) .apply(new CoGroupFunction<Tuple2<String, Integer>, Tuple2<String, Integer>, String>() {
/** * Completes the join operation with the user function that is executed * for each combination of elements with the same key in a window. * * <p>Note: This method's return type does not support setting an operator-specific parallelism. * Due to binary backwards compatibility, this cannot be altered. Use the * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism. */ public <T> DataStream<T> apply(FlatJoinFunction<T1, T2, T> function, TypeInformation<T> resultType) { //clean the closure function = input1.getExecutionEnvironment().clean(function); coGroupedWindowedStream = input1.coGroup(input2) .where(keySelector1) .equalTo(keySelector2) .window(windowAssigner) .trigger(trigger) .evictor(evictor) .allowedLateness(allowedLateness); return coGroupedWindowedStream .apply(new FlatJoinCoGroupFunction<>(function), resultType); }
/** * Completes the join operation with the user function that is executed * for each combination of elements with the same key in a window. * * <p>Note: This method's return type does not support setting an operator-specific parallelism. * Due to binary backwards compatibility, this cannot be altered. Use the * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism. */ public <T> DataStream<T> apply(JoinFunction<T1, T2, T> function, TypeInformation<T> resultType) { //clean the closure function = input1.getExecutionEnvironment().clean(function); coGroupedWindowedStream = input1.coGroup(input2) .where(keySelector1) .equalTo(keySelector2) .window(windowAssigner) .trigger(trigger) .evictor(evictor) .allowedLateness(allowedLateness); return coGroupedWindowedStream .apply(new JoinCoGroupFunction<>(function), resultType); }
/** * Completes the join operation with the user function that is executed * for each combination of elements with the same key in a window. * * <p>Note: This method's return type does not support setting an operator-specific parallelism. * Due to binary backwards compatibility, this cannot be altered. Use the * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism. */ public <T> DataStream<T> apply(JoinFunction<T1, T2, T> function, TypeInformation<T> resultType) { //clean the closure function = input1.getExecutionEnvironment().clean(function); coGroupedWindowedStream = input1.coGroup(input2) .where(keySelector1) .equalTo(keySelector2) .window(windowAssigner) .trigger(trigger) .evictor(evictor) .allowedLateness(allowedLateness); return coGroupedWindowedStream .apply(new JoinCoGroupFunction<>(function), resultType); }
/** * Completes the join operation with the user function that is executed * for each combination of elements with the same key in a window. * * <p>Note: This method's return type does not support setting an operator-specific parallelism. * Due to binary backwards compatibility, this cannot be altered. Use the * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism. */ public <T> DataStream<T> apply(FlatJoinFunction<T1, T2, T> function, TypeInformation<T> resultType) { //clean the closure function = input1.getExecutionEnvironment().clean(function); coGroupedWindowedStream = input1.coGroup(input2) .where(keySelector1) .equalTo(keySelector2) .window(windowAssigner) .trigger(trigger) .evictor(evictor) .allowedLateness(allowedLateness); return coGroupedWindowedStream .apply(new FlatJoinCoGroupFunction<>(function), resultType); }
/** * Completes the join operation with the user function that is executed * for each combination of elements with the same key in a window. * * <p>Note: This method's return type does not support setting an operator-specific parallelism. * Due to binary backwards compatibility, this cannot be altered. Use the * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism. */ public <T> DataStream<T> apply(FlatJoinFunction<T1, T2, T> function, TypeInformation<T> resultType) { //clean the closure function = input1.getExecutionEnvironment().clean(function); return input1.coGroup(input2) .where(keySelector1) .equalTo(keySelector2) .window(windowAssigner) .trigger(trigger) .evictor(evictor) .apply(new FlatJoinCoGroupFunction<>(function), resultType); }
/** * Completes the join operation with the user function that is executed * for each combination of elements with the same key in a window. * * <p>Note: This method's return type does not support setting an operator-specific parallelism. * Due to binary backwards compatibility, this cannot be altered. Use the * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism. */ public <T> DataStream<T> apply(JoinFunction<T1, T2, T> function, TypeInformation<T> resultType) { //clean the closure function = input1.getExecutionEnvironment().clean(function); return input1.coGroup(input2) .where(keySelector1) .equalTo(keySelector2) .window(windowAssigner) .trigger(trigger) .evictor(evictor) .apply(new JoinCoGroupFunction<>(function), resultType); }
/** * Specifies a {@link KeySelector} for elements from the first input with explicit type information. * * @param keySelector The KeySelector to be used for extracting the first input's key for partitioning. * @param keyType The type information describing the key type. */ public <KEY> Where<KEY> where(KeySelector<T1, KEY> keySelector, TypeInformation<KEY> keyType) { Preconditions.checkNotNull(keySelector); Preconditions.checkNotNull(keyType); return new Where<>(input1.clean(keySelector), keyType); }
/** * Specifies a {@link KeySelector} for elements from the first input. */ public <KEY> Where<KEY> where(KeySelector<T1, KEY> keySelector) { TypeInformation<KEY> keyType = TypeExtractor.getKeySelectorTypes(keySelector, input1.getType()); return new Where<>(input1.clean(keySelector), keyType); }
/** * Specifies a {@link KeySelector} for elements from the first input with explicit type information. * * @param keySelector The KeySelector to be used for extracting the first input's key for partitioning. * @param keyType The type information describing the key type. */ public <KEY> Where<KEY> where(KeySelector<T1, KEY> keySelector, TypeInformation<KEY> keyType) { Preconditions.checkNotNull(keySelector); Preconditions.checkNotNull(keyType); return new Where<>(input1.clean(keySelector), keyType); }
source.coGroup(source).where(new TestKeySelector<>()).equalTo(new TestKeySelector<>()); fail(); } catch (Exception ignored) {} .returns(BasicTypeInfo.INT_TYPE_INFO).print(); source.connect(source).keyBy(new TestKeySelector<>(), new TestKeySelector<>(), Types.STRING); source.coGroup(source).where(new TestKeySelector<>(), Types.STRING).equalTo(new TestKeySelector<>(), Types.STRING); source.join(source).where(new TestKeySelector<>(), Types.STRING).equalTo(new TestKeySelector<>(), Types.STRING); source.keyBy((in) -> in)
.equalTo(new Tuple2KeyExtractor()) .window(TumblingEventTimeWindows.of(Time.of(3, TimeUnit.MILLISECONDS))) .apply(new CoGroupFunction<Tuple2<String, Integer>, Tuple2<String, Integer>, String>() {
/** * Completes the join operation with the user function that is executed * for each combination of elements with the same key in a window. * * <p>Note: This method's return type does not support setting an operator-specific parallelism. * Due to binary backwards compatibility, this cannot be altered. Use the * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism. */ public <T> DataStream<T> apply(FlatJoinFunction<T1, T2, T> function, TypeInformation<T> resultType) { //clean the closure function = input1.getExecutionEnvironment().clean(function); coGroupedWindowedStream = input1.coGroup(input2) .where(keySelector1) .equalTo(keySelector2) .window(windowAssigner) .trigger(trigger) .evictor(evictor) .allowedLateness(allowedLateness); return coGroupedWindowedStream .apply(new FlatJoinCoGroupFunction<>(function), resultType); }
/** * Completes the join operation with the user function that is executed * for each combination of elements with the same key in a window. * * <p>Note: This method's return type does not support setting an operator-specific parallelism. * Due to binary backwards compatibility, this cannot be altered. Use the * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism. */ public <T> DataStream<T> apply(JoinFunction<T1, T2, T> function, TypeInformation<T> resultType) { //clean the closure function = input1.getExecutionEnvironment().clean(function); coGroupedWindowedStream = input1.coGroup(input2) .where(keySelector1) .equalTo(keySelector2) .window(windowAssigner) .trigger(trigger) .evictor(evictor) .allowedLateness(allowedLateness); return coGroupedWindowedStream .apply(new JoinCoGroupFunction<>(function), resultType); }
@Test public void testSetAllowedLateness() { Time lateness = Time.milliseconds(42L); CoGroupedStreams.WithWindow<String, String, String, TimeWindow> withLateness = dataStream1 .coGroup(dataStream2) .where(keySelector) .equalTo(keySelector) .window(tsAssigner) .allowedLateness(lateness); Assert.assertEquals(lateness.toMilliseconds(), withLateness.getAllowedLateness().toMilliseconds()); }
@Test public void testDelegateToCoGrouped() { Time lateness = Time.milliseconds(42L); CoGroupedStreams.WithWindow<String, String, String, TimeWindow> withLateness = dataStream1 .coGroup(dataStream2) .where(keySelector) .equalTo(keySelector) .window(tsAssigner) .allowedLateness(lateness); withLateness.apply(coGroupFunction, BasicTypeInfo.STRING_TYPE_INFO); Assert.assertEquals(lateness.toMilliseconds(), withLateness.getWindowedStream().getAllowedLateness()); }
/** * Specifies a {@link KeySelector} for elements from the first input with explicit type information. * * @param keySelector The KeySelector to be used for extracting the first input's key for partitioning. * @param keyType The type information describing the key type. */ public <KEY> Where<KEY> where(KeySelector<T1, KEY> keySelector, TypeInformation<KEY> keyType) { Preconditions.checkNotNull(keySelector); Preconditions.checkNotNull(keyType); return new Where<>(input1.clean(keySelector), keyType); }