@Override public Builder setCompression(boolean enabled, @Nullable String compressionCodec, @Nullable Map<String, String> codecConf) { outputBuilder.setCompression(enabled, compressionCodec, codecConf); inputBuilder.setCompression(enabled, compressionCodec, codecConf); return this; }
/** * Create a configuration builder for {@link org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput} * * @param keyClassName the key class name * @param valueClassName the value class name * @param partitionerClassName the partitioner class name * @param partitionerConf the partitioner configuration. This can be null, and is a {@link * java.util.Map} of key-value pairs. The keys should be limited to * the ones required by the partitioner. */ @InterfaceAudience.Private Builder(String keyClassName, String valueClassName, String partitionerClassName, @Nullable Map<String, String> partitionerConf) { this(); Preconditions.checkNotNull(keyClassName, "Key class name cannot be null"); Preconditions.checkNotNull(valueClassName, "Value class name cannot be null"); Preconditions.checkNotNull(partitionerClassName, "Partitioner class name cannot be null"); setKeyClassName(keyClassName); setValueClassName(valueClassName); setPartitioner(partitionerClassName, partitionerConf); }
/** * Set serialization class and the relevant comparator to be used for sorting. * Providing custom serialization class could change the way, keys needs to be compared in * sorting. Providing invalid comparator here could create invalid results. * * @param serializationClassName * @param comparatorClassName * @param serializerConf the serializer configuration. This can be null, and is a * {@link java.util.Map} of key-value pairs. The keys should be limited * to the ones required by the comparator. * @return this object for further chained method calls */ public Builder setKeySerializationClass(String serializationClassName, String comparatorClassName, @Nullable Map<String, String> serializerConf) { Preconditions.checkArgument(serializationClassName != null, "serializationClassName cannot be null"); Preconditions.checkArgument(comparatorClassName != null, "comparator cannot be null"); this.conf.set(CommonConfigurationKeys.IO_SERIALIZATIONS_KEY, serializationClassName + "," + conf.get(CommonConfigurationKeys.IO_SERIALIZATIONS_KEY)); setKeyComparatorClass(comparatorClassName, null); if (serializerConf != null) { // Merging the confs for now. Change to be specific in the future. ConfigUtils.mergeConfsWithExclusions(this.conf, serializerConf, TezRuntimeConfiguration.getRuntimeConfigKeySet()); } return this; }
/** * Set serialization class and the relevant comparator to be used for sorting. * Providing custom serialization class could change the way, keys needs to be compared in * sorting. Providing invalid comparator here could create invalid results. * * @param serializationClassName * @param comparatorClassName * @param serializerConf the serializer configuration. This can be null, and is a * {@link java.util.Map} of key-value pairs. The keys should be limited * to the ones required by the comparator. * @return this object for further chained method calls */ public Builder setKeySerializationClass(String serializationClassName, String comparatorClassName, @Nullable Map<String, String> serializerConf) { outputBuilder.setKeySerializationClass(serializationClassName, comparatorClassName, serializerConf); inputBuilder.setKeySerializationClass(serializationClassName, comparatorClassName, serializerConf); return this; }
/** * Set the key comparator class and it's associated configuration. This method should only be * used if the comparator requires some specific configuration, which is typically not the * case. {@link #setKeyComparatorClass(String)} is the preferred method for setting a * comparator. * * @param comparatorClassName the key comparator class name * @param comparatorConf the comparator configuration. This can be null, and is a {@link * java.util.Map} of key-value pairs. The keys should be limited to * the ones required by the comparator. * @return instance of the current builder */ public Builder setKeyComparatorClass(String comparatorClassName, @Nullable Map<String, String> comparatorConf) { outputBuilder.setKeyComparatorClass(comparatorClassName, comparatorConf); inputBuilder.setKeyComparatorClass(comparatorClassName, comparatorConf); return this; }
@Override public Builder setCombiner(String combinerClassName) { return this.setCombiner(combinerClassName, null); }
public static Builder newBuilder(String keyClass, String valueClass, String partitionerClassName, Map<String, String> partitionerConf) { return new Builder(keyClass, valueClass, partitionerClassName, partitionerConf); }
/** * Set the key comparator class * * @param comparatorClassName the key comparator class name * @return instance of the current builder */ public Builder setKeyComparatorClass(String comparatorClassName) { return this.setKeyComparatorClass(comparatorClassName, null); }
@Override public Builder setAdditionalConfiguration(String key, String value) { outputBuilder.setAdditionalConfiguration(key, value); inputBuilder.setAdditionalConfiguration(key, value); return this; }
@Override public SpecificBuilder<E> setAdditionalConfiguration(Map<String, String> confMap) { builder.setAdditionalConfiguration(confMap); return this; }
@InterfaceAudience.Private Builder(String keyClassName, String valueClassName, String partitionerClassName, Map<String, String> partitionerConf) { outputBuilder.setKeyClassName(keyClassName); outputBuilder.setValueClassName(valueClassName); outputBuilder.setPartitioner(partitionerClassName, partitionerConf); inputBuilder.setKeyClassName(keyClassName); inputBuilder.setValueClassName(valueClassName); }
@Override public SpecificBuilder setFromConfigurationUnfiltered( Configuration conf) { builder.setFromConfigurationUnfiltered(conf); return this; }
@Override public Builder setFromConfigurationUnfiltered(Configuration conf) { outputBuilder.setFromConfigurationUnfiltered(conf); inputBuilder.setFromConfigurationUnfiltered(conf); return this; }
@Override public SpecificBuilder<E> setFromConfiguration(Configuration conf) { builder.setFromConfiguration(conf); return this; }
@Override public Builder setAdditionalConfiguration(Map<String, String> confMap) { outputBuilder.setAdditionalConfiguration(confMap); inputBuilder.setAdditionalConfiguration(confMap); return this; }
@Override /** * Edge config options are derived from client-side tez-site.xml (recommended). * Optionally invoke setFromConfiguration to override these config options via commandline arguments. * * @param conf * @return this object for further chained method calls */ public Builder setFromConfiguration(Configuration conf) { outputBuilder.setFromConfiguration(conf); inputBuilder.setFromConfiguration(conf); return this; }
@Override public SpecificBuilder<E> setAdditionalConfiguration(String key, String value) { builder.setAdditionalConfiguration(key, value); return this; }
@Override public SpecificBuilder<E> setCombiner(String combinerClassName, Map<String, String> combinerConf) { builder.setCombiner(combinerClassName, combinerConf); return this; }
/** * Build and return an instance of the configuration * @return an instance of the acatual configuration */ public OrderedPartitionedKVEdgeConfig build() { return new OrderedPartitionedKVEdgeConfig(outputBuilder.build(), inputBuilder.build()); }