/** * Writes to Pub/Sub and adds each record's timestamp to the published messages in an attribute * with the specified name. The value of the attribute will be a number representing the number * of milliseconds since the Unix epoch. For example, if using the Joda time classes, {@link * Instant#Instant(long)} can be used to parse this value. * * <p>If the output from this sink is being read by another Beam pipeline, then {@link * PubsubIO.Read#withTimestampAttribute(String)} can be used to ensure the other source reads * these timestamps from the appropriate attribute. */ public Write<T> withTimestampAttribute(String timestampAttribute) { return toBuilder().setTimestampAttribute(timestampAttribute).build(); }
/** * Writes to Pub/Sub are batched to efficiently send data. The value of the attribute will be a * number representing the number of Pub/Sub messages to queue before sending off the bulk * request. For example, if given 1000 the write sink will wait until 1000 messages have been * received, or the pipeline has finished, whichever is first. * * <p>Pub/Sub has a limitation of 10mb per individual request/batch. This attribute was * requested dynamic to allow larger Pub/Sub messages to be sent using this source. Thus * allowing customizable batches and control of number of events before the 10mb size limit is * hit. */ public Write<T> withMaxBatchSize(int batchSize) { return toBuilder().setMaxBatchSize(batchSize).build(); }
/** * Writes to Pub/Sub, adding each record's unique identifier to the published messages in an * attribute with the specified name. The value of the attribute is an opaque string. * * <p>If the the output from this sink is being read by another Beam pipeline, then {@link * PubsubIO.Read#withIdAttribute(String)} can be used to ensure that* the other source reads * these unique identifiers from the appropriate attribute. */ public Write<T> withIdAttribute(String idAttribute) { return toBuilder().setIdAttribute(idAttribute).build(); }
/** Like {@code topic()} but with a {@link ValueProvider}. */ public Write<T> to(ValueProvider<String> topic) { return toBuilder() .setTopicProvider(NestedValueProvider.of(topic, new TopicTranslator())) .build(); }
/** * Used to write a PubSub message together with PubSub attributes. The user-supplied format * function translates the input type T to a PubsubMessage object, which is used by the sink to * separately set the PubSub message's payload and attributes. */ private Write<T> withFormatFn(SimpleFunction<T, PubsubMessage> formatFn) { return toBuilder().setFormatFn(formatFn).build(); }
/** * Writes to Pub/Sub are limited by 10mb in general. This attribute controls the maximum allowed * bytes to be sent to Pub/Sub in a single batched message. */ public Write<T> withMaxBatchBytesSize(int maxBatchBytesSize) { return toBuilder().setMaxBatchBytesSize(maxBatchBytesSize).build(); }