Codota Logo
FieldAccessDescriptor.resolve
Code IndexAdd Codota to your IDE (free)

How to use
resolve
method
in
org.apache.beam.sdk.schemas.FieldAccessDescriptor

Best Java code snippets using org.apache.beam.sdk.schemas.FieldAccessDescriptor.resolve (Showing top 15 results out of 315)

  • Add the Codota plugin to your IDE and get smart completions
private void myMethod () {
OutputStreamWriter o =
  • Codota IconOutputStream out;new OutputStreamWriter(out)
  • Codota IconOutputStream out;String charsetName;new OutputStreamWriter(out, charsetName)
  • Codota IconHttpURLConnection connection;new OutputStreamWriter(connection.getOutputStream())
  • Smart code suggestions by Codota
}
origin: org.apache.beam/beam-sdks-java-core

private FieldAccessDescriptor resolvedNestedFieldsHelper(
  Field field, FieldAccessDescriptor subDescriptor) {
 return subDescriptor.resolve(getFieldSchema(field));
}
origin: org.apache.beam/beam-sdks-java-core

FieldAggregation(
  FieldAccessDescriptor fieldsToAggregate,
  Field outputField,
  CombineFn<FieldT, AccumT, OutputT> fn,
  TupleTag<Object> combineTag,
  Schema aggregationSchema,
  @Nullable Schema inputSchema) {
 if (inputSchema != null) {
  this.fieldsToAggregate = fieldsToAggregate.resolve(inputSchema);
  this.inputSubSchema = Select.getOutputSchema(inputSchema, this.fieldsToAggregate);
  this.unnestedInputSubSchema = Unnest.getUnnestedSchema(inputSubSchema);
  this.needsUnnesting = !inputSchema.equals(unnestedInputSubSchema);
 } else {
  this.fieldsToAggregate = fieldsToAggregate;
  this.inputSubSchema = null;
  this.unnestedInputSubSchema = null;
  this.needsUnnesting = false;
 }
 this.outputField = outputField;
 this.fn = fn;
 this.combineTag = combineTag;
 this.aggregationSchema = aggregationSchema;
}
origin: org.apache.beam/beam-sdks-java-core

fieldAccessDescriptor.resolve(((SchemaCoder<?>) inputCoder).getSchema());
origin: org.apache.beam/beam-sdks-java-core

  fieldAccessDescriptor.withOrderByFieldInsertionOrder().resolve(schema);
Schema currentKeySchema = Select.getOutputSchema(schema, resolved);
if (keySchema == null) {
origin: org.apache.beam/beam-sdks-java-core

@Override
public PCollection<Row> expand(PCollection<T> input) {
 Schema inputSchema = input.getSchema();
 FieldAccessDescriptor resolved = fieldAccessDescriptor.resolve(inputSchema);
 Schema outputSchema = getOutputSchema(inputSchema, resolved);
 PCollection<Row> selected =
   input
     .apply(
       ParDo.of(
         new DoFn<T, Row>() {
          // TODO: This should be the same as resolved so that Beam knows which fields
          // are being accessed. Currently Beam only supports wildcard descriptors.
          // Once BEAM-4457 is fixed, fix this.
          @FieldAccess("filterFields")
          final FieldAccessDescriptor fieldAccessDescriptor =
            FieldAccessDescriptor.withAllFields();
          @ProcessElement
          public void process(
            @FieldAccess("filterFields") Row row, OutputReceiver<Row> r) {
           r.output(selectRow(row, resolved, inputSchema, outputSchema));
          }
         }))
     .setRowSchema(outputSchema);
 return selected;
}
origin: org.apache.beam/beam-sdks-java-core

@Test
public void testAllFields() {
 FieldAccessDescriptor fieldAccessDescriptor = FieldAccessDescriptor.withAllFields();
 assertTrue(fieldAccessDescriptor.resolve(SIMPLE_SCHEMA).allFields());
}
origin: org.apache.beam/beam-sdks-java-core

@Test
public void testArrayNestedField() {
 FieldAccessDescriptor level1 = FieldAccessDescriptor.withFieldNames("field2");
 FieldAccessDescriptor level2 =
   FieldAccessDescriptor.withFieldNames("field1").withNestedField("field1", level1);
 FieldAccessDescriptor resolved = level2.resolve(NESTED_ARRAY_SCHEMA);
 assertTrue(resolved.fieldIdsAccessed().isEmpty());
 assertEquals(1, resolved.nestedFields().size());
 resolved = resolved.nestedFields().get(1);
 assertEquals(Sets.newHashSet(2), resolved.fieldIdsAccessed());
}
origin: org.apache.beam/beam-sdks-java-core

@Test
public void testNestedFieldById() {
 FieldAccessDescriptor fieldAccessDescriptor =
   FieldAccessDescriptor.withFieldNames("field1")
     .withNestedField(1, FieldAccessDescriptor.withAllFields());
 fieldAccessDescriptor = fieldAccessDescriptor.resolve(NESTED_SCHEMA2);
 assertTrue(fieldAccessDescriptor.fieldIdsAccessed().isEmpty());
 assertEquals(1, fieldAccessDescriptor.nestedFields().size());
 FieldAccessDescriptor nestedAccess = fieldAccessDescriptor.nestedFields().get(1);
 assertTrue(nestedAccess.allFields());
}
origin: org.apache.beam/beam-sdks-java-core

@Test
public void testFieldIds() {
 FieldAccessDescriptor fieldAccessDescriptor =
   FieldAccessDescriptor.withFieldIds(1, 3).resolve(SIMPLE_SCHEMA);
 assertEquals(Sets.newHashSet(1, 3), fieldAccessDescriptor.fieldIdsAccessed());
}
origin: org.apache.beam/beam-sdks-java-core

@Test
public void testFieldNames() {
 FieldAccessDescriptor fieldAccessDescriptor =
   FieldAccessDescriptor.withFieldNames("field0", "field2").resolve(SIMPLE_SCHEMA);
 assertEquals(Sets.newHashSet(0, 2), fieldAccessDescriptor.fieldIdsAccessed());
}
origin: org.apache.beam/beam-sdks-java-core

@Test
public void testNestedFieldByName() {
 FieldAccessDescriptor fieldAccessDescriptor =
   FieldAccessDescriptor.withFieldNames("field1")
     .withNestedField("field1", FieldAccessDescriptor.withAllFields());
 fieldAccessDescriptor = fieldAccessDescriptor.resolve(NESTED_SCHEMA2);
 assertTrue(fieldAccessDescriptor.fieldIdsAccessed().isEmpty());
 assertEquals(1, fieldAccessDescriptor.nestedFields().size());
 FieldAccessDescriptor nestedAccess = fieldAccessDescriptor.nestedFields().get(1);
 assertTrue(nestedAccess.allFields());
}
origin: org.apache.beam/beam-runners-core-java

fieldAccessDescriptor = fieldAccessDescriptor.resolve(schemaCoder.getSchema());
origin: org.apache.beam/beam-sdks-java-core

 @Test
 public void testMapNestedField() {
  FieldAccessDescriptor level1 = FieldAccessDescriptor.withFieldNames("field2");
  FieldAccessDescriptor level2 =
    FieldAccessDescriptor.withFieldNames("field1").withNestedField("field1", level1);

  FieldAccessDescriptor resolved = level2.resolve(NESTED_MAP_SCHEMA);
  assertTrue(resolved.fieldIdsAccessed().isEmpty());
  assertEquals(1, resolved.nestedFields().size());
  resolved = resolved.nestedFields().get(1);
  assertEquals(Sets.newHashSet(2), resolved.fieldIdsAccessed());
 }
}
origin: org.apache.beam/beam-sdks-java-core

 @Override
 public PCollection<KV<Row, Iterable<InputT>>> expand(PCollection<InputT> input) {
  Schema schema = input.getSchema();
  FieldAccessDescriptor resolved = fieldAccessDescriptor.resolve(schema);
  keySchema = Select.getOutputSchema(schema, resolved);
  return input
    .apply(
      "Group by fields",
      ParDo.of(
        new DoFn<InputT, KV<Row, InputT>>() {
         @ProcessElement
         public void process(
           @Element InputT element,
           @Element Row row,
           OutputReceiver<KV<Row, InputT>> o) {
          o.output(KV.of(Select.selectRow(row, resolved, schema, keySchema), element));
         }
        }))
    .setCoder(KvCoder.of(SchemaCoder.of(keySchema), input.getCoder()))
    .apply(GroupByKey.create());
 }
}
origin: org.apache.beam/beam-sdks-java-core

@Test
public void testPartialAccessNestedField() {
 FieldAccessDescriptor level1 = FieldAccessDescriptor.withFieldNames("field2");
 FieldAccessDescriptor level2 =
   FieldAccessDescriptor.withFieldNames("field1").withNestedField("field1", level1);
 FieldAccessDescriptor level3 =
   FieldAccessDescriptor.withFieldNames("field1").withNestedField("field1", level2);
 FieldAccessDescriptor resolved = level3.resolve(NESTED_SCHEMA2);
 assertTrue(resolved.fieldIdsAccessed().isEmpty());
 assertEquals(1, resolved.nestedFields().size());
 resolved = resolved.nestedFields().get(1);
 assertTrue(resolved.fieldIdsAccessed().isEmpty());
 assertEquals(1, resolved.nestedFields().size());
 resolved = resolved.nestedFields().get(1);
 assertEquals(Sets.newHashSet(2), resolved.fieldIdsAccessed());
}
org.apache.beam.sdk.schemasFieldAccessDescriptorresolve

Popular methods of FieldAccessDescriptor

  • withAllFields
  • allFields
  • fieldIdsAccessed
  • nestedFields
  • withFieldIds
    Return a descriptor that access the specified fields.By default, if the field is a nested row (or a
  • withFieldNames
    Return a descriptor that access the specified fields.By default, if the field is a nested row (or a
  • builder
  • create
    Return an empty FieldAccessDescriptor.
  • getAllFields
  • getFieldIdsAccessed
  • getFieldInsertionOrder
  • getFieldNamesAccessed
  • getFieldInsertionOrder,
  • getFieldNamesAccessed,
  • getFieldSchema,
  • getNestedFieldsAccessedById,
  • getNestedFieldsAccessedByName,
  • resolveFieldIdsAccessed,
  • resolveNestedFieldsAccessed,
  • resolvedNestedFieldsHelper,
  • toBuilder

Popular in Java

  • Updating database using SQL prepared statement
  • setScale (BigDecimal)
  • getSupportFragmentManager (FragmentActivity)
    Return the FragmentManager for interacting with fragments associated with this activity.
  • scheduleAtFixedRate (ScheduledExecutorService)
    Creates and executes a periodic action that becomes enabled first after the given initial delay, and
  • IOException (java.io)
    Signals that an I/O exception of some sort has occurred. This class is the general class of exceptio
  • Timestamp (java.sql)
    A Java representation of the SQL TIMESTAMP type. It provides the capability of representing the SQL
  • UUID (java.util)
    UUID is an immutable representation of a 128-bit universally unique identifier (UUID). There are mul
  • DataSource (javax.sql)
    A factory for connections to the physical data source that this DataSource object represents. An alt
  • JOptionPane (javax.swing)
  • Scheduler (org.quartz)
    This is the main interface of a Quartz Scheduler. A Scheduler maintains a registery of org.quartz
Codota Logo
  • Products

    Search for Java codeSearch for JavaScript codeEnterprise
  • IDE Plugins

    IntelliJ IDEAWebStormAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimAtomGoLandRubyMineEmacsJupyter
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogCodota Academy Plugin user guide Terms of usePrivacy policyJava Code IndexJavascript Code Index
Get Codota for your IDE now