private Schema buildSchema(List<ColumnMetadata> columns, Map<String, Object> tableProperties) { List<ColumnSchema> kuduColumns = columns.stream() .map(this::toColumnSchema) .collect(ImmutableList.toImmutableList()); return new Schema(kuduColumns); }
.build()); schema = new Schema(columns);
private void createAndFillSchemasTable(KuduClient client) throws KuduException { List<String> existingSchemaNames = listSchemaNamesFromTablets(client); ColumnSchema schemaColumnSchema = new ColumnSchema.ColumnSchemaBuilder("schema", Type.STRING) .key(true).build(); Schema schema = new Schema(ImmutableList.of(schemaColumnSchema)); CreateTableOptions options = new CreateTableOptions(); options.addHashPartitions(ImmutableList.of(schemaColumnSchema.getName()), 2); KuduTable schemasTable = client.createTable(rawSchemasTableName, schema, options); KuduSession session = client.newSession(); try { session.setFlushMode(SessionConfiguration.FlushMode.AUTO_FLUSH_BACKGROUND); for (String schemaName : existingSchemaNames) { Insert insert = schemasTable.newInsert(); insert.getRow().addString(0, schemaName); session.apply(insert); } } finally { session.close(); } }
@Override public void updateSchema(VectorAccessible batch) throws IOException { BatchSchema schema = batch.getSchema(); int i = 0; try { if (!checkForTable(name)) { List<ColumnSchema> columns = new ArrayList<>(); for (MaterializedField f : schema) { columns.add(new ColumnSchema.ColumnSchemaBuilder(f.getName(), getType(f.getType())) .nullable(f.getType().getMode() == DataMode.OPTIONAL) .key(i == 0).build()); i++; } Schema kuduSchema = new Schema(columns); table = client.createTable(name, kuduSchema, new CreateTableOptions()); } } catch (Exception e) { throw new IOException(e); } }
columns.add(new ColumnSchema.ColumnSchemaBuilder("string", Type.STRING).nullable(true).build()); Schema schema = new Schema(columns);
/** * Get a schema that only contains the columns which are part of the key * @return new schema with only the keys */ public Schema getRowKeyProjection() { return new Schema(primaryKeyColumns); }
Schema schema = new Schema(columns); try
public Schema getSchema() { if(hasNotColumns()) return null; List<ColumnSchema> schemaColumns = new ArrayList<>(); for(KuduColumnInfo column : columns){ schemaColumns.add(column.columnSchema()); } return new Schema(schemaColumns); }
private Schema buildSchema(List<ColumnMetadata> columns, Map<String, Object> properties) { Optional<Map<String, ColumnDesign>> optColumnDesign = KuduTableProperties.getColumnDesign(properties); Map<String, ColumnDesign> columnDesignMap = optColumnDesign.orElse(ImmutableMap.of()); List<ColumnSchema> kuduColumns = columns.stream() .map(columnMetadata -> toColumnSchema(columnMetadata, columnDesignMap)) .collect(ImmutableList.toImmutableList()); return new Schema(kuduColumns); }
public static Schema pbToSchema(Common.SchemaPB schema) { List<ColumnSchema> columns = new ArrayList<>(schema.getColumnsCount()); List<Integer> columnIds = new ArrayList<>(schema.getColumnsCount()); for (Common.ColumnSchemaPB columnPb : schema.getColumnsList()) { columns.add(pbToColumnSchema(columnPb)); int id = columnPb.getId(); if (id < 0) { throw new IllegalArgumentException("Illegal column ID: " + id); } columnIds.add(id); } return new Schema(columns, columnIds); }
private static Schema getSchema() { ArrayList<ColumnSchema> columns = new ArrayList<ColumnSchema>(1); columns.add(new ColumnSchema.ColumnSchemaBuilder("key", STRING) .key(true) .build()); return new Schema(columns); }
public static Schema createSchemaWithTimestampColumns() { ArrayList<ColumnSchema> columns = new ArrayList<ColumnSchema>(); columns.add(new ColumnSchema.ColumnSchemaBuilder("key", Type.UNIXTIME_MICROS).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c1", Type.UNIXTIME_MICROS).nullable(true).build()); return new Schema(columns); }
public static Schema createSchemaWithBinaryColumns() { ArrayList<ColumnSchema> columns = new ArrayList<ColumnSchema>(); columns.add(new ColumnSchema.ColumnSchemaBuilder("key", Type.BINARY).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c1", Type.STRING).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c2", Type.DOUBLE).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c3", Type.BINARY).nullable(true).build()); return new Schema(columns); }
private static Schema createSchema() { ArrayList<ColumnSchema> columns = new ArrayList<>(3); columns.add(new ColumnSchema.ColumnSchemaBuilder("a", Type.STRING).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("b", Type.STRING).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c", Type.STRING).key(true).build()); return new Schema(columns); }
private Schema createManyStringsSchema() { ArrayList<ColumnSchema> columns = new ArrayList<ColumnSchema>(4); columns.add(new ColumnSchema.ColumnSchemaBuilder("c0", Type.STRING).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c1", Type.STRING).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c2", Type.STRING).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c3", Type.STRING).nullable(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c4", Type.STRING).nullable(true).build()); return new Schema(columns); }
public static Schema createManyStringsSchema() { ArrayList<ColumnSchema> columns = new ArrayList<ColumnSchema>(4); columns.add(new ColumnSchema.ColumnSchemaBuilder("key", Type.STRING).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c1", Type.STRING).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c2", Type.STRING).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c3", Type.STRING).nullable(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c4", Type.STRING).nullable(true).build()); return new Schema(columns); }
public static Schema getHeadsTableSchema() { return new Schema(ImmutableList.of( new ColumnSchema.ColumnSchemaBuilder(COLUMN_KEY_ONE, Type.INT64).key(true).build(), new ColumnSchema.ColumnSchemaBuilder(COLUMN_KEY_TWO, Type.INT64).key(true).build() )); }
private Schema createTableSchema(Type type) { ColumnSchema key = new ColumnSchema.ColumnSchemaBuilder("key", Type.INT64).key(true).build(); ColumnSchema val = new ColumnSchema.ColumnSchemaBuilder("value", type).nullable(true).build(); return new Schema(ImmutableList.of(key, val)); }
private Schema createAllTypesKeySchema() { ArrayList<ColumnSchema> columns = new ArrayList<ColumnSchema>(7); columns.add(new ColumnSchema.ColumnSchemaBuilder("c0", Type.INT8).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c1", Type.INT16).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c2", Type.INT32).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c3", Type.INT64).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c4", Type.UNIXTIME_MICROS).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c5", Type.STRING).key(true).build()); columns.add(new ColumnSchema.ColumnSchemaBuilder("c6", Type.BINARY).key(true).build()); return new Schema(columns); }
public static Schema getTableSchema() { return new Schema(ImmutableList.of( new ColumnSchema.ColumnSchemaBuilder(COLUMN_KEY_ONE, Type.INT64).key(true).build(), new ColumnSchema.ColumnSchemaBuilder(COLUMN_KEY_TWO, Type.INT64).key(true).build(), new ColumnSchema.ColumnSchemaBuilder(COLUMN_PREV_ONE, Type.INT64).nullable(true).build(), new ColumnSchema.ColumnSchemaBuilder(COLUMN_PREV_TWO, Type.INT64).nullable(true).build(), new ColumnSchema.ColumnSchemaBuilder(COLUMN_ROW_ID, Type.INT64).build(), new ColumnSchema.ColumnSchemaBuilder(COLUMN_CLIENT, Type.STRING).build(), new ColumnSchema.ColumnSchemaBuilder(COLUMN_UPDATE_COUNT, Type.INT32).build() )); }