Public

getPartition

getEmbeddedFileSet

dropPartition

Return a partition output for a specific partition key, in preparation for
creating a new partition.

getPartitionOutput

getPartitions

Add a partition for a given partition key, stored at a given path (relative to
the file set's base p

addPartition

consumePartitions

getPartitioning

Adds a set of new metadata entries for a particular partition. Note that
existing entries cannot be 

addMetadata

Asynchronous operation to concatenate the partition in Hive. Note that Hive only
supports certain fo

concatenatePartition

Removes a set of metadata entries for a particular partition. If any metadata
key does not exist, no

removeMetadata

Sets metadata entries for a particular partition. If the metadata entry key does
not already exist, 

setMetadata

ArrayList

Making http requests using okhttp

java.awt.Menu

An URLConnection for HTTP (RFC 2616 [http://tools.ietf.org/html/rfc2616]) used
to send and receive d

java.net.HttpURLConnection

 Calendar is an abstract base class for converting between a Date object and a
set of integer fields

java.util.Calendar

A List is a collection which maintains an ordering for its elements. Every
element in the List has a

java.util.List

Factory and utility methods for Executor, ExecutorService,
ScheduledExecutorService, ThreadFactory, 

java.util.concurrent.Executors

A counting semaphore. Conceptually, a semaphore maintains a set of permits. Each
#acquire blocks if 

java.util.concurrent.Semaphore

getExternalFilesDir

onRequestPermissionsResult

Returns a stream for the resource with the specified name. See
#getResource(String) for a descriptio

getResourceAsStream

/**
   * 1. Write 100 small files (orc format) to a Partition of a PartitionedFileSet.
   * 2. Execute a partition concatenate operation.
   * 3. As compared to before the concatenate operation, validate that the number of files is reduced, while
   *    the contents of the files remains the same.
   */

// Properties for Explore (to create a partitioned Hive table)

"org.apache.hadoop.hive.ql.io.orc.OrcSerde"

"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"

"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"

// 1. create 100 small files in the input FileSet

// this is a timestamp before concatenating, but after writing the files

// each input file will result in one output file, due to the FileInputFormat class and FileOutputFormat class

// all the files should have a lastModified smaller than now

// 3. check that the data files' lastModified timestamp is updated, and there should be fewer of them

// should have a lastModified larger than now

// even though the files were concatenated, the explore results should be unchanged

How to use concatenatePartitionmethodin co.cask.cdap.api.dataset.lib.PartitionedFileSet

Best Java code snippets using co.cask.cdap.api.dataset.lib.PartitionedFileSet.concatenatePartition (Showing top 1 results out of 315)

How to use
concatenatePartition
method
in
co.cask.cdap.api.dataset.lib.PartitionedFileSet