Codota Logo
PartitionedFileSet.dropPartition
Code IndexAdd Codota to your IDE (free)

How to use
dropPartition
method
in
co.cask.cdap.api.dataset.lib.PartitionedFileSet

Best Java code snippets using co.cask.cdap.api.dataset.lib.PartitionedFileSet.dropPartition (Showing top 10 results out of 315)

  • Add the Codota plugin to your IDE and get smart completions
private void myMethod () {
ArrayList a =
  • Codota Iconnew ArrayList<String>()
  • Codota Iconnew ArrayList()
  • Codota Iconnew ArrayList<Object>()
  • Smart code suggestions by Codota
}
origin: caskdata/cdap

 @Override
 public void apply() throws Exception {
  dataset.dropPartition(partitionKey1);
 }
});
origin: co.cask.hydrator/core-plugins

public void deleteMatchingPartitionsByTime(long upperLimit) throws IOException {
 if (upperLimit > 0 && upperLimit < Long.MAX_VALUE) {
  PartitionFilter filter = PartitionFilter.builder().addRangeCondition(SNAPSHOT_FIELD, null, upperLimit).build();
  Set<PartitionDetail> partitions = files.getPartitions(filter);
  for (PartitionDetail partition : partitions) {
   files.dropPartition(partition.getPartitionKey());
  }
 }
}
origin: cdapio/cdap

 @DELETE
 @Path("partitions/{partition}/subpartitions/{sub-partition}")
 public void drop(HttpServiceRequest request, HttpServiceResponder responder,
           @PathParam("partition") String partition, @PathParam("sub-partition") int subPartition) {
  PartitionKey key = PartitionKey.builder()
   .addStringField("partition", partition)
   .addIntField("sub-partition", subPartition)
   .build();
  if (pfs.getPartition(key) == null) {
   responder.sendString(404, "Partition not found.", Charsets.UTF_8);
   return;
  }
  pfs.dropPartition(key);
  responder.sendString(200, "Successfully dropped partition", Charsets.UTF_8);
 }
}
origin: caskdata/cdap

 @Override
 public void apply() throws Exception {
  // drop all existing partitions (2 of which are not consumed)
  for (PartitionDetail partitionDetail : dataset.getPartitions(PartitionFilter.ALWAYS_MATCH)) {
   dataset.dropPartition(partitionDetail.getPartitionKey());
  }
  // add 5 new ones
  for (PartitionKey partitionKey : partitionKeys2) {
   dataset.getPartitionOutput(partitionKey).addPartition();
  }
 }
});
origin: caskdata/cdap

 @Override
 public void apply() throws Exception {
  Location outputLocation = createPartition(pfs, PARTITION_KEY, "file");
  outputLocationRef.set(outputLocation);
  Assert.assertTrue(outputLocation.exists());
  Assert.assertNotNull(pfs.getPartition(PARTITION_KEY));
  Assert.assertTrue(pfs.getPartition(PARTITION_KEY).getLocation().exists());
  pfs.dropPartition(PARTITION_KEY);
  Assert.assertFalse(outputLocation.exists());
  Assert.assertNull(pfs.getPartition(PARTITION_KEY));
  pfs.dropPartition(PARTITION_KEY);
 }
});
origin: caskdata/cdap

 @Override
 public void apply() throws Exception {
  Assert.assertTrue(pfsBaseLocation.exists());
  // attempt to write a new partition - should fail
  try {
   pfs.getPartitionOutput(PARTITION_KEY);
   Assert.fail("External partitioned file set should not allow writing files");
  } catch (UnsupportedOperationException e) {
   // expected
  }
  // create an external file and add it as a partition
  File someFile = new File(absolutePath, "some.file");
  OutputStream out = new FileOutputStream(someFile);
  out.close();
  Assert.assertTrue(someFile.exists());
  pfs.addPartition(PARTITION_KEY, "some.file");
  Assert.assertNotNull(pfs.getPartition(PARTITION_KEY));
  Assert.assertTrue(pfs.getPartition(PARTITION_KEY).getLocation().exists());
  // now drop the partition and validate the file is still there
  pfs.dropPartition(PARTITION_KEY);
  Assert.assertNull(pfs.getPartition(PARTITION_KEY));
  Assert.assertTrue(someFile.exists());
 }
});
origin: cdapio/cdap

pfs.dropPartition(partitionOutput.getPartitionKey());
pfs.dropPartition(partition.getPartitionKey());
origin: caskdata/cdap

@Test
public void testRollbackOfPartitionCreateThenDelete() throws Exception {
 PartitionedFileSet pfs = dsFrameworkUtil.getInstance(pfsInstance);
 TransactionContext txContext = new TransactionContext(txClient, (TransactionAware) pfs);
 txContext.start();
 Assert.assertNull(pfs.getPartition(PARTITION_KEY));
 Location outputLocation = createPartition(pfs, PARTITION_KEY, "file");
 Assert.assertNotNull(pfs.getPartition(PARTITION_KEY));
 pfs.dropPartition(PARTITION_KEY);
 txContext.abort();
 // the file shouldn't exist because the transaction was aborted (AND because it was dropped at the end of the tx)
 Assert.assertFalse(outputLocation.exists());
}
origin: caskdata/cdap

pfs.dropPartition(PARTITION_KEY);
Assert.assertNull(pfs.getPartition(PARTITION_KEY));
Assert.assertFalse(outputLocation.exists());
origin: caskdata/cdap

 pfs.dropPartition(PartitionKey.builder().addField("i", 1).addField("l", 2L).addField("s", 0).build());
 Assert.fail("should have thrown exception due to incompatible field");
} catch (IllegalArgumentException e) {
co.cask.cdap.api.dataset.libPartitionedFileSetdropPartition

Javadoc

Remove a partition for a given partition key, silently ignoring if the key is not found.

Popular methods of PartitionedFileSet

  • getPartition
  • getEmbeddedFileSet
  • getPartitionOutput
    Return a partition output for a specific partition key, in preparation for creating a new partition.
  • getPartitions
  • addPartition
    Add a partition for a given partition key, stored at a given path (relative to the file set's base p
  • consumePartitions
  • getPartitioning
  • addMetadata
    Adds a set of new metadata entries for a particular partition. Note that existing entries cannot be
  • concatenatePartition
    Asynchronous operation to concatenate the partition in Hive. Note that Hive only supports certain fo
  • removeMetadata
    Removes a set of metadata entries for a particular partition. If any metadata key does not exist, no
  • setMetadata
    Sets metadata entries for a particular partition. If the metadata entry key does not already exist,
  • setMetadata

Popular in Java

  • Updating database using SQL prepared statement
  • orElseThrow (Optional)
    Return the contained value, if present, otherwise throw an exception to be created by the provided s
  • findViewById (Activity)
  • startActivity (Activity)
  • PrintWriter (java.io)
    Prints formatted representations of objects to a text-output stream. This class implements all of th
  • ConnectException (java.net)
    A ConnectException is thrown if a connection cannot be established to a remote host on a specific po
  • DateFormat (java.text)
    Formats or parses dates and times.This class provides factories for obtaining instances configured f
  • Stream (java.util.stream)
    A sequence of elements supporting sequential and parallel aggregate operations. The following exampl
  • HttpServletRequest (javax.servlet.http)
    Extends the javax.servlet.ServletRequest interface to provide request information for HTTP servlets.
  • IOUtils (org.apache.commons.io)
    General IO stream manipulation utilities. This class provides static utility methods for input/outpu
Codota Logo
  • Products

    Search for Java codeSearch for JavaScript codeEnterprise
  • IDE Plugins

    IntelliJ IDEAWebStormAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimAtomGoLandRubyMineEmacsJupyter
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogCodota Academy Plugin user guide Terms of usePrivacy policyJava Code IndexJavascript Code Index
Get Codota for your IDE now