Codota Logo
StreamExecutionEnvironment.getCachedFiles
Code IndexAdd Codota to your IDE (free)

How to use
getCachedFiles
method
in
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

Best Java code snippets using org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getCachedFiles (Showing top 4 results out of 315)

  • Common ways to obtain StreamExecutionEnvironment
private void myMethod () {
StreamExecutionEnvironment s =
  • Codota IconStreamExecutionEnvironment.getExecutionEnvironment()
  • Codota IconStreamExecutionEnvironment.createLocalEnvironment()
  • Smart code suggestions by Codota
}
origin: apache/flink

private JobGraph createJobGraph() {
  // make sure that all vertices start immediately
  jobGraph.setScheduleMode(ScheduleMode.EAGER);
  // Generate deterministic hashes for the nodes in order to identify them across
  // submission iff they didn't change.
  Map<Integer, byte[]> hashes = defaultStreamGraphHasher.traverseStreamGraphAndGenerateHashes(streamGraph);
  // Generate legacy version hashes for backwards compatibility
  List<Map<Integer, byte[]>> legacyHashes = new ArrayList<>(legacyStreamGraphHashers.size());
  for (StreamGraphHasher hasher : legacyStreamGraphHashers) {
    legacyHashes.add(hasher.traverseStreamGraphAndGenerateHashes(streamGraph));
  }
  Map<Integer, List<Tuple2<byte[], byte[]>>> chainedOperatorHashes = new HashMap<>();
  setChaining(hashes, legacyHashes, chainedOperatorHashes);
  setPhysicalEdges();
  setSlotSharingAndCoLocation();
  configureCheckpointing();
  JobGraphGenerator.addUserArtifactEntries(streamGraph.getEnvironment().getCachedFiles(), jobGraph);
  // set the ExecutionConfig last when it has been finalized
  try {
    jobGraph.setExecutionConfig(streamGraph.getExecutionConfig());
  }
  catch (IOException e) {
    throw new IllegalConfigurationException("Could not serialize the ExecutionConfig." +
        "This indicates that non-serializable types (like custom serializers) were registered");
  }
  return jobGraph;
}
origin: org.apache.flink/flink-streaming-java_2.11

private JobGraph createJobGraph() {
  // make sure that all vertices start immediately
  jobGraph.setScheduleMode(ScheduleMode.EAGER);
  // Generate deterministic hashes for the nodes in order to identify them across
  // submission iff they didn't change.
  Map<Integer, byte[]> hashes = defaultStreamGraphHasher.traverseStreamGraphAndGenerateHashes(streamGraph);
  // Generate legacy version hashes for backwards compatibility
  List<Map<Integer, byte[]>> legacyHashes = new ArrayList<>(legacyStreamGraphHashers.size());
  for (StreamGraphHasher hasher : legacyStreamGraphHashers) {
    legacyHashes.add(hasher.traverseStreamGraphAndGenerateHashes(streamGraph));
  }
  Map<Integer, List<Tuple2<byte[], byte[]>>> chainedOperatorHashes = new HashMap<>();
  setChaining(hashes, legacyHashes, chainedOperatorHashes);
  setPhysicalEdges();
  setSlotSharingAndCoLocation();
  configureCheckpointing();
  JobGraphGenerator.addUserArtifactEntries(streamGraph.getEnvironment().getCachedFiles(), jobGraph);
  // set the ExecutionConfig last when it has been finalized
  try {
    jobGraph.setExecutionConfig(streamGraph.getExecutionConfig());
  }
  catch (IOException e) {
    throw new IllegalConfigurationException("Could not serialize the ExecutionConfig." +
        "This indicates that non-serializable types (like custom serializers) were registered");
  }
  return jobGraph;
}
origin: org.apache.flink/flink-streaming-java

private JobGraph createJobGraph() {
  // make sure that all vertices start immediately
  jobGraph.setScheduleMode(ScheduleMode.EAGER);
  // Generate deterministic hashes for the nodes in order to identify them across
  // submission iff they didn't change.
  Map<Integer, byte[]> hashes = defaultStreamGraphHasher.traverseStreamGraphAndGenerateHashes(streamGraph);
  // Generate legacy version hashes for backwards compatibility
  List<Map<Integer, byte[]>> legacyHashes = new ArrayList<>(legacyStreamGraphHashers.size());
  for (StreamGraphHasher hasher : legacyStreamGraphHashers) {
    legacyHashes.add(hasher.traverseStreamGraphAndGenerateHashes(streamGraph));
  }
  Map<Integer, List<Tuple2<byte[], byte[]>>> chainedOperatorHashes = new HashMap<>();
  setChaining(hashes, legacyHashes, chainedOperatorHashes);
  setPhysicalEdges();
  setSlotSharingAndCoLocation();
  configureCheckpointing();
  JobGraphGenerator.addUserArtifactEntries(streamGraph.getEnvironment().getCachedFiles(), jobGraph);
  // set the ExecutionConfig last when it has been finalized
  try {
    jobGraph.setExecutionConfig(streamGraph.getExecutionConfig());
  }
  catch (IOException e) {
    throw new IllegalConfigurationException("Could not serialize the ExecutionConfig." +
        "This indicates that non-serializable types (like custom serializers) were registered");
  }
  return jobGraph;
}
origin: org.apache.flink/flink-streaming-java_2.10

private JobGraph createJobGraph() {
  // make sure that all vertices start immediately
  jobGraph.setScheduleMode(ScheduleMode.EAGER);
  // Generate deterministic hashes for the nodes in order to identify them across
  // submission iff they didn't change.
  Map<Integer, byte[]> hashes = defaultStreamGraphHasher.traverseStreamGraphAndGenerateHashes(streamGraph);
  // Generate legacy version hashes for backwards compatibility
  List<Map<Integer, byte[]>> legacyHashes = new ArrayList<>(legacyStreamGraphHashers.size());
  for (StreamGraphHasher hasher : legacyStreamGraphHashers) {
    legacyHashes.add(hasher.traverseStreamGraphAndGenerateHashes(streamGraph));
  }
  Map<Integer, List<Tuple2<byte[], byte[]>>> chainedOperatorHashes = new HashMap<>();
  setChaining(hashes, legacyHashes, chainedOperatorHashes);
  setPhysicalEdges();
  setSlotSharing();
  configureCheckpointing();
  // add registered cache file into job configuration
  for (Tuple2<String, DistributedCache.DistributedCacheEntry> e : streamGraph.getEnvironment().getCachedFiles()) {
    DistributedCache.writeFileInfoToConfig(e.f0, e.f1, jobGraph.getJobConfiguration());
  }
  // set the ExecutionConfig last when it has been finalized
  try {
    jobGraph.setExecutionConfig(streamGraph.getExecutionConfig());
  }
  catch (IOException e) {
    throw new IllegalConfigurationException("Could not serialize the ExecutionConfig." +
        "This indicates that non-serializable types (like custom serializers) were registered");
  }
  return jobGraph;
}
org.apache.flink.streaming.api.environmentStreamExecutionEnvironmentgetCachedFiles

Javadoc

Get the list of cached files that were registered for distribution among the task managers.

Popular methods of StreamExecutionEnvironment

  • execute
  • getExecutionEnvironment
    Creates an execution environment that represents the context in which the program is currently execu
  • addSource
    Ads a data source with a custom type information thus opening a DataStream. Only in very special cas
  • getConfig
    Gets the config object.
  • enableCheckpointing
    Enables checkpointing for the streaming job. The distributed state of the streaming dataflow will be
  • setStreamTimeCharacteristic
    Sets the time characteristic for all streams create from this environment, e.g., processing time, ev
  • setParallelism
    Sets the parallelism for operations executed through this environment. Setting a parallelism of x he
  • fromElements
    Creates a new data stream that contains the given elements. The elements must all be of the same typ
  • setStateBackend
    Sets the state backend that describes how to store and checkpoint operator state. It defines both wh
  • createLocalEnvironment
    Creates a LocalStreamEnvironment. The local execution environment will run the program in a multi-th
  • fromCollection
    Creates a data stream from the given iterator.Because the iterator will remain unmodified until the
  • getCheckpointConfig
    Gets the checkpoint config, which defines values like checkpoint interval, delay between checkpoints
  • fromCollection,
  • getCheckpointConfig,
  • getParallelism,
  • getStreamGraph,
  • setRestartStrategy,
  • socketTextStream,
  • readTextFile,
  • generateSequence,
  • clean,
  • getStreamTimeCharacteristic

Popular in Java

  • Reactive rest calls using spring rest template
  • compareTo (BigDecimal)
  • scheduleAtFixedRate (Timer)
    Schedules the specified task for repeated fixed-rate execution, beginning after the specified delay.
  • scheduleAtFixedRate (ScheduledExecutorService)
    Creates and executes a periodic action that becomes enabled first after the given initial delay, and
  • SocketException (java.net)
    This SocketException may be thrown during socket creation or setting options, and is the superclass
  • ByteBuffer (java.nio)
    A buffer for bytes. A byte buffer can be created in either one of the following ways: * #allocate(i
  • Time (java.sql)
    Java representation of an SQL TIME value. Provides utilities to format and parse the time's represen
  • NumberFormat (java.text)
    The abstract base class for all number formats. This class provides the interface for formatting and
  • JFrame (javax.swing)
  • Scheduler (org.quartz)
    This is the main interface of a Quartz Scheduler. A Scheduler maintains a registery of org.quartz
Codota Logo
  • Products

    Search for Java codeSearch for JavaScript codeEnterprise
  • IDE Plugins

    IntelliJ IDEAWebStormAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimAtomGoLandRubyMineEmacsJupyter
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogCodota Academy Plugin user guide Terms of usePrivacy policyJava Code IndexJavascript Code Index
Get Codota for your IDE now