org.apache.spark.streaming.dstream

Class ReceiverInputDStream<T>

Object

org.apache.spark.streaming.dstream.DStream<T>

org.apache.spark.streaming.dstream.InputDStream<T>

org.apache.spark.streaming.dstream.ReceiverInputDStream<T>

All Implemented Interfaces:

java.io.Serializable, Logging

Direct Known Subclasses:

FlumeInputDStream, FlumePollingInputDStream, KafkaInputDStream, MQTTInputDStream, PluggableInputDStream, RawInputDStream, SocketInputDStream, TwitterInputDStream

public abstract class ReceiverInputDStream<T>
extends InputDStream<T>

Abstract class for defining any InputDStream that has to start a receiver on worker nodes to receive external data. Specific implementations of ReceiverInputDStream must define the getReceiver() function that gets the receiver object of type Receiver that will be sent to the workers to receive data.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ReceiverInputDStream(StreamingContext ssc_, scala.reflect.ClassTag<T> evidence$1)

Method Summary

Methods

Modifier and Type	Method and Description
scala.Option<RDD<T>>	compute(Time validTime) Generates RDDs with blocks received by the receiver of this stream.
abstract Receiver<T>	getReceiver() Gets the receiver object that will be sent to the worker nodes to receive data.
int	id() This is an unique identifier for the receiver input stream.
void	start() Method called to start receiving data.
void	stop() Method called to stop receiving data.

Methods inherited from class org.apache.spark.streaming.dstream.InputDStream

dependencies, isTimeValid, lastValidTime, slideDuration

Methods inherited from class org.apache.spark.streaming.dstream.DStream

cache, checkpoint, checkpointData, checkpointDuration, clearCheckpointData, clearMetadata, context, count, countByValue, countByValueAndWindow, countByWindow, creationSite, filter, flatMap, foreach, foreach, foreachRDD, foreachRDD, generatedRDDs, generateJob, getCreationSite, getOrCompute, glom, graph, initialize, isInitialized, map, mapPartitions, mustCheckpoint, parentRememberDuration, persist, persist, print, print, reduce, reduceByWindow, reduceByWindow, register, remember, rememberDuration, repartition, restoreCheckpointData, saveAsObjectFiles, saveAsTextFiles, setContext, setGraph, slice, slice, ssc, storageLevel, toPairDStreamFunctions, transform, transform, transformWith, transformWith, union, updateCheckpointData, validate, window, window, zeroTime

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

ReceiverInputDStream

public ReceiverInputDStream(StreamingContext ssc_,
                    scala.reflect.ClassTag<T> evidence$1)

Method Detail

id

public int id()

This is an unique identifier for the receiver input stream.

getReceiver

public abstract Receiver<T> getReceiver()

Gets the receiver object that will be sent to the worker nodes to receive data. This method needs to defined by any specific implementation of a ReceiverInputDStream.

start

public void start()

Description copied from class: InputDStream

Method called to start receiving data. Subclasses must implement this method.

Specified by:

start in class InputDStream<T>

stop

public void stop()

Description copied from class: InputDStream

Method called to stop receiving data. Subclasses must implement this method.

Specified by:

stop in class InputDStream<T>

compute

public scala.Option<RDD<T>> compute(Time validTime)

Generates RDDs with blocks received by the receiver of this stream.

Specified by:

compute in class DStream<T>