KafkaUtils

Value Members

1. final def !=(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

2. final def !=(arg0: Any): Boolean

   Definition Classes

   Any

3. final def ##(): Int

   Definition Classes

   AnyRef → Any

4. final def ==(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

5. final def ==(arg0: Any): Boolean

   Definition Classes

   Any

6. final def asInstanceOf[T0]: T0

   Definition Classes

   Any

7. def clone(): AnyRef

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

8. def createStream[K, V, U <: Decoder[_], T <: Decoder[_]](jssc: JavaStreamingContext, keyTypeClass: Class[K], valueTypeClass: Class[V], keyDecoderClass: Class[U], valueDecoderClass: Class[T], kafkaParams: Map[String, String], topics: Map[String, Integer], storageLevel: StorageLevel): JavaPairReceiverInputDStream[K, V]

   Create an input stream that pulls messages form a Kafka Broker.

   Create an input stream that pulls messages form a Kafka Broker.

   jssc

   JavaStreamingContext object

   keyTypeClass

   Key type of RDD

   valueTypeClass

   value type of RDD

   keyDecoderClass

   Type of kafka key decoder

   valueDecoderClass

   Type of kafka value decoder

   kafkaParams

   Map of kafka configuration parameters, see http://kafka.apache.org/08/configuration.html

   topics

   Map of (topic_name -> numPartitions) to consume. Each partition is consumed in its own thread

   storageLevel

   RDD storage level.

9. def createStream(jssc: JavaStreamingContext, zkQuorum: String, groupId: String, topics: Map[String, Integer], storageLevel: StorageLevel): JavaPairReceiverInputDStream[String, String]

   Create an input stream that pulls messages form a Kafka Broker.

   Create an input stream that pulls messages form a Kafka Broker.

   jssc

   JavaStreamingContext object

   zkQuorum

   Zookeeper quorum (hostname:port,hostname:port,..).

   groupId

   The group id for this consumer.

   topics

   Map of (topic_name -> numPartitions) to consume. Each partition is consumed in its own thread.

   storageLevel

   RDD storage level.

10. def createStream(jssc: JavaStreamingContext, zkQuorum: String, groupId: String, topics: Map[String, Integer]): JavaPairReceiverInputDStream[String, String]

    Create an input stream that pulls messages form a Kafka Broker.

    Create an input stream that pulls messages form a Kafka Broker. Storage level of the data will be the default StorageLevel.MEMORY_AND_DISK_SER_2.

    jssc

    JavaStreamingContext object

    zkQuorum

    Zookeeper quorum (hostname:port,hostname:port,..)

    groupId

    The group id for this consumer

    topics

    Map of (topic_name -> numPartitions) to consume. Each partition is consumed in its own thread

11. def createStream[K, V, U <: Decoder[_], T <: Decoder[_]](ssc: StreamingContext, kafkaParams: Map[String, String], topics: Map[String, Int], storageLevel: StorageLevel)(implicit arg0: ClassTag[K], arg1: ClassTag[V], arg2: ClassTag[U], arg3: ClassTag[T]): ReceiverInputDStream[(K, V)]

    Create an input stream that pulls messages from a Kafka Broker.

    Create an input stream that pulls messages from a Kafka Broker.

    ssc

    StreamingContext object

    kafkaParams

    Map of kafka configuration parameters, see http://kafka.apache.org/08/configuration.html

    topics

    Map of (topic_name -> numPartitions) to consume. Each partition is consumed in its own thread.

    storageLevel

    Storage level to use for storing the received objects

12. def createStream(ssc: StreamingContext, zkQuorum: String, groupId: String, topics: Map[String, Int], storageLevel: StorageLevel = StorageLevel.MEMORY_AND_DISK_SER_2): ReceiverInputDStream[(String, String)]

    Create an input stream that pulls messages from a Kafka Broker.

    Create an input stream that pulls messages from a Kafka Broker.

    ssc

    StreamingContext object

    zkQuorum

    Zookeeper quorum (hostname:port,hostname:port,..)

    groupId

    The group id for this consumer

    topics

    Map of (topic_name -> numPartitions) to consume. Each partition is consumed in its own thread

    storageLevel

    Storage level to use for storing the received objects (default: StorageLevel.MEMORY_AND_DISK_SER_2)

13. final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

14. def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

15. def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

16. final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

17. def hashCode(): Int

    Definition Classes

    AnyRef → Any

18. final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

19. final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

20. final def notify(): Unit

    Definition Classes

    AnyRef

21. final def notifyAll(): Unit

    Definition Classes

    AnyRef

22. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

23. def toString(): String

    Definition Classes

    AnyRef → Any

24. final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

25. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

26. final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )