Class

org.apache.spark.streaming.scheduler

BatchInfo

Related Doc: package scheduler

Permalink

case class BatchInfo(batchTime: Time, streamIdToInputInfo: Map[Int, StreamInputInfo], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long], outputOperationInfos: Map[Int, OutputOperationInfo]) extends Product with Serializable

:: DeveloperApi :: Class having information on completed batches.

batchTime

Time of the batch

streamIdToInputInfo

A map of input stream id to its input info

submissionTime

Clock time of when jobs of this batch was submitted to the streaming scheduler queue

processingStartTime

Clock time of when the first job of this batch started processing

processingEndTime

Clock time of when the last job of this batch finished processing

outputOperationInfos

The output operations in this batch

Annotations
@DeveloperApi()
Source
BatchInfo.scala
Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. BatchInfo
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new BatchInfo(batchTime: Time, streamIdToInputInfo: Map[Int, StreamInputInfo], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long], outputOperationInfos: Map[Int, OutputOperationInfo])

    Permalink

    batchTime

    Time of the batch

    streamIdToInputInfo

    A map of input stream id to its input info

    submissionTime

    Clock time of when jobs of this batch was submitted to the streaming scheduler queue

    processingStartTime

    Clock time of when the first job of this batch started processing

    processingEndTime

    Clock time of when the last job of this batch finished processing

    outputOperationInfos

    The output operations in this batch

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. val batchTime: Time

    Permalink

    Time of the batch

  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  10. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  11. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  12. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  13. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  14. def numRecords: Long

    Permalink

    The number of recorders received by the receivers in this batch.

  15. val outputOperationInfos: Map[Int, OutputOperationInfo]

    Permalink

    The output operations in this batch

  16. def processingDelay: Option[Long]

    Permalink

    Time taken for the all jobs of this batch to finish processing from the time they started processing.

    Time taken for the all jobs of this batch to finish processing from the time they started processing. Essentially, it is processingEndTime - processingStartTime.

  17. val processingEndTime: Option[Long]

    Permalink

    Clock time of when the last job of this batch finished processing

  18. val processingStartTime: Option[Long]

    Permalink

    Clock time of when the first job of this batch started processing

  19. def schedulingDelay: Option[Long]

    Permalink

    Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler.

    Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler. Essentially, it is processingStartTime - submissionTime.

  20. val streamIdToInputInfo: Map[Int, StreamInputInfo]

    Permalink

    A map of input stream id to its input info

  21. val submissionTime: Long

    Permalink

    Clock time of when jobs of this batch was submitted to the streaming scheduler queue

  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  23. def totalDelay: Option[Long]

    Permalink

    Time taken for all the jobs of this batch to finish processing from the time they were submitted.

    Time taken for all the jobs of this batch to finish processing from the time they were submitted. Essentially, it is processingDelay + schedulingDelay.

  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped