Packages

package io

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. Protected

Package Members

  1. package dynamic

    IO package for dynamic destinations.

    IO package for dynamic destinations. Import All.

    import com.spotify.scio.io.dynamic._

Type Members

  1. final case class BinaryIO(path: String) extends ScioIO[Array[Byte]] with Product with Serializable

    A ScioIO class for writing raw bytes to files.

    A ScioIO class for writing raw bytes to files. Like TextIO, but without newline delimiters and operating over Array[Byte] instead of String.

    path

    a path to write to.

  2. final case class ClosedTap[T] extends Product with Serializable
  3. final case class CustomIO[T](id: String) extends TestIO[T] with Product with Serializable

    Special version of ScioIO for use with ScioContext.customInput and SCollection.saveAsCustomOutput.

  4. final class EmptyTapOf[A] extends TapT[A]
  5. final case class ReadIO[T](id: String) extends TestIO[T] with Product with Serializable

    Special version of ScioIO for use with SCollection.readAll and SCollection.readAllBytes.

  6. trait ScioIO[T] extends AnyRef

    Base trait for all Read/Write IO classes.

    Base trait for all Read/Write IO classes. Every IO connector must implement this. This trait has two abstract implicit methods #read, #write that need to be implemented in every subtype. Look at the com.spotify.scio.io.TextIO subclass for a reference implementation. IO connectors can choose to override #readTest and #writeTest if custom test logic is necessary.

  7. trait Tap[T] extends Serializable

    Placeholder to an external data set that can either be load into memory as an iterator or opened in a new ScioContext as an SCollection.

  8. class TapNotAvailableException extends Exception

    Exception for when a tap is not available.

  9. final class TapOf[A] extends TapT[A]
  10. sealed trait TapT[A] extends Serializable
  11. trait Taps extends AnyRef

    Utility for managing Future[Tap[T]]s.

  12. trait TestIO[T] extends ScioIO[T]

    Base trait for ScioIO without business logic, for stubbing mock data with JobTest.

  13. final case class TextIO(path: String) extends ScioIO[String] with Product with Serializable
  14. final case class TextTap(path: String) extends Tap[String] with Product with Serializable

    Tap for text files on local file system or GCS.

  15. case class UnsupportedTap[T](msg: String) extends Tap[T] with Product with Serializable

Value Members

  1. object BinaryIO extends Serializable
  2. case object EmptyTap extends Tap[Nothing] with Product with Serializable
  3. object EmptyTapOf extends Serializable
  4. object MaterializeTap extends Serializable
  5. object ScioIO
  6. object TapOf extends Serializable
  7. object TapT extends Serializable
  8. object Taps

    Companion object for Taps.

  9. object TapsSysProps
    Annotations
    @registerSysProps()
  10. object TextIO extends Serializable

Ungrouped