package io
- Alphabetic
- Public
- Protected
Package Members
Type Members
- final case class BinaryIO(path: String) extends ScioIO[Array[Byte]] with Product with Serializable
A ScioIO class for writing raw bytes to files.
A ScioIO class for writing raw bytes to files. Like TextIO, but without newline delimiters and operating over Array[Byte] instead of String.
- path
a path to write to.
- final case class ClosedTap[T] extends Product with Serializable
- final case class CustomIO[T](id: String) extends TestIO[T] with Product with Serializable
Special version of ScioIO for use with ScioContext.customInput and SCollection.saveAsCustomOutput.
- final class EmptyTapOf[A] extends TapT[A]
- trait KeyedIO[T] extends AnyRef
- final case class ReadIO[T](id: String) extends TestIO[T] with Product with Serializable
Special version of ScioIO for use with SCollection.readAll and SCollection.readAllBytes.
- trait ScioIO[T] extends AnyRef
Base trait for all Read/Write IO classes.
Base trait for all Read/Write IO classes. Every IO connector must implement this. This trait has two abstract implicit methods #read, #write that need to be implemented in every subtype. Look at the com.spotify.scio.io.TextIO subclass for a reference implementation. IO connectors can choose to override #readTest and #writeTest if custom test logic is necessary.
- trait Tap[T] extends Serializable
Placeholder to an external data set that can either be load into memory as an iterator or opened in a new ScioContext as an SCollection.
- class TapNotAvailableException extends Exception
Exception for when a tap is not available.
- final class TapOf[A] extends TapT[A]
- sealed trait TapT[A] extends Serializable
- trait Taps extends AnyRef
Utility for managing
Future[Tap[T]]s. - trait TestIO[T] extends ScioIO[T]
Base trait for ScioIO without business logic, for stubbing mock data with
JobTest. - final case class TextIO(path: String) extends ScioIO[String] with Product with Serializable
- final case class TextTap(path: String, params: ReadParam) extends Tap[String] with Product with Serializable
Tap for text files on local file system or GCS.
- final case class UnsupportedTap[T](msg: String) extends Tap[T] with Product with Serializable
- trait WriteResultIO[T] extends AnyRef
Value Members
- object BinaryIO extends Serializable
- case object EmptyTap extends Tap[Nothing] with Product with Serializable
- object EmptyTapOf extends Serializable
- object MaterializeTap extends Serializable
- object ScioIO
- object TapOf extends Serializable
- object TapT extends Serializable
- object Taps
Companion object for Taps.
- object TapsSysProps extends SysProps
- object TextIO extends Serializable