Packages

package avro

Main package for Avro APIs. Import all.

import com.spotify.scio.avro._
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. avro
  2. Syntax
  3. SCollectionSyntax
  4. ScioContextSyntax
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Package Members

  1. package syntax
  2. package types

Type Members

  1. sealed trait AvroIO[T] extends ScioIO[T]
  2. final case class AvroTaps(self: Taps) extends Product with Serializable
  3. final case class GenericRecordIO(path: String, schema: Schema) extends AvroIO[GenericRecord] with Product with Serializable
  4. final case class GenericRecordParseIO[T](path: String, parseFn: (GenericRecord) => T)(implicit coder: Coder[T]) extends AvroIO[T] with Product with Serializable

    Given a parseFn, read GenericRecord and apply a function mapping => T before producing output.

    Given a parseFn, read GenericRecord and apply a function mapping => T before producing output. This IO applies the function at the time of de-serializing Avro GenericRecords.

    This IO doesn't define write, and should not be used to write Avro GenericRecords.

  5. final case class GenericRecordParseTap[T](path: String, parseFn: (GenericRecord) => T)(implicit evidence$3: Coder[T]) extends Tap[T] with Product with Serializable

    Tap for reading GenericRecord Avro files and applying a parseFn to parse it to the given type T

  6. final case class GenericRecordTap(path: String, schema: Schema) extends Tap[GenericRecord] with Product with Serializable

    Tap for GenericRecord Avro files.

  7. final case class ObjectFileIO[T](path: String)(implicit evidence$1: Coder[T]) extends ScioIO[T] with Product with Serializable
  8. case class ObjectFileTap[T](path: String)(implicit evidence$4: Coder[T]) extends Tap[T] with Product with Serializable

    Tap for object files.

    Tap for object files. Note that serialization is not guaranteed to be compatible across Scio releases.

  9. final case class ProtobufIO[T <: Message](path: String)(implicit evidence$2: ClassTag[T]) extends ScioIO[T] with Product with Serializable
  10. final case class SpecificRecordIO[T <: SpecificRecord](path: String)(implicit evidence$3: ClassTag[T], evidence$4: Coder[T]) extends AvroIO[T] with Product with Serializable
  11. final case class SpecificRecordTap[T <: SpecificRecord](path: String)(implicit evidence$1: ClassTag[T], evidence$2: Coder[T]) extends Tap[T] with Product with Serializable

    Tap for SpecificRecord Avro files.

  12. type doc = avro.types.doc

    Annotation for Avro field and record documentation.

Value Members

  1. val AvroType: avro.types.AvroType.type

    Typed Avro annotations and converters.

  2. implicit def avroGenericRecordSCollectionOps(c: SCollection[GenericRecord]): GenericRecordSCollectionOps
    Definition Classes
    SCollectionSyntax
  3. implicit def avroObjectFileSCollectionOps[T](c: SCollection[T]): ObjectFileSCollectionOps[T]
    Definition Classes
    SCollectionSyntax
  4. implicit def avroProtobufSCollectionOps[T <: Message](c: SCollection[T]): ProtobufSCollectionOps[T]
    Definition Classes
    SCollectionSyntax
  5. implicit def avroScioContextOps(c: ScioContext): ScioContextOps
    Definition Classes
    ScioContextSyntax
  6. implicit def avroSpecificRecordSCollectionOps[T <: SpecificRecord](c: SCollection[T]): SpecificRecordSCollectionOps[T]
    Definition Classes
    SCollectionSyntax
  7. implicit def avroTypedAvroSCollectionOps[T <: HasAvroAnnotation](c: SCollection[T]): TypedAvroSCollectionOps[T]
    Definition Classes
    SCollectionSyntax
  8. object AvroIO
  9. object AvroSysProps
    Annotations
    @registerSysProps()
  10. object AvroTyped
  11. object ObjectFileIO extends Serializable
  12. object ProtobufIO extends Serializable

Inherited from Syntax

Inherited from SCollectionSyntax

Inherited from ScioContextSyntax

Inherited from AnyRef

Inherited from Any

Ungrouped