final class ScioContextOps extends AnyVal
Enhanced version of ScioContext with Parquet Avro methods.
- Source
- ScioContextSyntax.scala
Linear Supertypes
Ordering
- Alphabetic
- By Inheritance
Inherited
- ScioContextOps
- AnyVal
- Any
- Hide All
- Show All
Visibility
- Public
- Protected
Instance Constructors
- new ScioContextOps(self: ScioContext)
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- Any
- final def ##: Int
- Definition Classes
- Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def getClass(): Class[_ <: AnyVal]
- Definition Classes
- AnyVal → Any
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def parquetAvroFile[T <: GenericRecord](path: String, projection: Schema = ReadParam.DefaultProjection, predicate: FilterPredicate = ReadParam.DefaultPredicate, conf: Configuration = ReadParam.DefaultConfiguration, suffix: String = ReadParam.DefaultSuffix)(implicit arg0: ClassTag[T]): ParquetAvroFile[T]
Get an SCollection for a Parquet file as Avro records.
Get an SCollection for a Parquet file as Avro records. Since Avro records produced by Parquet column projection may be incomplete and may fail serialization, you must map the result to extract projected fields from the Avro records.
Note that due to limitations of the underlying
HadoopInputFormatIO, dynamic work rebalancing is not supported. Pipelines may not autoscale up or down during the initial read and subsequent fused transforms. - def toString(): String
- Definition Classes
- Any