final class SCollectionOps[T <: IndexedRecord] extends AnyVal
Enhanced version of SCollection with Parquet Avro methods.
- Source
- SCollectionSyntax.scala
Linear Supertypes
Ordering
- Alphabetic
- By Inheritance
Inherited
- SCollectionOps
- AnyVal
- Any
- Hide All
- Show All
Visibility
- Public
- Protected
Instance Constructors
- new SCollectionOps(self: SCollection[T])
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- Any
- final def ##: Int
- Definition Classes
- Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def getClass(): Class[_ <: AnyVal]
- Definition Classes
- AnyVal → Any
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def saveAsParquetAvroFile(path: String, schema: Schema = WriteParam.DefaultSchema, numShards: Int = WriteParam.DefaultNumShards, suffix: String = WriteParam.DefaultSuffix, compression: CompressionCodecName = WriteParam.DefaultCompression, conf: Configuration = WriteParam.DefaultConfiguration, shardNameTemplate: String = WriteParam.DefaultShardNameTemplate, tempDirectory: String = WriteParam.DefaultTempDirectory, filenamePolicySupplier: FilenamePolicySupplier = WriteParam.DefaultFilenamePolicySupplier, prefix: String = WriteParam.DefaultPrefix)(implicit ct: ClassTag[T], coder: Coder[T]): ClosedTap[T]
Save this SCollection of Avro records as a Parquet file.
Save this SCollection of Avro records as a Parquet file.
- path
output location of the write operation
- schema
must be not null if
Tis of type GenericRecord.- numShards
number of shards per output directory
- suffix
defaults to .parquet
- compression
defaults to snappy
- def toString(): String
- Definition Classes
- Any