Packages

final class ScioContextOps extends AnyVal

Enhanced version of ScioContext with BigQuery methods.

Source
ScioContextSyntax.scala
Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ScioContextOps
  2. AnyVal
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new ScioContextOps(self: ScioContext)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    Any
  2. final def ##: Int
    Definition Classes
    Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def bigQuerySelect(sqlQuery: Query): SCollection[TableRow]

    Get an SCollection for a BigQuery SELECT query.

    Get an SCollection for a BigQuery SELECT query. Both Legacy SQL and Standard SQL dialects are supported. By default the query dialect will be automatically detected. To override this behavior, start the query string with #legacysql or #standardsql.

  6. def bigQuerySelect(sqlQuery: Query, flattenResults: Boolean): SCollection[TableRow]

    Get an SCollection for a BigQuery SELECT query.

    Get an SCollection for a BigQuery SELECT query. Both Legacy SQL and Standard SQL dialects are supported. By default the query dialect will be automatically detected. To override this behavior, start the query string with #legacysql or #standardsql.

  7. def bigQueryStorage(query: Query): SCollection[TableRow]

    Get an SCollection for a BigQuery SELECT query using the storage API.

    Get an SCollection for a BigQuery SELECT query using the storage API.

    query

    SQL query

  8. def bigQueryStorage(table: Table, selectedFields: List[String] = BigQueryStorage.ReadParam.DefaultSelectFields, rowRestriction: String = null): SCollection[TableRow]

    Get an SCollection for a BigQuery table using the storage API.

    Get an SCollection for a BigQuery table using the storage API.

    selectedFields

    names of the fields in the table that should be read. If empty, all fields will be read. If the specified field is a nested field, all the sub-fields in the field will be selected. Fields will always appear in the generated class in the same order as they appear in the table, regardless of the order specified in selectedFields.

    rowRestriction

    SQL text filtering statement, similar ti a WHERE clause in a query. Currently, we support combinations of predicates that are a comparison between a column and a constant value in SQL statement. Aggregates are not supported. For example:

    "a > DATE '2014-09-27' AND (b > 5 AND c LIKE 'date')"
  9. def bigQueryTable[F](table: Table, format: Format[F])(implicit arg0: Coder[F]): SCollection[F]

    Get an SCollection for a BigQuery table using the specified Format.

    Get an SCollection for a BigQuery table using the specified Format.

    Reading records as GenericRecord **should** offer better performance over TableRow records.

    Note: When using Format.GenericRecord Bigquery types DATE, TIME and DATETIME are read as STRING.

  10. def bigQueryTable(table: Table): SCollection[TableRow]

    Get an SCollection for a BigQuery table.

  11. def getClass(): Class[_ <: AnyVal]
    Definition Classes
    AnyVal → Any
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. def tableRowJsonFile(path: String, compression: Compression = TableRowJsonIO.ReadParam.DefaultCompression, emptyMatchTreatment: EmptyMatchTreatment = TableRowJsonIO.ReadParam.DefaultEmptyMatchTreatment, suffix: String = TableRowJsonIO.ReadParam.DefaultSuffix): SCollection[TableRow]

    Get an SCollection for a BigQuery TableRow JSON file.

  14. def toString(): String
    Definition Classes
    Any
  15. def typedBigQuery[T <: HasAnnotation](newSource: Option[Source])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T], arg1: Coder[T]): SCollection[T]

    Get a typed SCollection for BigQuery Table or a SELECT query using the Storage API.

  16. def typedBigQuery[T <: HasAnnotation](newSource: Source)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T], arg1: Coder[T]): SCollection[T]
  17. def typedBigQuery[T <: HasAnnotation]()(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T], arg1: Coder[T]): SCollection[T]
  18. def typedBigQueryStorage[T <: HasAnnotation](table: Table, selectedFields: List[String], rowRestriction: String)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T], arg1: Coder[T]): SCollection[T]
  19. def typedBigQueryStorage[T <: HasAnnotation](table: Table, rowRestriction: String)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T], arg1: Coder[T]): SCollection[T]
  20. def typedBigQueryStorage[T <: HasAnnotation](rowRestriction: String)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T], arg1: Coder[T]): SCollection[T]
  21. def typedBigQueryStorage[T <: HasAnnotation](table: Table)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T], arg1: Coder[T]): SCollection[T]
  22. def typedBigQueryStorage[T <: HasAnnotation]()(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T], arg1: Coder[T]): SCollection[T]

    Get a typed SCollection for a BigQuery storage API.

    Get a typed SCollection for a BigQuery storage API.

    Note that T must be annotated with BigQueryType.fromStorage or BigQueryType.fromQuery

Inherited from AnyVal

Inherited from Any

Ungrouped