Type Members

1.  sealed trait BigQueryIO[T] extends ScioIO[T]
2.  final case class BigQuerySelect(sqlQuery: Query) extends BigQueryIO[TableRow] with Product with Serializable

   Get an SCollection for a BigQuery SELECT query.

   Get an SCollection for a BigQuery SELECT query. Both Legacy SQL and Standard SQL dialects are supported. By default the query dialect will be automatically detected. To override this behavior, start the query string with #legacysql or #standardsql.

3.  final case class BigQueryStorage(table: Table, selectedFields: List[String], rowRestriction: Option[String]) extends BigQueryIO[TableRow] with Product with Serializable

   Get an IO for a BigQuery table using the storage API.

4.  final case class BigQueryStorageSelect(sqlQuery: Query) extends BigQueryIO[TableRow] with Product with Serializable
5.  final case class BigQueryStorageTap(table: Table, readOptions: TableReadOptions) extends Tap[TableRow] with Product with Serializable

   Tap for BigQuery tables using storage api.

6.  type BigQueryTag = bigquery.types.BigQueryTag

   BigQuery tag for macro generated classes/fields.

7.  final case class BigQueryTap(table: TableReference) extends Tap[TableRow] with Product with Serializable

   Tap for BigQuery tables.

8.  final case class BigQueryTaps(self: Taps) extends Product with Serializable
9.  final case class BigQueryTypedSelect[T](reader: TypedRead[T], sqlQuery: Query, fromTableRow: (TableRow) => T)(implicit evidence$1: Coder[T]) extends BigQueryIO[T] with Product with Serializable
10.  final case class BigQueryTypedTable[T](reader: TypedRead[T], writer: Write[T], table: Table, fn: (GenericRecord, TableSchema) => T)(implicit evidence$5: Coder[T]) extends BigQueryIO[T] with WriteResultIO[T] with Product with Serializable
11.  final case class BigQueryTypedTap[T](table: Table, fn: (GenericRecord, TableSchema) => T)(implicit evidence$1: Coder[T]) extends Tap[T] with Product with Serializable
12.  final case class Clustering(fields: Seq[String] = Seq()) extends Product with Serializable

    Scala wrapper for com.google.api.services.bigquery.model.Clustering.

13.  class MockBigQuery extends AnyRef

    Mock BigQuery environment for integration test.

    Mock BigQuery environment for integration test.

    Use mockTable to feed data into live BigQuery service and queryResult to query them.

14.  class MockTable extends AnyRef

    A BigQuery table being mocked for test.

15.  final case class Query(underlying: String) extends Source with Product with Serializable

    A wrapper type Query which wraps a SQL String.

16.  sealed trait Sharding extends AnyRef

    Scala representation for BQ write sharding.

17.  sealed trait Source extends AnyRef
18.  sealed trait Table extends Source

    Table abstracts the multiple ways of referencing Bigquery tables.

    Table abstracts the multiple ways of referencing Bigquery tables. Tables can be referenced by a table spec String or by a table reference GTableReference.

    Example:

    val table = Table.Spec("bigquery-public-data:samples.shakespeare")
    sc.bigQueryTable(table)
      .filter(r => "hamlet".equals(r.getString("corpus")) && "Polonius".equals(r.getString("word")))
      .saveAsTextFile("./output.txt")
    sc.run()

    Or create a Table from a GTableReference:

    val tableReference = new TableReference
    tableReference.setProjectId("bigquery-public-data")
    tableReference.setDatasetId("samples")
    tableReference.setTableId("shakespeare")
    val table = Table.Ref(tableReference)

    A helper method is provided to replace the "$LATEST" placeholder in the table name to the latest common partition.

    val table = Table.Spec("some_project:some_data.some_table_$LATEST").latest()

19.  type TableRow = google.api.services.bigquery.model.TableRow

    Alias for BigQuery TableRow.

20.  final case class TableRowJsonIO(path: String) extends ScioIO[TableRow] with Product with Serializable

    Get an IO for a BigQuery TableRow JSON file.

21.  final case class TableRowJsonTap(path: String, params: ReadParam) extends Tap[TableRow] with Product with Serializable

    Tap for BigQuery TableRow JSON files.

22.  final case class TimePartitioning(type: String, field: String = null, expirationMs: Long = 0, requirePartitionFilter: Boolean = false) extends Product with Serializable

    Scala wrapper for com.google.api.services.bigquery.model.TimePartitioning.

23.  type description = bigquery.types.description

    Annotation for BigQuery field description.