Scio v0.10.0

TL;DR

• Google Cloud Platform Modules
• Coder Implicits

Google Cloud Platform Modules

All Google Cloud Platform related components are moved under scio-google-cloud-platform. This includes Datastore, originally part of scio-core and the following modules.

• scio-bigquery
• scio-bigtable
• scio-spanner

Update your build.sbt accordingly, for example:

    libraryDependencies ++= Seq(
      "com.spotify" %% "scio-core" % scioVersion,
-     "com.spotify" %% "scio-bigquery" % scioVersion,
-     "com.spotify" %% "scio-bigtable" % scioVersion,
-     "com.spotify" %% "scio-spanner" % scioVersion,
+     "com.spotify" %% "scio-google-cloud-platform" % scioVersion,
      "com.spotify" %% "scio-test" % scioVersion % Test
    )

Coder Implicits

Coder inference has been greatly simplified and many cases of (implicit koder: Coder[K], voder: Coder[V]) implicit arguments are removed from the public API. While this is source level compatible for the most part, there are a few exceptions. They should also be covered by the scalafix rules.

    val ints: SCollection[Int] = sc.parallelize(1 to 100)
    val kvs: SCollection[(String, Int)] = ints.keyBy("key" + _)

    // SCollection methods with implicit Ordering[T]
-   ints.top(10, Ordering[Int].reverse)
-   ints.quantileApprox(10, Ordering[Int].reverse)
+   ints.top(10)(Ordering[Int].reverse)
+   ints.quantileApprox(10)(Ordering[Int].reverse)

    // PairSCollection methods with implicit Ordering[T]
-   kvs.topByKey(10, Ordering[Int].reverse)
-   kvs.approxQuantilesByKey(10, Ordering[Int].reverse)
+   kvs.topByKey(10)(Ordering[Int].reverse)
+   kvs.approxQuantilesByKey(10)(Ordering[Int].reverse)