Json

Scio supports reading and writing type-safe Json to a case class via circe. Scio must be able to derive Encoder and Decoder instances for the record type.

If you need support for custom encoders or decoders, see the circe documentation

Reading Json

Read Json into a record type with jsonFile:

import com.spotify.scio._
import com.spotify.scio.values.SCollection
import com.spotify.scio.extra.json._

case class Record(i: Int, d: Double, s: String)

val sc: ScioContext = ???
val records: SCollection[Record] = sc.jsonFile[Record]("input.json")

Writing Json

Write to Json with saveAsJsonFile, which optionally takes a custom printer argument of type io.circe.Printer for controlling formatting.

import com.spotify.scio._
import com.spotify.scio.extra.json._
import com.spotify.scio.values.SCollection

case class Record(i: Int, d: Double, s: String)

val elements: SCollection[Record] = ???
elements.saveAsJsonFile("gs://<output-path>")