Json
Scio supports reading and writing type-safe Json to a case class via circe. Scio must be able to derive Encoder
and Decoder
instances for the record type.
If you need support for custom encoders or decoders, see the circe documentation
Reading Json
Read Json into a record type with jsonFile
:
import com.spotify.scio._
import com.spotify.scio.values.SCollection
import com.spotify.scio.extra.json._
case class Record(i: Int, d: Double, s: String)
val sc: ScioContext = ???
val records: SCollection[Record] = sc.jsonFile[Record]("input.json")
Writing Json
Write to Json with saveAsJsonFile
, which optionally takes a custom printer
argument of type io.circe.Printer
for controlling formatting.
import com.spotify.scio._
import com.spotify.scio.extra.json._
import com.spotify.scio.values.SCollection
case class Record(i: Int, d: Double, s: String)
val elements: SCollection[Record] = ???
elements.saveAsJsonFile("gs://<output-path>")
0.14.7-20-88d07bb-20240916T134149Z*