When using e.g. AvroSortedBucketIO, if a file pattern is passed as part of the input path, scio should output a helpful error message. Current behavior in 0.9.3 leads to a NullPointerException.

val myEntityPaths: List[String] = List(
  "gs://bucket/dir1/*.avro",
  "gs://bucket/dir2/*.avro",
)
sc.sortMergeCoGroup(
  classOf[String],
  AvroSortedBucketIO
    .read(new TupleTag[MyE

In v0.9.0 we started offering --keep_intermediate_avro_files. Avro files are a great candidate for long term storage of variant data for several reasons, mainly because they can be loaded into BigQuery fast and free.

The only missing data is information in VCF header. Storing them with AVRO files will enable us to recov