Skip to content

spotify/tfreader

main
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 

tfreader

Simple native CLI tool to read TensorFlow TFRecords.

Install

MacOs

brew tap spotify/public
brew install tfreader

Linux

Right now we only have binaries available under releases

Usage

Usage: tfr [options] <files? | STDIN>
TensorFlow TFRecord reader CLI tool
Options:

  -c, --check-crc32     Enable checks CRC32 on each record
  -f, --flat            Output examples as flat JSON objects
  -n, --number  <arg>   Number of records to output
  -r, --record  <arg>   Record type to be read { example | prediction_log }
  -h, --help            Show help message

 trailing arguments:
  files (not required)   files? | STDIN

Examples

Google Cloud Storage

tfr -n 1 gs://<bucket>/<path>/part-00000-of-00004.tfrecords | jq .

Local Filesystem

tfr -n 1 core/src/test/resources/part-00000-of-00004.tfrecords | jq .

stdin

cat core/src/test/resources/part-00000-of-00004.tfrecords | tfr -n 1 | jq .

Output

Flat

{
  "tips": [0],
  "trip_seconds": [60],
  "payment_type": ["Cash"],
  "trip_miles": [0],
  "dropoff_longitude": [-87.63785],
  "dropoff_latitude": [41.893215],
  "pickup_longitude": [-87.63187],
  "pickup_latitude": [41.89204],
  "trip_start_timestamp": [1402934400],
  "trip_start_day": [2],
  "trip_start_hour": [16],
  "trip_start_month": [6],
  "fare": [3.25],
  "dropoff_census_tract": ["17031081800"],
  "dropoff_community_area": ["8"],
  "pickup_community_area": ["8"],
  "trip_id": ["8106c1f6-e6f3-426f-9aaf-b4e9703b4f10"]
}

Default

{
  "features": {
    "feature": {
      "tips": {
        "floatList": {
          "value": [0]
        }
      },
      "trip_seconds": {
        "int64List": {
          "value": [60]
        }
      },
      "payment_type": {
        "bytesList": {
          "value": ["Cash"]
        }
      },
      "trip_miles": {
        "floatList": {
          "value": [0]
        }
      },
      "dropoff_longitude": {
        "floatList": {
          "value": [-87.63785]
        }
      },
      "dropoff_latitude": {
        "floatList": {
          "value": [41.893215]
        }
      },
      "pickup_longitude": {
        "floatList": {
          "value": [-87.63187]
        }
      },
      "pickup_latitude": {
        "floatList": {
          "value": [41.89204]
        }
      },
      "trip_start_timestamp": {
        "int64List": {
          "value": [1402934400]
        }
      },
      "trip_start_day": {
        "int64List": {
          "value": [2]
        }
      },
      "trip_start_hour": {
        "int64List": {
          "value": [16]
        }
      },
      "trip_start_month": {
        "int64List": {
          "value": [6]
        }
      },
      "fare": {
        "floatList": {
          "value": [3.25]
        }
      },
      "dropoff_census_tract": {
        "bytesList": {
          "value": ["17031081800"]
        }
      },
      "dropoff_community_area": {
        "bytesList": {
          "value": ["8"]
        }
      },
      "pickup_community_area": {
        "bytesList": {
          "value": ["8"]
        }
      },
      "trip_id": {
        "bytesList": {
          "value": ["8106c1f6-e6f3-426f-9aaf-b4e9703b4f10"]
        }
      }
    }
  }
}