Triton Inference Server

Repositories

• core
  C BSD-3-Clause 5 5 0 1 Updated Aug 3, 2021

• onnxruntime_backend

  The Triton backend for the ONNX Runtime.

  backend inference triton-inference-server onnx-runtime
  C++ BSD-3-Clause 11 17 11 2 Updated Aug 3, 2021

• server

  The Triton Inference Server provides an optimized cloud and edge inferencing solution.

  machine-learning cloud deep-learning gpu inference edge datacenter
  C++ 594 2,488 87 (1 issue needs help) 13 Updated Aug 3, 2021

• openvino_backend

  OpenVINO backend for Triton.
  C++ BSD-3-Clause 2 6 0 3 Updated Aug 3, 2021

• model_analyzer

  Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

  deep-learning gpu inference performance-analysis
  Python Apache-2.0 20 63 6 3 Updated Aug 3, 2021

• python_backend

  Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.
  C++ BSD-3-Clause 22 58 0 2 Updated Aug 3, 2021

• backend

  Common source, scripts and utilities for creating Triton backends.
  C++ BSD-3-Clause 11 44 0 1 Updated Aug 3, 2021

• fil_backend

  FIL backend for the Triton Inference Server
  Jupyter Notebook Apache-2.0 8 11 17 3 Updated Aug 3, 2021

• common

  Common source, scripts and utilities shared across all Triton repositories.
  C++ BSD-3-Clause 9 4 0 1 Updated Aug 3, 2021

• third_party

  Third-party source packages that are modified for use in Triton.
  C BSD-3-Clause 3 0 0 0 Updated Aug 3, 2021

• dali_backend

  The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.

  python deep-learning gpu image-processing dali data-preprocessing nvidia-dali
  C++ MIT 7 42 3 3 Updated Aug 3, 2021

• pytorch_backend

  The Triton backend for the PyTorch TorchScript models.
  C++ BSD-3-Clause 8 15 0 1 Updated Aug 2, 2021

• hugectr_backend
  C++ BSD-3-Clause 2 8 0 0 Updated Jul 30, 2021

• fastertransformer_backend
  C++ BSD-3-Clause 7 18 0 0 Updated Jul 29, 2021

• client

  Triton Python and C++ client libraries and example, and client examples for go, java and scala.
  C++ BSD-3-Clause 20 35 0 0 Updated Jul 28, 2021

• tensorrt_backend

  The Triton backend for TensorRT.
  C++ BSD-3-Clause 0 0 0 0 Updated Jul 28, 2021

• identity_backend

  Example Triton backend that demonstrates most of the Triton Backend API.
  C++ BSD-3-Clause 4 2 0 0 Updated Jul 28, 2021

• model_navigator

  The Triton Model Navigator is a tool that provides the ability to automate the process of model deployment on the Triton Inference Server.

  deep-learning gpu inference
  Python 1 20 2 0 Updated Jul 28, 2021

• tensorflow_backend

  The Triton backend for TensorFlow 1 and TensorFlow 2.
  C++ BSD-3-Clause 6 15 0 2 Updated Jul 15, 2021

• square_backend

  Simple Triton backend used for testing.
  C++ BSD-3-Clause 0 0 0 0 Updated Jul 8, 2021

• repeat_backend

  An example Triton backend that demonstrates sending zero, one, or multiple responses for each request.
  C++ BSD-3-Clause 1 0 0 0 Updated Jul 8, 2021

• checksum_repository_agent

  The Triton repository agent that verifies model checksums.
  C++ BSD-3-Clause 0 0 0 0 Updated Jul 8, 2021

• paddlepaddle_backend
  C++ BSD-3-Clause 2 6 0 0 Updated Jul 7, 2021

• contrib

  Community contributions to Triton that are not officially supported or maintained by the Triton project.
  BSD-3-Clause 3 3 0 1 Updated Nov 4, 2020