Releases: intel-analytics/BigDL

Highlights

Note: BigDL v2.2.0 has been updated to include functional and security updates. Users should update to the latest version.

• Nano
  • Extend BigDL Nano inference to support iGPU and more data types (INT8/BF16/FP16 quantization)
  • More performance features (e.g., InferenceOptimizer for Keras, Nano decorator for PyTorch training loop, Nano Context Manager for thread number control and autocast, etc.)
  • Support installation with more PyTorch/TensorFlow versions and conditional dependencies on different platforms
• PPML
  • Upgrade BigDL PPML solution to support new LibOS (e.g., Gramine1.3.1, Occlum0.29.2) with better security, higher performance, more stability and easier deployment.
  • Support more Big Data frameworks (Spark 3.1.3, Flink, Hive etc.), more Python and Data Science tools (Numpy, Pandas, sklearn, Torch Serv, Triton, Flask etc.), and distributed DL training using Orca
  • Improve the Attestation (e.g., MREnclave Attestation), Key Management (e.g., multi-KMS) & Encryption (e.g., transparent encryption) features for better end-to-end secure pipeline.
  • Initial support of BigDL PPML on SPR TDX (Virtual Machine and TDX Confidential Container)
• Chronos
  • Extend BigDL Chronos to support Windows and Mac, and new Python versions (3.8/3.9)
  • Provide a benchmark tool for Chronos users to evaluate Chronos performance on their platform
  • More performance features (e.g., accuracy and performance improvement for TCNForecaster, lower memory usage, auto optimization search, faster and portable TSDataset, etc.)
• Friesian
  • LightGBM training support
  • Performance improvements for online serving pipeline
• Orca
  • Improve Orca Estimator APIs for better user experience
  • Memory optimization for distributed training with Spark DataFrame,
  • Better support for image inputs and visualization with Xshards
  • Distributed MMCV applications using Orca
• Documentation
  • Tutorials for running BigDL Orca on YARN/K8s/Databricks
  • BigDL PPML solutions on Azure
  • How-to guides and examples for Chronos forecasting and deployment process

Highlights

Note: BigDL v2.1.0 has been updated to include functional and security updates. Users should update to the latest version.

• Orca
  • Improve user experience and API consistency for Orca Estimators.
  • Support directly save and load TensorFlow model format in Orca TensorFlow2 Estimator.
  • Provide more examples (e.g. PyTorch brain image segmentation, XShards tutorials for distributed Python data processing), etc.
  • Support customized metrics in Orca PyTorch Estimator.
• Nano
  • New inference optimization pipelines, with more optimization methods and a new InferenceOptimizer
  • More training optimization methods (bf16, channel last)
  • Add TorchNano support for PyTorch model customized training loop
  • Auto-scale learning rate for multi-instance training
  • Built-in AutoML support through hyperparameter optimization
  • Support a wide range versions of pytorch (1.9-1.12) and tensorflow (2.7-2.9)
• DLlib
  • Add LightGBM support
  • Improve Keras-style model summary API
  • Add Python support for loading HDFS files
• Chronos
  • Add new Autoformer (https://arxiv.org/abs/2106.13008) Forecaster and pipeline that are optimized on CPU
  • Tensorflow 2 support for LSTM, Seq2Seq, TCN and MTNet Forecasters
  • Add light-weight (does not rely on Spark/Ray Tune) auto tunning
  • Better support on distributed workflow (spark df and distributed pandas processing)
  • Add more installation options is now supported to make the installation lighter
• Friesian:
  • Integration of DeepRec (https://github.com/alibaba/DeepRec) with Friesian.
  • Add more reference examples, e.g. multi-task recommendation, TFRS (https://www.tensorflow.org/recommenders) list-wise ranking, LightGBM training, etc.
  • Add a reference example for offline distributed similarity search (using FAISS)
  • More operations in FeatureTable (e.g. string embeddings with BERT, etc.).
• PPML
  • Upgrade BigDL PPML on Gramine.
  • Improve the attestation and key managing process
  • More Big Data frameworks on BigDL PPML (including spark, flink, hive, hdfs, etc.)
  • Add PPMLContext API for encryption IO and KMS, supports different file formats, encryption algorithms and KMS services
  • Support PSI, Pytorch NN, Keras NN, FGBoost (federated XGBoost) in VFL scenario, linear regression & logistic regression for VFL

Highlights

Note: BigDL v2.0.0 has been updated to include functional and security updates. Users should update to the latest version.

Highlights

• Continue RNN optimization. We support both LSTM and GRU integration with MKL-DNN which acheives ~3x performance

• ONNX support. We support loading third party framework models via ONNX

• Richer data preprocssing support and segmentation inference pipeline support

Details

• [New Feature] Full MaskRCNN model support with data processing
• [New Feature] Support variable-size Resize
• [New Feature] Support batch input for region proposal
• [New Feature] Support samples of different size in one minibatch
• [New Feature] MAP validation method implementation
• [New Feature] ROILabel enhancement to support both object detection and segmentation
• [New Feature] Grey image support for segmentation
• [New Feature] Add TopBlocks support for Feature Pyramid Networks (FPN)
• [New Feature] GRU integration with MKL-DNN support
• [New Feature] MaskHead support for MaskRCNN
• [New Feature] BoxHead support for MaskRCNN
• [New Feature] RegionalProposal support for MaskRCNN
• [New Feature] Shape operation support for ONNX
• [New Feature] Gemm operation support for ONNX
• [New Feature] Gather operation support for ONNX
• [New Feature] AveragePool operation support for ONNX
• [New Feature] BatchNormalization operation support for ONNX
• [New Feature] Concat operation support for ONNX
• [New Feature] Conv operation support for ONNX
• [New Feature] MaxPool operation support for ONNX
• [New Feature] Reshape operation support for ONNX
• [New Feature] Relu operation support for ONNX
• [New Feature] SoftMax operation support for ONNX
• [New Feature] Sum operation support for ONNX
• [New Feature] Squeeze operation support for ONNX
• [New Feature] Const operation support for ONNX
• [New Feature] ONNX model loader implementation
• [New Feature] RioAlign layer support
• [Enhancement] Align batch normalization layer between mklblas and mkl-dnn
• [Enhancement] Python API enhancement to support nested list input
• [Enhancement] Multi-model training/inference support with MKL-DNN
• [Enhancement] BatchNormalization fusion with Scale
• [Enhancement] SoftMax companion object support no argument initialization
• [Enhancement] Python support for training with MKL-DNN
• [Enhancement] Docs enhancement
• [Bug Fix] Fix model version comparison
• [Bug Fix] Fix graph backward bug for ParallelTable
• [Bug Fix] Fix memory leak for training with MKL-DNN
• [Bug Fix] Fix performance caused by denormal values during training
• [Bug Fix] Fix SoftMax segment fault issue under MKL-DNN
• [Bug Fix] Fix TimeDistributedCriterion python API inconsistent with Scala

Highlights

• Continue VNNI acceleration support, we add optimization for more CNN models including object detection models, enhance model scales generation support for VNNI.

• Add attention based model support, we add Transformer implementation for both lanuage model and translation model.

• RNN optimization, We support LSTM integration with MKL-DNN which acheives ~3x performance speedup.

Details

• [New Feature] Add attention layer support
• [New Feature] Add FeedForwardNetwork layer support
• [New Feature] Add ExpandSize layer support
• [New Feature] Add TableOperation layer to support table calculation with different input sizes
• [New Feature] Add LayerNormalizaiton layer support
• [New Feature] Add Transformer support for both language and translation models
• [New Feature] Add beam search support in Transformer model
• [New Feature] Add Layer-wise adaptve rate scaling optim method
• [New Feature] Add LSTM integration with MKL-DNN support
• [New Feature] Add dilated convolution integration with MKL-DNN support
• [New Feature] Add parameter process for LarsSGD optim method
• [New Feature] Support Affinity binding option with mkl-dnn
• [Enhancement] Document enhancement for configuration and build
• [Enhancement] Reflection enhancement to get default values for constructor parameters
• [Enhhancement] User one AllReducemParameter for multi-optim method training
• [Enhancement] CAddTable layer enhancement to support input expansion along specific dimension
• [Enhancement] Resnet-50 preprocessing pipeline enhancement to replace RandomCropper with CenterCropper
• [Enhancement] Calculate model scales for arbitrary mask
• [Enhancment] Enable global average pooling
• [Enhancement] Check input shape and underlying MKL-DNN layout consistency
• [Enhancement] Threadpool enhancement to throw proper exception at executor runtime
• [Enhancement] Support mkl-dnn format conversion from ntc to tnc
• [Bug Fix] Fix backward graph generation topology ordering issue
• [Bug Fix] Fix MemoryData hash code calculation
• [Bug Fix] Fix log output for BCECriterion
• [Bug Fix] Fix setting mask for container quantization
• [Bug Fix] Fix validation accuracy issue when multi-executor running with the same worker
• [Bug Fix] Fix INT8 layer fusion between conlution with multi-group masks and BatchNormalization
• [Bug Fix] Fix JoinTable scales generation issue
• [Bug Fix] Fix CMul forward issue with special input format
• [Bug Fix] Fix weights change issue after model fusion issue
• [Bug Fix] Fix SpatinalConvolution primitives initializaiton issue

Highlights

• Add MKL-DNN Int8 support, especially for VNNI acceleration support. Low precision inference accelerates both latency and throughput significantly
• Add support for runnning MKL-BLAS models under MKL-DNN. We leverage MKL-DNN to speed up both training and inference for MKL-BLAS models
• Add Spark 2.4 support. Our examples and APIs are fully compatible with Spark 2.4, we released the binary for Spark 2.4 together with other Spark versions

Details

• [New Feature] Add MKL-DNN Int8 support, especially for VNNI support
• [New Feature] Add support for runnning MKL-BLAS models under MKL-DNN
• [New Feature] Add Spark 2.4 support
• [New Feature] Add auto fusion to speed up model inference
• [New Feature] Memoery reorder support for low precision inference
• [New Feature] Add bytes support for DNN Tensor
• [New Feature] Add SAME padding in MKL-DNN layers
• [New Feature] Add combined (add/or) triggers for training completion
• [Enhancement] Inception-V1 python training support enhancement
• [Enhancement] Distributed Optimizer enhancement to support customized optimizer
• [Enhancement] Add compute output shape for DNN supported layers
• [Enhancement] New MKL-DNN computing thread pool
• [Enhancement] Add MKL-DNN support for Predictor
• [Enhancement] Documentation enhancement for Sparse Tensor, MKL-DNN support, etc
• [Enhancement] Add ceilm mode for AvgPooling and MaxPooling layers
• [Enhacement] Add binary classification support for DLClassifierModel
• [Enhacement] Improvement to support conversion between NHWC and NCHW for memory reoder
• [Bug Fix] Fix SoftMax layer with narrowed input
• [Bug Fix] TensorFlow loader to support checking all data types
• [Bug Fix] Fix Add operation bug to support double type when loading TensorFlow graph
• [Bug Fix] Fix one-step weight update missing issue in validation during training
• [Bug Fix] Fix scala compiler security issue in 2.10 & 2.11
• [Bug Fix] Fix model broadcast cache UUID issue
• [Bug Fix] Fix predictor issue for batch size == 1