pretraining

Here are 81 public repositories matching this topic...

OFA-Sys / OFA

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

prompt chinese image-captioning pretrained-models visual-question-answering multimodal text-to-image-synthesis vision-language pretraining referring-expression-comprehension prompt-tuning

Updated Jan 11, 2023
Python

ChandlerBang / awesome-self-supervised-gnn

Star

Papers about pretraining and self-supervised learning on Graph Neural Networks (GNN).

machine-learning deep-learning graph-mining graph-neural-networks self-supervised-learning pre-training pretraining

Updated Jan 7, 2023
Python

yuewang-cuhk / awesome-vision-language-pretraining-papers

Star

Recent Advances in Vision and Language PreTrained Models (VL-PTMs)

bert vision-and-language multimodal-deep-learning pretraining vl-ptms

Updated Aug 19, 2022

X-modaler is a versatile and high-performance codebase for cross-modal analytics(e.g., image captioning, video captioning, vision-language pre-training, visual question answering, visual commonsense reasoning, and cross-modal retrieval).

image-captioning video-captioning visual-question-answering vision-and-language cross-modal-retrieval pretraining tden

Updated Dec 7, 2022
Python

Alibaba-MIIL / ImageNet21K

Star

Official Pytorch Implementation of: "ImageNet-21K Pretraining for the Masses"(NeurIPS, 2021) paper

mixer multi-label-classification downstream pretraining vision-transformer imagenet21k semantic-softmax single-label

Updated Jan 11, 2023
Python

dvlab-research / Entity

Star

EntitySeg Toolbox: Towards Open-World and High-Quality Image Segmentation

computer-vision deep-learning cnn pytorch segmentation object-detection pretrained-models image-segmentation semantic-segmentation pretrained-weights instance-segmentation panoptic-segmentation fcos pretraining detectron2 condinst

Updated Dec 5, 2022
Python

microsoft / AzureML-BERT

Star

End-to-End recipes for pre-training and fine-tuning BERT using Azure Machine Learning Service

nlp pytorch tuning pretrained-models language-model bert azure-machine-learning finetuning bert-model pretraining azureml-bert

Updated Feb 13, 2022
Jupyter Notebook

PITI-Synthesis / PITI

Star

PITI: Pretraining is All You Need for Image-to-Image Translation

computer-vision image-generation image-to-image-translation image-synthesis pretraining

Updated Dec 13, 2022
Python

PaddlePaddle / PaddleFleetX

Star

Paddle Distributed Training Examples. 飞桨分布式训练示例 Resnet Bert GPT MOE DataParallel ModelParallel PipelineParallel HybridParallel AutoParallel Zero Sharding Recompute GradientMerge Offload AMP DGC LocalSGD Wide&Deep

benchmark cloud lightning elastic unsupervised-learning large-scale data-parallelism paddlepaddle model-parallelism distributed-algorithm self-supervised-learning pipeline-parallelism pretraining fleet-api paddlecloud

Updated Jan 10, 2023
Python

amazon-science / bigdetection

Star

BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training

computer-vision object-detection few-shot pretraining

Updated Jun 16, 2022
Python

j-min / VL-T5

Star

PyTorch code for "Unifying Vision-and-Language Tasks via Text Generation" (ICML 2021)

transformers vision-and-language pretraining vl-t5 vl-bart

Updated Aug 19, 2022
Python

microsoft / UniVL

Star

An official implementation for " UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation"

video localization caption alignment segmentation coin multimodality joint multimodal-sentiment-analysis pretrain pretraining msrvtt video-text-retrieval video-text video-language youcookii retrieval-task caption-task

Updated Nov 28, 2022
Python

michiyasunaga / LinkBERT

Star

[ACL 2022] LinkBERT: A Knowledgeable Language Model 😎 Pretrained with Document Links

knowledge transformer question-answering pretrained-models language-model graph-machine-learning biomedical-applications pretraining

Updated Apr 5, 2022
Python

microsoft / LMOps

Star

General technology for enabling AI capabilities w/ LLMs and Generative AI models

nlp prompt agi lm gpt language-model pretraining llm promptist x-prompt lmops

Updated Jan 11, 2023
Python

guolinke / TUPE

Star

Transformer with Untied Positional Encoding (TUPE). Code of paper "Rethinking Positional Encoding in Language Pre-training". Improve existing models like BERT.

transformer language-model bert pretraining