captioning

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Code for "Aligning Linguistic Words and Visual Semantic Units for Image Captioning", ACM MM 2019

Fully-Convolutional Point Networks for Large-Scale Point Clouds

Applying Common-Sense Reasoning to Multi-Modal Dense Video Captioning and Video Question Answering | Python3 | PyTorch | CNNs | Causality | Reasoning | LSTMs | Transformers | Multi-Head Self Attention | Published in IEEE Winter Conference on Applications of Computer Vision (WACV) 2021

A Base Tensorflow Project for Medical Report Generation

Python code for handling the Clotho dataset.

[CVPR21] Visual Semantic Role Labeling for Video Understanding (https://arxiv.org/abs/2104.00990)

What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering

Audio captioning baseline system for DCASE 2020 challenge.

A Tennis dataset and models for event detection & commentary generation

[CVPR 2022] X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning

My notes on some Deep Learning papers

Toolkit for supporting the EBU-TT Live specification

CaMEL: Mean Teacher Learning for Image Captioning. ICPR 2022

S2VT (seq2seq) video captioning with bahdanau & luong attention implementation in Tensorflow

Some papers about *diverse* image (a few videos) captioning

Official python implementation of R3-Transformer

Sample app to display live captioning to a WebRTC video session with the Deepgram API.