NeXtVLAD: An Efficient Neural Network to Aggregate Frame-level Features for Large-scale Video Classification

November 12, 2018 · Entered Twilight · 🏛 ECCV Workshops

"Last commit was 6.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: CONTRIBUTING.md, ECCV2018_phoenix_lin_presentation.pdf, LICENSE, README.md, YOUTUBE8M_README.md, __init__.py, average_precision_calculator.py, cloudml-4gpu.yaml, cloudml-gpu-distributed.yaml, cloudml-gpu.yaml, convert_prediction_from_json_to_csv.py, eccv2018submission.pdf, eval.py, eval_util.py, export_model.py, feature_extractor, frame_level_models.py, inference.py, losses.py, mean_average_precision_calculator.py, model_utils.py, models.py, nextvlad.py, readers.py, scripts, train.py, utils.py, video_level_models.py, yt8m_pca

Authors Rongcheng Lin, Jing Xiao, Jianping Fan arXiv ID 1811.05014 Category cs.CV: Computer Vision Citations 108 Venue ECCV Workshops Repository https://github.com/linrongc/youtube-8m ⭐ 208 Last Checked 1 month ago

Abstract

This paper introduces a fast and efficient network architecture, NeXtVLAD, to aggregate frame-level features into a compact feature vector for large-scale video classification. Briefly speaking, the basic idea is to decompose a high-dimensional feature into a group of relatively low-dimensional vectors with attention before applying NetVLAD aggregation over time. This NeXtVLAD approach turns out to be both effective and parameter efficient in aggregating temporal information. In the 2nd Youtube-8M video understanding challenge, a single NeXtVLAD model with less than 80M parameters achieves a GAP score of 0.87846 in private leaderboard. A mixture of 3 NeXtVLAD models results in 0.88722, which is ranked 3rd over 394 teams. The code is publicly available at https://github.com/linrongc/youtube-8m.