Repolode
Enviar
showlab

showlab/livecc

A video LLM trained with streaming speech transcription that produces real-time commentary on live video, from the CVPR 2025 LiveCC paper.

Ver en GitHub
Estrellas
480
Forks
59
Issues abiertos
16
Último commit
hace 12 meses

¿Qué es?

LiveCC is a video large language model trained on videos paired with streaming speech transcription, so it can generate running commentary while a video plays rather than only answering questions about a finished clip. The project is the research code for the CVPR 2025 paper of the same name, covering training, inference and evaluation.

The repository provides a Gradio web demo and a command-line demo for trying the model, along with pre-training and supervised fine-tuning scripts built on DeepSpeed and Hugging Face Transformers. Published checkpoints, datasets such as Live-WhisperX-526K, and a benchmark called LiveSports3KCC are linked from the README. Evaluation scripts cover VideoMME and OVOBench, and a data production pipeline is included for users who want to rebuild the training data.

The code targets Python 3.11 or newer and is built on Qwen2-VL-based models, with Flash Attention and Liger kernels used for efficiency.

Funciones clave

  • 01Generates real-time video commentary using streaming speech transcription training.
  • 02Includes a Gradio web demo and a command-line demo.
  • 03Provides pre-training and SFT scripts using DeepSpeed.
  • 04Ships evaluation scripts for LiveSports3KCC, VideoMME and OVOBench.
  • 05Links published checkpoints and training datasets on Hugging Face.

Casos de uso

  • Researchers studying video LLMs trained on speech-aligned data.
  • Teams building live sports or event commentary prototypes.
  • Developers reproducing the LiveCC paper's training and evaluation results.

Temas

#video-llm#multimodal#speech-transcription#real-time#computer-vision#deep-learning#pytorch

Preguntas frecuentes

¿Qué es livecc?
A video LLM trained with streaming speech transcription that produces real-time commentary on live video, from the CVPR 2025 LiveCC paper. LiveCC is a video large language model trained on videos paired with streaming speech transcription, so it can generate running commentary while a video plays rather than only answering questions about a finished clip. The project is the research code for the CVPR 2025 paper of the same name, covering training, inference and evaluation.
¿livecc es gratuito? ¿Qué licencia tiene?
livecc no declara ninguna licencia. Su código es público, pero sin licencia no tienes permiso explícito para reutilizarlo; consulta al autor antes de depender de él.
¿livecc sigue mantenido?
Sin commits en más de 6 meses. El último commit de livecc fue el 29 oct 2025.
¿Para qué puedo usar livecc?
Researchers studying video LLMs trained on speech-aligned data. Teams building live sports or event commentary prototypes. Developers reproducing the LiveCC paper's training and evaluation results.
¿Dónde consigo livecc?
El código fuente y las instrucciones de instalación están en GitHub: https://github.com/showlab/livecc

Resumen generado por IA a partir del README del proyecto. Las estadísticas de GitHub se actualizan a diario.

Proyectos relacionados