triviathang/YOUTUBE_AUDIO_IMPLEMENTATION_STATUS.md

# YouTube Audio Feature - Implementation Status

## ✅ COMPLETED (Backend + Infrastructure)

### Infrastructure (100%)
- ✅ Added Python dependencies: celery, redis, yt-dlp, pydub
- ✅ Updated Dockerfile.backend to install ffmpeg
- ✅ Added Redis, Celery worker, and Celery Flower services to docker-compose.yml
- ✅ Configured audio volume mounts in Docker

### Configuration (100%)
- ✅ Added audio and Celery configuration to `backend/config.py`
  - Audio folder, file size limits, allowed extensions
  - Celery broker/backend URLs
  - yt-dlp settings (format, quality)

### Database Models (100%)
- ✅ Added `YOUTUBE_AUDIO` to `QuestionType` enum
- ✅ Added YouTube fields to Question model:
  - `youtube_url` - YouTube video URL
  - `audio_path` - Path to trimmed audio file
  - `start_time` - Start time in seconds
  - `end_time` - End time in seconds
- ✅ Created `DownloadJob` model to track async processing
  - Tracks status (pending/processing/completed/failed)
  - Progress (0-100%)
  - Error messages
  - Celery task ID

### Backend Services (100%)
- ✅ Created `backend/celery_app.py` - Celery configuration
- ✅ Created `backend/services/audio_service.py` - Audio file utilities
  - UUID-based filename generation
  - File deletion cleanup
- ✅ Created `backend/services/youtube_service.py` - YouTube validation
  - URL validation with regex
  - Video duration fetching
  - Timestamp range validation
- ✅ Created `backend/tasks/youtube_tasks.py` - Celery download task
  - Downloads full audio via yt-dlp
  - Trims to timestamp range using pydub
  - Updates progress in database
  - Handles errors and cleanup

### API Routes (100%)
- ✅ Updated `backend/routes/questions.py`:
  - Accepts YouTube audio question type
  - Validates URL and timestamps
  - Creates question with pending status
  - Spawns Celery download task
  - Returns 202 with job ID
  - Deletes audio files on question deletion
- ✅ Created `backend/routes/download_jobs.py`:
  - `GET /api/download-jobs/<id>` - Get job status
  - `GET /api/download-jobs/question/<question_id>` - Get job by question
- ✅ Updated `backend/routes/admin.py` with audio control endpoints:
  - `POST /api/admin/game/<id>/audio/play` - Broadcast play command
  - `POST /api/admin/game/<id>/audio/pause` - Broadcast pause command
  - `POST /api/admin/game/<id>/audio/stop` - Broadcast stop command
  - `POST /api/admin/game/<id>/audio/seek` - Broadcast seek command

### WebSocket Broadcasts (100%)
- ✅ Updated `backend/services/game_service.py` with audio broadcast functions:
  - `broadcast_audio_play()` - Sends play event to contestant room
  - `broadcast_audio_pause()` - Sends pause event
  - `broadcast_audio_stop()` - Sends stop event
  - `broadcast_audio_seek()` - Sends seek event with position

### App Integration (100%)
- ✅ Updated `backend/app.py`:
  - Creates audio folder on startup
  - Registers download_jobs blueprint

### Frontend API Client (100%)
- ✅ Updated `frontend/frontend/src/services/api.js`:
  - Added `downloadJobsAPI` for job status polling
  - Added `audioControlAPI` for playback controls

---

## 🚧 REMAINING (Frontend UI Components)

### 1. Question Creation Form
**File**: `frontend/frontend/src/components/questionbank/QuestionBankView.jsx`

**Required Changes**:
- Add `"youtube_audio"` option to type dropdown
- Add form state for YouTube fields:
  ```javascript
  const [formData, setFormData] = useState({
    // ... existing fields
    youtube_url: '',
    start_time: 0,
    end_time: 0
  });
  const [downloadJob, setDownloadJob] = useState(null);
  const [downloadProgress, setDownloadProgress] = useState(0);
  ```
- Add conditional form fields when type === 'youtube_audio':
  - YouTube URL input
  - Start time number input (seconds)
  - End time number input (seconds)
  - Display calculated duration
- Update submit handler to POST YouTube data as JSON
- Implement polling logic:
  ```javascript
  const pollDownloadStatus = async (jobId) => {
    const pollInterval = setInterval(async () => {
      const response = await downloadJobsAPI.getStatus(jobId);
      setDownloadProgress(response.data.progress);
      if (response.data.status === 'completed') {
        clearInterval(pollInterval);
        // Reload questions, show success message
      }
    }, 2000);
  };
  ```
- Add download progress indicator UI (fixed position, bottom-right)

### 2. Audio Player Component
**File**: `frontend/frontend/src/components/audio/AudioPlayer.jsx` (NEW)

**Component Structure**:
```jsx
export default function AudioPlayer({ audioPath, isAdmin, socket, gameId }) {
  const audioRef = useRef(null);
  const [isPlaying, setIsPlaying] = useState(false);
  const [currentTime, setCurrentTime] = useState(0);
  const [duration, setDuration] = useState(0);

  // Contestant mode: Listen to WebSocket events
  useEffect(() => {
    if (!isAdmin && socket) {
      socket.on('audio_play', () => audioRef.current?.play());
      socket.on('audio_pause', () => audioRef.current?.pause());
      socket.on('audio_stop', () => {
        audioRef.current?.pause();
        audioRef.current.currentTime = 0;
      });
      socket.on('audio_seek', (data) => {
        audioRef.current.currentTime = data.position;
      });
    }
  }, [socket, isAdmin]);

  // Admin controls
  const handlePlay = async () => {
    await audioControlAPI.play(gameId);
    audioRef.current?.play();
  };
  // ... similar for pause, stop

  return (
    <div style={{ background: '#f5f5f5', padding: '1.5rem', borderRadius: '8px' }}>
      <audio ref={audioRef} src={audioPath} preload="auto" />
      {/* Progress bar */}
      {/* Time display */}
      {/* Admin controls (conditional) */}
    </div>
  );
}
```

**Styling**:
- Progress bar: clickable (admin only), blue fill (#2196F3)
- Time display: MM:SS format
- Buttons: Play (green #4CAF50), Pause (orange #ff9800), Stop (red #f44336)

### 3. Contestant View Update
**File**: `frontend/frontend/src/components/contestant/ContestantView.jsx`

**Required Changes**:
```jsx
import AudioPlayer from '../audio/AudioPlayer';

// In render, after image block:
{currentQuestion.type === 'youtube_audio' && currentQuestion.audio_path && (
  <AudioPlayer
    audioPath={currentQuestion.audio_path}
    isAdmin={false}
    socket={socket}
    gameId={gameId}
  />
)}
```

### 4. Admin View Update
**File**: `frontend/frontend/src/components/admin/GameAdminView.jsx`

**Required Changes**:
```jsx
import AudioPlayer from '../audio/AudioPlayer';

// In current question display section:
{currentQuestion?.type === 'youtube_audio' && currentQuestion?.audio_path && (
  <AudioPlayer
    audioPath={currentQuestion.audio_path}
    isAdmin={true}
    socket={socket}
    gameId={gameId}
  />
)}
```

---

## 📋 Testing Checklist

### Backend Tests
- [ ] Start containers: `docker compose up --build`
- [ ] Create migration: `docker compose exec backend uv run flask db migrate -m "Add YouTube audio support"`
- [ ] Apply migration: `docker compose exec backend uv run flask db upgrade`
- [ ] Test invalid YouTube URL: Should return 400 error
- [ ] Test invalid timestamps: Should return validation error
- [ ] Test job status endpoint: `curl http://localhost:5001/api/download-jobs/1`

### Frontend Tests (after UI completion)
- [ ] Create YouTube audio question
  - Submit URL with timestamps
  - Verify progress indicator appears
  - Wait for completion alert
  - Verify question appears in list
- [ ] Test audio playback
  - Start game with YouTube question
  - Open contestant view (tab 1)
  - Open admin view (tab 2)
  - Admin clicks Play → Contestant audio plays
  - Admin clicks Pause → Contestant audio pauses
  - Admin clicks Stop → Contestant audio resets
- [ ] Test question navigation
  - Admin clicks Next → Audio stops
  - Return to audio question → Audio ready to play again

---

## 🎯 Next Steps

1. **Complete frontend UI components** (4 files to update/create)
2. **Run database migration** (see MIGRATION_GUIDE.md)
3. **Test end-to-end workflow**
4. **Optional enhancements**:
   - Add audio waveform visualization
   - Add timestamp selector UI with embedded YouTube player
   - Add video thumbnail preview
   - Support multiple clips per question

---

## 📁 Files Modified/Created

### Backend (15 files)
- ✅ `pyproject.toml`
- ✅ `docker-compose.yml`
- ✅ `Dockerfile.backend`
- ✅ `backend/config.py`
- ✅ `backend/models.py`
- ✅ `backend/app.py`
- ✅ `backend/celery_app.py` (NEW)
- ✅ `backend/services/audio_service.py` (NEW)
- ✅ `backend/services/youtube_service.py` (NEW)
- ✅ `backend/tasks/__init__.py` (NEW)
- ✅ `backend/tasks/youtube_tasks.py` (NEW)
- ✅ `backend/routes/questions.py`
- ✅ `backend/routes/download_jobs.py` (NEW)
- ✅ `backend/routes/admin.py`
- ✅ `backend/services/game_service.py`

### Frontend (4 files - 1 complete, 3 remaining)
- ✅ `frontend/frontend/src/services/api.js`
- 🚧 `frontend/frontend/src/components/questionbank/QuestionBankView.jsx`
- 🚧 `frontend/frontend/src/components/audio/AudioPlayer.jsx` (NEW)
- 🚧 `frontend/frontend/src/components/contestant/ContestantView.jsx`
- 🚧 `frontend/frontend/src/components/admin/GameAdminView.jsx`

### Documentation
- ✅ `MIGRATION_GUIDE.md` (NEW)
- ✅ `YOUTUBE_AUDIO_IMPLEMENTATION_STATUS.md` (NEW)

---

## 🔑 Key Implementation Details

### Async Flow
1. User submits YouTube question → API creates Question with null `audio_path`
2. Celery task starts → Downloads full audio → Trims clip → Saves MP3
3. Frontend polls job status every 2 seconds
4. On completion → Question's `audio_path` updated → Frontend shows success

### Audio Playback Architecture
- **Admin**: Play/Pause/Stop buttons → API → WebSocket broadcast to contestants
- **Contestants**: Listen to WebSocket events → Control HTML5 `<audio>` element
- No direct contestant controls (fully synchronized)

### File Storage
- Audio files: `backend/static/audio/{uuid}.mp3`
- Served at: `/static/audio/{uuid}.mp3`
- Max size: 50MB, Max duration: 5 minutes

### Error Handling
- Invalid URL → 400 error with message
- Timestamp validation → 400 error
- Download failure → Job status set to "failed" with error message
- Question deletion → Cleanup audio file automatically