Roman/ohm_streaming
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

docs: Enhance CLAUDE.md with comprehensive project documentation

Browse Source 
- Update project overview with all supported hosts and providers
- Add complete testing commands and structure documentation
- Document all downloaders including new Sibnet and Lpayer support
- Add detailed API endpoints categorization
- Include anime features, video streaming, and recommendations sections
- Document BaseDownloader interface and factory pattern
- Add examples for adding new hosts and anime providers
- Include key implementation details (resume, domain handling, task lifecycle)
- Document all dependencies including jieba for fuzzy search
- Add testing structure with fixtures and markers
- Include important notes about HTTP client cleanup

Generated with [Claude Code](https://claude.com/claude-code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
This commit is contained in:
root
2026-01-24 16:14:49 +00:00
parent 63af6fd4d9
commit 92ef76ed2a
1 changed files with 210 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+210 -40
View File
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project Overview
Ohm Stream Downloader is a FastAPI-based web application for downloading media files from various file hosting services (1fichier, Doodstream, Rapidfile, etc.). It features a web interface, parallel downloads, pause/resume support, and direct file serving.
Ohm Stream Downloader is a FastAPI-based web application for downloading anime episodes and media files from various file hosting services (1fichier, Doodstream, Rapidfile, Uptobox, VidMoly, SendVid, Sibnet, Lpayer) and anime streaming platforms (Anime-Sama, Neko-Sama, Anime-Ultime, Vostfree). It features a modern web interface, parallel downloads, pause/resume support, video streaming, and personalized recommendations.
## Development Commands
@@ -17,10 +17,31 @@ source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
# Run development server (auto-reload)
uvicorn main:app --reload --host 0.0.0.0 --port 8000
uvicorn main:app --reload --host 0.0.0.0 --port 3000
# Access web interface
# Open http://localhost:8000/web in browser
# Open http://localhost:3000/web in browser
# Run all tests
pytest
# Run tests with coverage report
pytest --cov=app --cov-report=html
# Run only unit tests (fast, isolated)
pytest -m "unit"
# Run only integration tests
pytest -m "integration"
# Exclude slow tests
pytest -m "not slow"
# Verbose output
pytest -v
# Show print debugging
pytest -s
```
## Architecture
@@ -30,56 +51,136 @@ uvicorn main:app --reload --host 0.0.0.0 --port 8000
Ohm_streaming/
├── main.py # FastAPI application & API endpoints
├── app/
│ ├── models/ # Pydantic models (DownloadTask, DownloadStatus, etc.)
│ ├── models/ # Pydantic models (DownloadTask, AnimeMetadata, etc.)
│ ├── downloaders/ # Host-specific downloaders
│ │ ├── base.py # BaseDownloader abstract class
│ │ ├── unfichier.py # 1fichier.com handler
│ │ ├── doodstream.py # Doodstream handler
│ │ └── rapidfile.py # Rapidfile handler
│ └── download_manager.py # Manages download queue, progress, parallel downloads
│ │ ├── rapidfile.py # Rapidfile handler
│ │ ├── uptobox.py # Uptobox handler
│ │ ├── vidmoly.py # VidMoly handler
│ │ ├── sendvid.py # SendVid handler
│ │ ├── sibnet.py # Sibnet handler
│ │ ├── lpayer.py # Lpayer handler
│ │ ├── vidmoly_old.py # Old VidMoly implementation (deprecated)
│ │ ├── animesama.py # Anime-Sama (anime provider)
│ │ ├── animeultime.py # Anime-Ultime (anime provider)
│ │ ├── nekosama.py # Neko-Sama (anime provider)
│ │ ├── vostfree.py # Vostfree (anime provider)
│ │ └── __init__.py # Factory function and registry
│ ├── providers.py # Provider configuration (domains, icons, colors)
│ ├── download_manager.py # Manages download queue, progress, parallel downloads
│ ├── favorites.py # Favorites management system (JSON-based)
│ ├── recommendation_engine.py # Analyzes download history for recommendations
│ ├── recommendations.py # Fetches latest releases from anime sources
│ └── kitsu_api.py # Kitsu API integration for metadata
├── downloads/ # Downloaded files storage
├── templates/
│ └── index.html # Web interface (single-page app)
└── static/ # Static assets (CSS, JS, images)
│ ├── index.html # Main web interface
│ ├── player.html # Video player page
│ └── base.html # Base template
├── static/ # Static assets (CSS, JS, images)
└── tests/ # Test suite with fixtures
```
**Core Components:**
1. **DownloadManager** (`app/download_manager.py`)
- Manages all download tasks with parallel download limit (default: 3 concurrent)
- Handles pause/resume/cancel operations
- Tracks progress, speed, and file chunks for resume support
- Uses semaphore to limit concurrent downloads
### 1. DownloadManager (`app/download_manager.py`)
- Manages all download tasks with parallel download limit (default: 3 concurrent)
- Handles pause/resume/cancel operations
- Tracks progress, speed, and file chunks for resume support
- Uses `asyncio.Semaphore` to limit concurrent downloads
- Auto-restores completed downloads from disk on server startup
2. **Downloaders** (`app/downloaders/`)
- Each host has its own downloader class inheriting from `BaseDownloader`
- `can_handle(url)` - Checks if downloader supports the URL
- `get_download_link(url)` - Extracts direct download link and filename from host page
- Uses httpx for async HTTP requests and BeautifulSoup for HTML parsing
### 2. Downloaders (`app/downloaders/`)
3. **Download Task Flow:**
- Client sends URL via POST `/api/download`
- DownloadManager creates task with unique ID
- Appropriate downloader extracts direct link
- File downloaded in chunks (1MB) to `downloads/` directory
- Progress tracked in real-time (bytes, speed, percentage)
- Resume uses HTTP Range headers to continue from last byte
**Factory Pattern:**
- `get_downloader(url)` in `__init__.py` selects appropriate downloader
- Each downloader inherits from `BaseDownloader` abstract class
- Order matters: anime providers checked first, then file hosts
- Falls back to `GenericDownloader` if no match
**API Endpoints:**
- `POST /api/download` - Create new download task (starts automatically)
- `GET /api/downloads` - List all download tasks with status
- `GET /api/download/{task_id}` - Get specific task details
- `POST /api/download/{task_id}/pause` - Pause active download
- `POST /api/download/{task_id}/resume` - Resume paused download
- `DELETE /api/download/{task_id}` - Cancel/delete download
**BaseDownloader Interface:**
- `can_handle(url)` - Check if downloader supports the URL
- `get_download_link(url)` - Extract direct download link and filename
- `search_anime(query, lang)` - Search anime (anime providers only)
- `get_episodes(anime_url, lang)` - Get episode list (anime providers only)
- `get_anime_metadata(anime_url)` - Get metadata dict (anime providers only) - Note: not in base.py, implemented by anime providers
**Key Patterns:**
- All downloaders use httpx.AsyncClient for HTTP requests
- BeautifulSoup with lxml for HTML parsing
- Async/await throughout for non-blocking I/O
- Fuzzy search using jieba for Chinese text segmentation and typo tolerance
### 3. Provider Configuration (`app/providers.py`)
- `ANIME_PROVIDERS` - Anime streaming sites configuration
- `FILE_HOSTS` - File hosting services configuration
- Each provider has: name, domains, icon, color, url_pattern
- `detect_provider_from_url(url)` - Identify provider from URL
### 4. API Endpoints
**Download Management:**
- `POST /api/download` - Create new download task
- `GET /api/downloads` - List all download tasks
- `GET /api/download/{task_id}` - Get task details
- `POST /api/download/{task_id}/pause` - Pause download
- `POST /api/download/{task_id}/resume` - Resume download
- `DELETE /api/download/{task_id}` - Delete task (keeps completed files)
- `GET /api/download/{task_id}/file` - Download completed file
- `GET /web` - Web interface
**Web Interface:**
**Anime Features:**
- `GET /api/anime/search` - Unified search across all providers
- `GET /api/anime/metadata` - Get anime metadata
- `GET /api/anime/episodes` - Get episode list
- `POST /api/anime/download` - Download single episode
- `POST /api/anime/download-season` - Download entire season
**Video Streaming:**
- `GET /video/{task_id}` - Stream video with Range support
- `GET /stream/{filename}` - Stream by filename
- `GET /player/{task_id}` - Video player page
- `GET /watch/{filename}` - Player by filename
**Recommendations & Favorites:**
- `GET /api/recommendations` - Personalized recommendations
- `GET /api/releases/latest` - Latest anime releases
- `GET /api/favorites` - List favorites
- `POST /api/favorites` - Add favorite
- `DELETE /api/favorites/{anime_id}` - Remove favorite
### 5. Web Interface
- Single-page app at `/web` (templates/index.html)
- Auto-refreshes every second to show progress
- Shows progress bar, speed, file size
- Controls: Pause, Resume, Cancel, Download completed file
- Video player with seeking support (HTTP Range headers)
- Dark theme with gradients and animations
## Test Structure
**Test Organization (tests/):**
- `conftest.py` - Pytest configuration and fixtures
- `test_models.py` - Pydantic model tests
- `test_downloaders.py` - Downloader tests
- `test_download_manager.py` - DownloadManager tests
- `test_favorites.py` - Favorites system tests
- `test_api.py` - FastAPI endpoint tests
**Fixtures in conftest.py:**
- `temp_dir` - Temporary directory
- `temp_download_dir` - Temporary download directory
- `download_manager` - DownloadManager instance
- `favorites_manager` - FavoritesManager instance
- `mock_httpx_client` - Mock for httpx.AsyncClient
- `sample_download_task` - Sample task data
- `sample_anime_metadata` - Sample metadata
**Test Markers:**
- `unit` - Unit tests (isolated, fast) - auto-applied
- `integration` - Integration tests (API endpoints) - auto-applied
- `asyncio` - Async tests - auto-applied
- `slow` - Slow tests - manual
- `network` - Requires network - manual
## Adding New Host Support
@@ -87,27 +188,96 @@ To add support for a new file hosting service:
1. Create new file in `app/downloaders/` (e.g., `myhost.py`)
2. Inherit from `BaseDownloader`
3. Implement `can_handle(url)` to detect your host URLs
4. Implement `get_download_link(url)` to extract direct download link
5. Import and add to `downloaders` list in `app/downloaders/__init__.py`
3. Implement required methods
4. Add to imports in `app/downloaders/__init__.py`
5. Add to `downloaders` list in `get_downloader()`
6. Add configuration to `FILE_HOSTS` in `app/providers.py`
Example:
```python
from .base import BaseDownloader
from bs4 import BeautifulSoup
class MyHostDownloader(BaseDownloader):
def can_handle(self, url: str) -> bool:
return "myhost.com" in url.lower()
async def get_download_link(self, url: str) -> tuple[str, str]:
# Fetch page, parse HTML, extract download URL
soup = BeautifulSoup(await self._fetch_page(url), 'lxml')
# ... extraction logic ...
return download_url, filename
def close(self):
# IMPORTANT: Always close the HTTP client
await self.client.aclose()
```
**Important:** Always close the HTTP client in your downloader to avoid resource leaks.
## Adding New Anime Provider
To add a new anime streaming provider:
1. Create downloader inheriting from `BaseDownloader`
2. Implement anime-specific methods:
- `search_anime(query, lang)` - Return list of anime with title, url, cover_image
- `get_episodes(anime_url, lang)` - Return list of episodes
- `get_anime_metadata(anime_url)` - Return metadata dict
3. Add to `ANIME_PROVIDERS` in `app/providers.py`
4. Add to factory in `app/downloaders/__init__.py`
5. Update `main.py` to include in unified search
Metadata should include:
- synopsis, genres, rating, release_year, studio, poster_image, total_episodes, status
## Configuration
Edit `main.py` to configure:
- `max_parallel` - Maximum concurrent downloads (default: 3)
- `download_dir` - Storage location (default: "downloads")
## Key Implementation Details
**Resume Support:**
- Downloads use HTTP Range headers to resume from last byte
- Files downloaded in 1MB chunks
- Partial files cleaned up on cancel
- Resume position tracked in `downloaded_bytes` field
**Domain Handling:**
- Anime providers use dynamic domain detection (e.g., Anime-Sama fetches current domain from anime-sama.pw)
- Multiple domains per provider supported in configuration
- Domain detection via `detect_provider_from_url(url)` in providers.py
**Task Lifecycle:**
- PENDING → DOWNLOADING → PAUSED / COMPLETED / CANCELLED / FAILED
- Active downloads tracked in `active_downloads` dict
- All tasks stored in `tasks` dict with UUID keys
- Completed files preserved when deleting tasks (only partial files removed)
**Video Streaming:**
- Range header support for seeking in video player
- Serves from `/downloads` directory via StaticFiles
- Video extensions: .mp4, .mkv, .avi, .mov, .wmv, .flv, .webm
**Error Handling:**
- Graceful degradation with status tracking
- Network errors caught and reported in task status
- Automatic retry on resume
- Downloads > 1MB considered complete to skip small error files
## Dependencies
**Core:**
- fastapi - Web framework
- uvicorn - ASGI server
- httpx - Async HTTP client
- beautifulsoup4, lxml - HTML parsing
- aiofiles - Async file operations
- jieba - Chinese text segmentation for fuzzy search
**Testing:**
- pytest - Test framework
- pytest-asyncio - Async test support
- pytest-cov - Coverage reporting
- pytest-mock - Mocking support