feat: fix auth, provider health checks, search, and redesign UI
CI / Test (Python 3.11) (push) Has been cancelled
CI / Test (Python 3.12) (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Type Check (push) Has been cancelled
CI / Summary (push) Has been cancelled

- Fix register/login: dict-style access on UserTable ORM objects
- Fix HTMX auth: inject JWT token in all HTMX request headers
- Fix FS7 search: use DLE AJAX endpoint /engine/ajax/search.php
- Fix ZT search: use ?p=series&search=QUERY (not DLE format)
- Fix provider health: load hardcoded providers + domain manager
- Add self.id to all anime/series providers
- Redesign homepage: Netflix-style horizontal scroll cards (.hc)
- Redesign search results: grouped by title, poster + synopsis + 3 buttons
- Add Télécharger dropdown: season download + episode picker
- Fix navbar CSS: restore .tabs flex layout, remove orphan rules
- Fix HTMX spinner: remove inline display:none, use CSS indicator
- Add AGENTS.md files across project for developer documentation
This commit is contained in:
root
2026-03-28 00:14:31 +00:00
parent 5d23a3d663
commit 3dc5dd8fe9
36 changed files with 2735 additions and 1989 deletions
+122 -359
View File
@@ -1,393 +1,156 @@
# AGENTS.md - Agentic Coding Guidelines
# AGENTS.md — Ohm Stream Downloader
This file provides guidance for AI agents working in this repository.
FastAPI anime/series downloader with HTMX+Alpine.js frontend, SQLModel/SQLite DB,
3-tier scraper architecture, JWT auth, Sonarr webhooks, and auto-download scheduler.
## Quick Start
## COMMANDS
```bash
# Setup
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
# Run dev server
# Dev server
uvicorn main:app --reload --host 0.0.0.0 --port 3000
```
## Build, Lint & Test Commands
# --- Tests (pytest, configured in pytest.ini, asyncio_mode=auto) ---
### Running Tests
pytest # All tests (coverage + verbose by default)
pytest -m "unit" # Fast unit tests only
pytest -m "integration" # API integration tests
pytest -m "not slow" # CI default — excludes slow tests
pytest -m "network" # Tests requiring network access
```bash
# All tests
pytest
# With coverage (HTML report in htmlcov/)
pytest --cov=app --cov-report=html
# Unit only (fast)
pytest -m "unit"
# Integration tests only
pytest -m "integration"
# Exclude slow tests
pytest -m "not slow"
# Exclude network tests (mocked only)
pytest -m "not network"
# Verbose with print debugging
pytest -v -s
# Generate HTML report
pytest --html=report.html --self-contained-html
# Timeout per test (seconds)
pytest --timeout=30
```
### Running Single Tests
```bash
# Specific file
# Single file / class / test
pytest tests/test_sonarr.py -v
# Specific class
pytest tests/test_sonarr.py::TestSonarrHandler -v
# Specific test
pytest tests/test_sonarr.py::TestSonarrHandler::test_add_mapping -v
# Pattern match
pytest -k "test_download" -v
# Debug
pytest -s # Show print() output
pytest --cov=app --cov-report=html # HTML coverage report in htmlcov/
# --- Lint & Format (ruff) ---
ruff check app/ # Lint
ruff format --check app/ # Format check (CI enforces this)
ruff format app/ # Auto-format
# --- Type Check ---
mypy app/ --ignore-missing-imports # Type check (CI enforces)
# --- DB Migrations ---
alembic revision --autogenerate -m "description"
alembic upgrade head
# --- Frontend (optional) ---
npm test # Vitest JS tests
npx playwright test # E2E browser tests
```
## Code Style
## CODE STYLE
### Imports (PEP 8 order)
1. Standard library (`os`, `json`, `asyncio`)
2. Third-party (`httpx`, `beautifulsoup4`, `fastapi`)
3. Local app (`app.config`, `app.utils`)
```python
import os
import asyncio
from typing import Optional
import httpx
from fastapi import APIRouter, HTTPException
from app.config import get_settings
from app.models import DownloadTask, DownloadStatus
```
### Imports
Three groups separated by blank lines: stdlib → third-party → local (`app.*`).
Always absolute imports (no relative). No wildcard imports (`from X import *` enforced).
### Formatting
- **Line length**: 120 chars max
- **Indentation**: 4 spaces
- **Blank lines**: 2 between top-level, 1 between inline
PEP 8, 120 chars max line length. Single quotes for strings, double quotes for docstrings.
Ruff handles linting and formatting (no local config — CI-only).
### Type Annotations
- Use explicit types
- Use `Optional[X]` not `X | None`
- Use `list[X]`, `dict[X, Y]`
### Types
Explicit type hints on all function signatures and return types.
Use `Optional[X]` from `typing`. Use `list[str]` / `dict[str, X]` (lowercase generics).
Pydantic models for all API schemas. Return type annotations required on public methods.
```python
# Good
async def get_download_link(url: str, target_filename: Optional[str] = None) -> tuple[str, str]:
results: list[dict[str, str]] = []
# Avoid
async def get_download_link(url, target_filename=None):
results = []
```
### Naming Conventions
| Element | Convention | Example |
|---------|------------|---------|
| Modules | snake_case | `download_manager.py` |
| Classes | PascalCase | `DownloadManager` |
| Functions | snake_case | `get_download_link()` |
| Constants | UPPER_SNAKE | `MAX_PARALLEL_DOWNLOADS` |
| Variables | snake_case | `download_task` |
| Enums | PascalCase | `DownloadStatus` |
| Enum values | UPPER_SNAKE | `DownloadStatus.PENDING` |
### Async/Await
- Always use for I/O operations
- Close clients properly to avoid leaks
```python
async def close(self):
await self.client.aclose()
```
### Naming
- `snake_case` for functions, variables, constants
- `PascalCase` for classes and enums
- `UPPER_SNAKE_CASE` for enum values (`DownloadStatus.PENDING`)
- `logger = logging.getLogger(__name__)` at module level
- `_` prefix for private methods (`_fetch_page`, `_sanitize`)
- `get_*` for factory functions (`get_downloader`, `get_anime_site`)
### Error Handling
- Use try/except for recoverable errors
- Raise specific exceptions (`HTTPException`, `ValueError`)
- Never use empty except blocks
- Log errors appropriately
- `HTTPException` for API errors with proper status codes
- `raise ValueError()` for business logic validation
- `try/except` with logging — never bare `except:` (known tech debt exists)
- `response.raise_for_status()` for HTTP errors
- Never return `None` for missing URLs from downloaders — raise an exception
```python
try:
result = await client.get(url)
except httpx.TimeoutException:
logger.warning(f"Request timeout for {url}")
raise HTTPException(status_code=504, detail="Request timeout")
### Docstrings
Triple double quotes `"""`. Module docstrings describe purpose. Class docstrings describe
responsibility. Complex methods use Args/Returns sections. Not all methods require docstrings.
## ARCHITECTURE
```
main.py # App entry, middleware, startup, router registration
app/
├── routers/ # 11 APIRouter modules (one per feature domain)
├── downloaders/ # 3-tier: anime_sites/ → series_sites/ → video_players/
├── models/ # Pydantic/SQLModel (Base → Table → Schema pattern)
├── config.py # Pydantic Settings from .env
├── database.py # SQLModel engine (created at import time)
├── download_manager.py # Async queue, semaphore-based parallelism
├── auth.py # JWT + bcrypt, SQLModel user storage
├── providers.py # ANIME_PROVIDERS, SERIES_PROVIDERS, FILE_HOSTS registries
└── utils.py # sanitize_filename(), is_safe_filename()
templates/ # Jinja2 + HTMX + Alpine.js
static/js/ # Vanilla ES modules (no build step)
tests/ # pytest suite (conftest.py has shared fixtures)
config/ # Runtime JSON files (users, watchlist, sonarr)
alembic/ # DB migrations
```
### File Operations
- Always sanitize filenames: `app.utils.sanitize_filename()`
- Validate paths: `app.utils.is_safe_filename()`
## KEY CONVENTIONS
### Testing
- Use pytest with pytest-asyncio
- Mark tests: `@pytest.mark.unit`, `@pytest.mark.integration`, `@pytest.mark.slow`, `@pytest.mark.network`
- Tests in `test_api.py` are auto-marked as integration, others as unit
- Use fixtures from `tests/conftest.py`
- **URL pipe format**: `video_url|anime_page_url|episode_title` — metadata preserved through download pipeline
- **Module-level init**: `database.py` creates engine on import; `main.py` creates `download_manager` on import
- **Async everywhere**: Always `async/await` for I/O — use `httpx.AsyncClient`, never `requests`
- **Factory pattern**: `get_downloader(url)` routes anime → series → video player → generic
- **Router deps**: `Depends(lambda: download_manager)`, `Depends(get_current_user_from_token)`, `Depends(lambda: templates)`
- **Dual storage**: Some features use JSON files (legacy) + SQLModel tables (newer)
- **Frontend**: No JS build step. HTMX for server interactions, Alpine.js for client state, Plyr.io for video
- **Circular import avoidance**: `episode_checker.py` uses lazy init (`set_download_manager()`)
```python
@pytest.mark.unit
@pytest.mark.asyncio
async def test_download_manager():
manager = DownloadManager(max_parallel=3)
assert manager.max_parallel == 3
## ANTI-PATTERNS (DO NOT)
# Mark slow tests
@pytest.mark.slow
async def test_full_download_flow():
...
- Use sync `requests` — always `httpx.AsyncClient`
- Return `None` for missing URLs from downloaders — raise an exception
- Skip `sanitize_filename()` on extracted filenames — path traversal risk
- Forget `await self.close()` in downloaders — resource leak
- Hardcode User-Agent in individual players — use base class headers
- Use `from X import *` — always explicit imports
- Import `download_manager` from `main.py` in app/ modules — causes circular imports
- Store secrets in `config/*.json` — use `.env`
- Use `as any`, `@ts-ignore` to suppress type errors (if adding TS)
# Mark tests requiring network
@pytest.mark.network
async def test_external_api():
...
```
## TEST CONVENTIONS
### Security
- Never hardcode secrets - use environment variables
- Validate all inputs (URLs, filenames)
- Use HMAC for webhook verification when configured
- Limit CORS origins - never use `*` in production
- `tests/` directory with `conftest.py` for shared fixtures
- Markers: `@pytest.mark.unit`, `@pytest.mark.integration`, `@pytest.mark.network`, `@pytest.mark.slow`
- `asyncio_mode = auto` — async test functions run without explicit marker
- Test naming: `test_<verb>_<noun>` in `Test*` classes
- 300s timeout configured in pytest.ini; `testpaths = tests`
- Legacy test files at project root: `test_watchlist_simple.py`, `test_watchlist_e2e.py`
## Architecture Patterns
## ADDING NEW PROVIDERS
### Three-Tier Downloader Architecture
**Video player**: Create in `app/downloaders/video_players/`, inherit `BaseVideoPlayer`,
implement `can_handle()` + `get_download_link(url, target_filename=None)`, register in
`__init__.py`, add to `FILE_HOSTS` in `providers.py`.
The project uses a three-tier downloader system:
**Anime/series site**: Create in `app/downloaders/anime_sites/` or `series_sites/`, inherit
base class, implement `search_anime()` + `get_episodes()` + `get_anime_metadata()` +
`get_download_link()`, register in `__init__.py`, add to `providers.py`.
1. **Anime Catalogs** (`app/downloaders/anime_sites/`)
- `animesama.py` - Anime-Sama (primary)
- `animeultime.py` - Anime-Ultime
- `nekosama.py` - Neko-Sama
- `vostfree.py` - Vostfree
- `frenchmanga.py` - French-Manga
## NOTES
2. **Series Catalogs** (`app/downloaders/series_sites/`)
- `fs7.py` - French Stream
3. **Video Players** (`app/downloaders/video_players/`)
- `sibnet.py`, `doodstream.py`, `vidmoly.py`, `uqload.py`
- `uptobox.py`, `unfichier.py`, `rapidfile.py`
- `sendvid.py`, `lpayer.py`, `vidzy.py`, `luluv.py`
- `oneupload.py`, `smoothpre.py`
Each tier has a base class and factory pattern. When adding providers:
1. Inherit from appropriate base class (`base.py`)
2. Implement required methods (`search_anime`, `get_episodes`, `get_download_link`)
3. Register in `app/providers.py`
4. Add URL detection patterns
**URL Convention**: Pipe-separated format preserves metadata:
```
video_url|anime_page_url|episode_title
```
### Core Modules
| Module | Purpose |
|--------|---------|
| `app/watchlist.py` | Episode tracking & auto-download |
| `app/auto_download_scheduler.py` | APScheduler for periodic checks |
| `app/episode_checker.py` | New episode detection |
| `app/sonarr_handler.py` | Sonarr webhook integration |
| `app/recommendation_engine.py` | Personalized anime recommendations |
| `app/favorites.py` | User favorites management |
| `app/auth.py` | JWT authentication |
| `app/download_manager.py` | Download queue management |
## Key Files
| File | Purpose |
|------|---------|
| `main.py` | FastAPI app, all API endpoints |
| `app/config.py` | Pydantic Settings configuration |
| `app/download_manager.py` | Download queue & task management |
| `app/utils.py` | `sanitize_filename`, `is_safe_filename` |
| `app/auth.py` | JWT auth, user management |
| `app/providers.py` | Provider definitions & URL detection |
| `app/models/__init__.py` | Core Pydantic models |
| `app/models/watchlist.py` | Watchlist models |
| `app/models/sonarr.py` | Sonarr integration models |
| `app/models/auth.py` | Authentication models |
## Frontend Architecture
### JavaScript Modules (`static/js/`)
| Module | Purpose |
|--------|---------|
| `main.js` | Application entry point |
| `api.js` | API client functions |
| `auth.js` | Authentication handling |
| `tabs.js` | Tab navigation |
| `anime.js` | Anime search & display |
| `anime-details.js` | Anime detail views |
| `watchlist.js` | Watchlist API calls |
| `watchlist-ui.js` | Watchlist UI rendering |
| `downloads.js` | Download management UI |
| `recommendations.js` | Recommendations display |
| `series-search.js` | TV series search |
| `utils.js` | Utility functions |
### Templates (`templates/`)
| Template | Purpose |
|----------|---------|
| `base.html` | Base layout with CSS/JS imports |
| `index.html` | Main SPA interface |
| `login.html` | Login/register page |
| `watchlist.html` | Watchlist management page |
| `player.html` | Video player page |
| `components/` | Reusable HTML components |
## Configuration
- Use `.env` from `.env.example`
- `JWT_SECRET_KEY` must change in production
- Config files stored in `config/`:
- `users.json` - User database
- `watchlist.json` - Watchlist data
- `watchlist_settings.json` - Auto-download settings
- `sonarr.json` - Sonarr integration config
- `sonarr_mappings.json` - Series to anime mappings
## API Endpoints Overview
### Authentication
- `POST /api/auth/register` - Register new user
- `POST /api/auth/login` - Login, get JWT token
- `GET /api/auth/me` - Get current user info
- `POST /api/auth/logout` - Logout (client-side)
### Downloads
- `POST /api/download` - Create download task
- `GET /api/downloads` - List all downloads
- `GET /api/download/{task_id}` - Get download status
- `POST /api/download/{task_id}/pause` - Pause download
- `POST /api/download/{task_id}/resume` - Resume download
- `DELETE /api/download/{task_id}` - Cancel/delete download
- `GET /api/download/{task_id}/file` - Download completed file
### Anime Search & Metadata
- `GET /api/anime/search` - Search across all anime providers
- `GET /api/series/search` - Search TV series providers
- `GET /api/anime/metadata` - Get detailed anime metadata
- `GET /api/anime/episodes` - Get episode list
- `GET /api/anime/seasons` - Get available seasons
- `POST /api/anime/download-season` - Download all episodes
### Watchlist
- `GET /api/watchlist` - List watchlist items
- `POST /api/watchlist` - Add to watchlist
- `PUT /api/watchlist/{item_id}` - Update watchlist item
- `DELETE /api/watchlist/{item_id}` - Remove from watchlist
- `GET /api/watchlist/settings` - Get auto-download settings
- `PUT /api/watchlist/settings` - Update settings
- `POST /api/watchlist/check` - Trigger manual episode check
### Favorites
- `GET /api/favorites` - List favorites
- `POST /api/favorites` - Add to favorites
- `DELETE /api/favorites/{anime_id}` - Remove from favorites
- `POST /api/favorites/toggle` - Toggle favorite status
### Recommendations
- `GET /api/recommendations` - Get personalized recommendations
- `GET /api/releases/latest` - Get latest releases
- `GET /api/releases/seasonal` - Get seasonal anime
### Sonarr Integration
- `POST /api/sonarr/webhook` - Receive Sonarr webhooks
- `GET /api/sonarr/mappings` - List Sonarr mappings
- `POST /api/sonarr/mappings` - Create mapping
- `DELETE /api/sonarr/mappings/{series_id}` - Delete mapping
## Dependencies
### Core
- `fastapi` - Web framework
- `uvicorn` - ASGI server
- `httpx` - Async HTTP client
- `aiohttp` - Alternative HTTP client
- `pydantic` / `pydantic-settings` - Data validation & settings
### Scraping & Parsing
- `beautifulsoup4` - HTML parsing
- `lxml` - XML/HTML parser
- `jieba` - Chinese text segmentation
### Authentication
- `python-jose` - JWT handling
- `passlib[bcrypt]` - Password hashing
### Scheduler
- `apscheduler` - Job scheduling for auto-downloads
### Cryptography
- `pycryptodome` - AES decryption for video players
### Testing
- `pytest` + `pytest-asyncio` - Async test support
- `pytest-cov` - Coverage reporting
- `pytest-mock` - Mocking utilities
- `pytest-timeout` - Test timeout protection
- `pytest-html` - HTML test reports
## CI/CD
### GitHub Actions
This project uses GitHub Actions for continuous integration. The workflow is defined in `.github/workflows/ci.yml`.
**Workflow Features:**
- Runs on push and pull requests to `main` and `dev` branches
- Tests on Python 3.11 and 3.12
- Excludes slow tests by default (`-m "not slow"`)
- Generates HTML coverage reports
- Runs linting with ruff
- Runs type checking with mypy
**Artifacts:**
- Coverage reports are uploaded as artifacts after each run
- Access via the "Actions" tab on GitHub
**Running locally:**
```bash
# Run tests (excluding slow tests)
pytest -m "not slow" --cov=app --cov-report=html
# Run all tests including slow ones
pytest
# Run only unit tests
pytest -m "unit"
# Run only integration tests
pytest -m "integration"
# Run linting
pip install ruff && ruff check app/
# Run type checking
pip install mypy && mypy app/
```
- Python 3.11+, CI tests on 3.11 and 3.12
- No `pyproject.toml` — uses `requirements.txt` with exact version pinning
- `GEMINI.md` and `CLAUDE.md` exist with tool-specific instructions (merge if conflicting)
- French-language project (animes, séries, VOSTFR) but all code and comments in English
- ~20 empty `except:` blocks in downloaders/tests — known tech debt
- `JWT_SECRET_KEY` min 32 chars, default rejected at startup; generate via `Settings.generate_secret()`
- Sub-AGENTS.md files exist in `app/`, `app/routers/`, `app/downloaders/`, `app/models/`,
`app/downloaders/anime_sites/`, `app/downloaders/video_players/` for deeper context