diff --git a/README.md b/README.md index 4ed7f43..eebe0df 100644 --- a/README.md +++ b/README.md @@ -1,274 +1,127 @@ # Ohm Stream Downloader -**Application web complète pour télécharger des animes, séries TV et fichiers depuis divers hébergeurs.** +**Application web complète pour rechercher, streamer et télécharger des animes, séries TV et films.** -Interface moderne avec recherche unifiée, watchlist automatique, métadonnées enrichies, téléchargements parallèles et streaming vidéo. +Interface moderne (SPA-like) avec recherche unifiée, watchlist automatique, métadonnées enrichies, téléchargements parallèles et intégration Sonarr. ## ✨ Fonctionnalités -### 🎬 Recherche d'Animes & Séries TV -- **Recherche unifiée** : Recherchez animes et séries TV simultanément -- **Providers Anime** : Anime-Sama, Neko-Sama, Anime-Ultime, Vostfree, French-Manga -- **Providers Séries** : FS7 (French-Stream) -- **Métadonnées riches** : Synopsis, genres, notes, année de sortie, studio, nombre d'épisodes -- **Téléchargement par épisode** : Sélectionnez et téléchargez des épisodes individuels -- **Téléchargement de saison complète** : Téléchargez tous les épisodes d'un coup -- **Streaming vidéo** : Regardez vos animes directement dans le navigateur -- **Recherche floue** : Gestion des fautes de frappe et variations de noms +### 🎬 Recherche & Streaming +- **Recherche unifiée** : Recherchez animes et séries TV simultanément. +- **Providers Anime** : Anime-Sama, Neko-Sama, Anime-Ultime, Vostfree, French-Manga. +- **Providers Séries** : FS7 (French-Stream). +- **Métadonnées riches** : Synopsis, genres, notes, studio via intégration Kitsu. +- **Streaming vidéo** : Lecteur intégré supportant divers hébergeurs. +- **Téléchargement flexible** : Épisode par épisode ou saison complète. -### 📋 Watchlist (Suivi Automatique) -- **Ajout à la watchlist** : Suivez vos animes préférés depuis la recherche -- **Téléchargement automatique** : Télécharge tous les épisodes dès le suivi -- **Vérification automatique** : Le planificateur vérifie les nouveaux épisodes automatiquement -- **Intervalle configurable** : Paramétrez la fréquence de vérification (1-168 heures) -- **Notifications** : Recevez des alertes pour les nouveaux épisodes -- **Filtres** : Visualisez tous / actifs / en pause / terminés -- **Contrôle granulaire** : Pausez, reprenez, vérifiez manuellement chaque anime +### 📋 Watchlist & Automatisation +- **Suivi intelligent** : Ajoutez des animes à votre watchlist pour ne rater aucun épisode. +- **Auto-Download** : Téléchargement automatique des nouveaux épisodes dès leur sortie. +- **Planificateur (Scheduler)** : Vérification périodique configurable (1h à 168h). +- **Filtres avancés** : Visualisation par statut (Actif, En pause, Terminé). +- **Intégration Sonarr** : Support des webhooks pour une automatisation complète du homelab. -### 📁 Hébergeurs de Fichiers Supportés -- **1fichier** (1fichier.com, 1fichier.fr) -- **Uptobox** (uptobox.com, uptobox.fr) -- **Doodstream** (doodstream.com, dood.to, dood.lol, etc.) -- **Rapidfile** (rapidfile.net, rapidfile.com) -- **Uqload** (uqload.co, uqload.io) -- **OneUpload** (oneupload.co) -- **SendVid** (sendvid.com) -- **VidZ** (vidzi.tv) +### 🚀 Gestionnaire de Téléchargements +- **Multi-threading** : Jusqu'à 5 téléchargements simultanés avec gestion de file d'attente. +- **Pause/Reprise** : Support du protocole HTTP Range pour reprendre les téléchargements interrompus. +- **Progression Temps Réel** : Vitesse (Mo/s), pourcentage et estimation du temps restant. +- **Sanitisation** : Nettoyage automatique des noms de fichiers pour une compatibilité maximale. -### 🎥 Hébergeurs Vidéo Supportés -- **VidMoly** (vidmoly.to, vidmoly.com) -- **SendVid** (sendvid.com) -- **DoodStream** (doodstream.com) -- **LPlayer** (lplayer.net) -- **VidZy** (vidzy.tv) +## 🏗️ Architecture (Three-Tier System) -### 🚀 Gestion des Téléchargements -- **Téléchargements parallèles** : Hasta 5 téléchargements simultanés -- **Pause/Reprise** : Contrôle total sur vos téléchargements -- **Progression en temps réel** : Vitesse, progression, taille -- **Reprise automatique** : Support des HTTP Range pour reprendre les téléchargements interrompus +L'application repose sur un système à trois couches pour une robustesse maximale : +1. **Catalogues (Anime/Series Sites)** : Extraction des listes d'épisodes et métadonnées. +2. **Players Vidéo (Video Players)** : Extraction des liens de téléchargement direct depuis les embeds (VidMoly, DoodStream, etc.). +3. **Manager (Download Manager)** : Orchestration asynchrone des transferts de fichiers. -### 🌐 Interface Web -- **Design moderne** : Interface sombre avec onglets -- **5 onglets** : Accueil, Recherche, Séries, Providers, Watchlist -- **Responsive** : Fonctionne sur desktop et mobile -- **Mise à jour automatique** : Rafraîchissement automatique du contenu -- **Métadonnées visuelles** : Affichage des informations anime avec icônes +## 📁 Hébergeurs Supportés -### 🔌 API REST -- **Endpoints REST** : Intégration facile avec d'autres applications -- **Documentation automatique** : Swagger UI disponible +| Type | Services Supportés | +| :--- | :--- | +| **Catalogues** | Anime-Sama, Neko-Sama, Anime-Ultime, Vostfree, French-Manga, FS7 | +| **Players/Hosts** | VidMoly, DoodStream, 1fichier, Uptobox, SendVid, Sibnet, Lplayer, Uqload, Rapidfile, LuLuvid, Smoothpre, Vidzy | ## 📋 Configuration Requise -- Python 3.8+ -- pip +- **Python 3.11+** +- **Node.js** (pour les tests frontend uniquement) +- **Playwright** (pour l'extraction dynamique sur certains sites) -## 🚀 Installation +## 🚀 Installation Rapide ```bash # Cloner le repository git clone https://git.lanro.eu/Roman/ohm_streaming.git cd ohm_streaming -# Créer l'environnement virtuel +# Environnement Python python3 -m venv venv -source venv/bin/activate # Windows: venv\Scripts\activate - -# Installer les dépendances +source venv/bin/activate pip install -r requirements.txt -# Lancer le serveur de développement +# Initialisation Playwright (requis pour VidMoly) +playwright install chromium + +# Lancer l'application uvicorn main:app --reload --host 0.0.0.0 --port 3000 ``` +Accès Web : `http://localhost:3000/web` -Accédez à l'interface : http://localhost:3000/web +## 🧪 Tests & Qualité -## 📖 Utilisation - -### Interface Web - -1. **Onglet Accueil** : - - Dernières sorties anime et séries populaires - - Recommandations personnalisées - -2. **Onglet Recherche** : - - Entrez le nom d'un anime (ex: "Naruto", "Frieren") - - Sélectionnez la langue (VOSTFR ou VF) - - Cliquez sur "Rechercher" - - Sélectionnez un épisode et cliquez sur "Télécharger" - - Ou utilisez "Toute la saison" pour tout télécharger - - Cliquez sur "➕ Suivre" pour ajouter à la watchlist - -3. **Onglet Séries** : - - Recherchez des séries TV (Breaking Bad, Game of Thrones, etc.) - - Téléchargez des épisodes - -4. **Onglet Providers** : - - Liste des hébergeurs de fichiers disponibles - -5. **Onglet Watchlist** : - - Visualisez vos animes suivis - - Contrôlez le planificateur automatique - - Paramétrez l'intervalle de vérification - - Filtrez par statut (tous, actifs, en pause, terminés) - -### API Endpoints - -#### Authentication -| Méthode | Endpoint | Description | -|---------|----------|-------------| -| POST | `/api/auth/register` | Créer un compte | -| POST | `/api/auth/login` | Connexion (JWT) | -| GET | `/api/auth/me` | Profil utilisateur | - -#### Téléchargements -| Méthode | Endpoint | Description | -|---------|----------|-------------| -| POST | `/api/download` | Créer un nouveau téléchargement | -| GET | `/api/downloads` | Lister tous les téléchargements | -| GET | `/api/download/{task_id}` | Statut d'un téléchargement | -| POST | `/api/download/{task_id}/pause` | Mettre en pause | -| POST | `/api/download/{task_id}/resume` | Reprendre | -| DELETE | `/api/download/{task_id}` | Annuler/Supprimer | -| GET | `/api/download/{task_id}/file` | Télécharger le fichier terminé | - -#### Anime -| Méthode | Endpoint | Description | -|---------|----------|-------------| -| GET | `/api/anime/search` | Rechercher un anime | -| GET | `/api/anime/metadata` | Obtenir les métadonnées | -| GET | `/api/anime/episodes` | Liste des épisodes | -| POST | `/api/anime/download` | Télécharger un épisode | -| POST | `/api/anime/download-season` | Télécharger toute une saison | - -#### Watchlist -| Méthode | Endpoint | Description | -|---------|----------|-------------| -| GET | `/api/watchlist` | Liste des animes suivis | -| POST | `/api/watchlist` | Ajouter à la watchlist | -| DELETE | `/api/watchlist/{item_id}` | Supprimer de la watchlist | -| POST | `/api/watchlist/check-all` | Vérifier tous les animes | -| GET | `/api/watchlist/settings` | Paramètres | -| PUT | `/api/watchlist/settings` | Mettre à jour les paramètres | - -### Exemples API - -**Rechercher un anime :** ```bash -curl "http://localhost:3000/api/anime/search?q=frieren&lang=vostfr" -``` +# Backend (Pytest) +pytest # Tous les tests +pytest -m "unit" # Tests unitaires rapides -**Ajouter à la watchlist :** -```bash -curl -X POST http://localhost:3000/api/watchlist \ - -H "Authorization: Bearer YOUR_TOKEN" \ - -H "Content-Type: application/json" \ - -d '{"anime_title": "Frieren", "anime_url": "https://anime-sama.si/catalogue/frieren/saison1/vostfr/", "provider_id": "animesama", "lang": "vostfr"}' +# Frontend (Vitest & Playwright) +npm test # Tests unitaires JS +npx playwright test # Tests E2E ``` ## 🏗️ Structure du Projet ``` Ohm_streaming/ -├── main.py # Application FastAPI & endpoints API +├── main.py # Point d'entrée & API FastAPI ├── app/ -│ ├── models/ # Modèles Pydantic -│ ├── downloaders/ # Downloaders par provider -│ │ ├── anime_sites/ # Providers anime -│ │ └── series_sites/ # Providers séries -│ ├── providers.py # Configuration des providers -│ ├── download_manager.py # Gestionnaire de file d'attente -│ ├── watchlist.py # Gestion de la watchlist -│ ├── episode_checker.py # Vérification des nouveaux épisodes -│ └── auto_download_scheduler.py # Planificateur automatique -├── templates/ # Templates HTML -│ ├── index.html # Interface web principale -│ └── components/ # Composants réutilisables -├── static/ # Fichiers statiques (CSS, JS) -└── requirements.txt # Dépendances Python +│ ├── downloaders/ # Logique d'extraction (Scraping) +│ │ ├── anime_sites/ # Catalogues Anime +│ │ ├── series_sites/ # Catalogues Séries +│ │ └── video_players/ # Extracteurs de liens directs +│ ├── routers/ # Routes API modulaires (Auth, Watchlist, etc.) +│ ├── download_manager.py # Moteur de téléchargement asynchrone +│ ├── watchlist.py # Logique métier du suivi +│ └── scheduler.py # Planificateur de tâches +├── static/ # Frontend (JS Vanilla, CSS) +├── templates/ # Vues Jinja2 +└── config/ # Données persistantes (JSON) ``` -## ⚙️ Configuration - -Les paramètres peuvent être configurés via variables d'environnement ou fichier `.env` : - -``` -JWT_SECRET_KEY=votre-clé-secrète -DOWNLOAD_DIR=downloads -MAX_PARALLEL_DOWNLOADS=5 -``` - -## 🔧 Ajouter un Provider - -Voir la documentation dans le code source pour ajouter de nouveaux providers. - -## 🤝 Contribution - -Les contributions sont les bienvenues ! - -1. Fork le projet -2. Créez une branche (`git checkout -b feature/AmazingFeature`) -3. Commit (`git commit -m 'Add some AmazingFeature'`) -4. Push (`git push origin feature/AmazingFeature`) -5. Ouvrez une Pull Request - ## 🗺️ Plan d'Évolution Global (Modernisation) -Ce plan détaille les étapes nécessaires pour transformer Ohm Stream Downloader en une application de production robuste, sécurisée et évolutive. +### ✅ Phase 1 : Restructuration (Terminé) +- Migration vers une architecture modulaire pour les downloaders. +- Séparation stricte entre catalogues et hébergeurs vidéo. +- Amélioration de la gestion des erreurs et des retries. -### Phase 1 : Consolidation de la Donnée (Fondation) -*Objectif : Remplacer les fichiers JSON par une base de données relationnelle.* -- **Migration SQL** : Utiliser **SQLModel** (SQLAlchemy + Pydantic) pour gérer la persistance. - - Tables : `users`, `watchlist`, `tasks`, `favorites`, `settings`. -- **Gestion des Migrations** : Mettre en place **Alembic** pour suivre l'évolution du schéma sans perte de données. -- **Support Multi-base** : Configurer SQLite par défaut et PostgreSQL pour les déploiements avancés. +### ✅ Phase 2 : Consolidation & SQL (Terminé) +- Migration complète des fichiers JSON vers **SQLModel** (SQLite). +- Mise en place d'**Alembic** pour les migrations de base de données. +- Centralisation des métadonnées et persistance robuste. -### Phase 2 : Robustesse du Scraping (Cœur Technique) -*Objectif : Rendre l'extraction de données résiliente aux changements des sites tiers.* -- **Abstraction DSL (Domain Specific Language)** : Déporter les sélecteurs CSS et Regex dans des fichiers **YAML/JSON**. - - Permet de mettre à jour un provider sans modifier le code Python. -- **Découplage des Métadonnées** : Utiliser exclusivement les API de **Kitsu**, **Anilist** ou **MyAnimeList** pour les fiches d'animes. - - Le scraping ne sert plus qu'à récupérer les flux vidéo. -- **Health Checks Automatisés** : Script quotidien vérifiant que chaque provider répond toujours correctement (alerte en cas d'échec). -- **Service Playwright (Headless)** : Intégrer un service optionnel pour scraper les sites protégés par Cloudflare ou du JS complexe. -- **Résolution de Domaines (Auto-Mirrors)** : Détection automatique des changements de domaine (.si, .co, .pw) via DNS-over-HTTPS. +### 🏗️ Phase 3 : UX & Modernisation Frontend (En cours) +- Adoption de **HTMX/Alpine.js** pour dynamiser l'interface. +- Intégration du lecteur vidéo avancé **Plyr.io**. +- Amélioration de la réactivité de la recherche et de la watchlist. -### Phase 3 : Modernisation du Frontend (UX & Maintenance) -*Objectif : Simplifier le code JS et améliorer l'expérience utilisateur.* -- **Adoption de HTMX/Alpine.js** : Réduire la complexité du Vanilla JS en utilisant **HTMX** pour les mises à jour partielles (DOM diffing) et **Alpine.js** pour la réactivité légère. -- **Lecteur Vidéo Professionnel** : Intégrer **Plyr** ou **Video.js** pour supporter : - - Les sous-titres (.srt, .vtt). - - La gestion avancée du cache et du buffering. - - Une interface personnalisée et responsive. -- **Système de Toasts & Notifications** : Alertes visuelles pour la progression des tâches et les nouveaux épisodes détectés. -- **Design "Mobile First"** : Optimisation complète des CSS pour une utilisation fluide sur smartphone (PWA). +## 📝 Licence & Sécurité -### Phase 4 : Sécurité et DevOps (Professionnalisation) -*Objectif : Sécuriser les accès et faciliter le déploiement.* -- **Dockerisation Complète** : `docker-compose.yml` incluant App, Redis (cache), PostgreSQL et Playwright. -- **Journalisation Structurée** : Remplacer les `print` par un logger structuré (ex: `structlog`) avec rotation des logs. -- **Rate Limiting** : Protection des endpoints API contre le brute-force et le spam de recherche. -- **Gestion Stricte des Secrets** : Validation rigoureuse des variables d'environnement et suppression des IPs codées en dur (CORS). - -### Phase 5 : Nouvelles Fonctionnalités (Valeur Ajoutée) -*Objectif : Étendre les capacités du service.* -- **Transcodage à la volée** : Option FFmpeg pour convertir les .mkv incompatibles vers .mp4. -- **Bot de Notification** : Intégration Telegram/Discord pour être alerté dès qu'un épisode de la watchlist est téléchargé. -- **Multi-Utilisateurs Réel** : Bibliothèques et historiques de lecture totalement isolés par compte. -- **Support des Sous-titres Externes** : Upload de fichiers de sous-titres personnalisés pour le streaming. - -## 📝 Licence - -Ce projet est à usage éducatif uniquement. Respectez les droits d'auteur et les lois locales. - -## ⚠️ Avertissement - -Ce logiciel est destiné à un usage personnel et éducatif. Les utilisateurs sont responsables de vérifier qu'ils ont le droit de télécharger du contenu protégé par des droits d'auteur dans leur juridiction. +- Ce projet est à usage **éducatif et personnel** uniquement. +- Respectez les droits d'auteur et les conditions d'utilisation des sites sources. +- Ne partagez jamais votre `JWT_SECRET_KEY` en production. --- - +**Version actuelle : 2.3** +**Dernière mise à jour : Mars 2026** **Développé avec ❤️ pour la communauté anime** - -*Version actuelle : 2.2* -*Dernière mise à jour : Février 2026* diff --git a/alembic.ini b/alembic.ini new file mode 100644 index 0000000..c10d4ca --- /dev/null +++ b/alembic.ini @@ -0,0 +1,116 @@ +# A generic, single database configuration. + +[alembic] +# path to migration scripts +script_location = alembic + +# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s +# Uncomment the line below if you want the files to be prepended with date and time +# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file +# for all available tokens +# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s + +# sys.path path, will be prepended to sys.path if present. +# defaults to the current working directory. +prepend_sys_path = . + +# timezone to use when rendering the date within the migration file +# as well as the filename. +# If specified, requires the python>=3.9 or backports.zoneinfo library. +# Any required deps can installed by adding `alembic[tz]` to the pip requirements +# string value is passed to ZoneInfo() +# leave blank for localtime +# timezone = + +# max length of characters to apply to the +# "slug" field +# truncate_slug_length = 40 + +# set to 'true' to run the environment during +# the 'revision' command, regardless of autogenerate +# revision_environment = false + +# set to 'true' to allow .pyc and .pyo files without +# a source .py file to be detected as revisions in the +# versions/ directory +# sourceless = false + +# version location specification; This defaults +# to alembic/versions. When using multiple version +# directories, initial revisions must be specified with --version-path. +# The path separator used here should be the separator specified by "version_path_separator" below. +# version_locations = %(here)s/bar:%(here)s/bat:alembic/versions + +# version path separator; As mentioned above, this is the character used to split +# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep. +# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas. +# Valid values for version_path_separator are: +# +# version_path_separator = : +# version_path_separator = ; +# version_path_separator = space +version_path_separator = os # Use os.pathsep. Default configuration used for new projects. + +# set to 'true' to search source files recursively +# in each "version_locations" directory +# new in Alembic version 1.10 +# recursive_version_locations = false + +# the output encoding used when revision files +# are written from script.py.mako +# output_encoding = utf-8 + +sqlalchemy.url = driver://user:pass@localhost/dbname + + +[post_write_hooks] +# post_write_hooks defines scripts or Python functions that are run +# on newly generated revision scripts. See the documentation for further +# detail and examples + +# format using "black" - use the console_scripts runner, against the "black" entrypoint +# hooks = black +# black.type = console_scripts +# black.entrypoint = black +# black.options = -l 79 REVISION_SCRIPT_FILENAME + +# lint with attempts to fix using "ruff" - use the exec runner, execute a binary +# hooks = ruff +# ruff.type = exec +# ruff.executable = %(here)s/.venv/bin/ruff +# ruff.options = --fix REVISION_SCRIPT_FILENAME + +# Logging configuration +[loggers] +keys = root,sqlalchemy,alembic + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = WARN +handlers = console +qualname = + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine + +[logger_alembic] +level = INFO +handlers = +qualname = alembic + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(levelname)-5.5s [%(name)s] %(message)s +datefmt = %H:%M:%S diff --git a/alembic/README b/alembic/README new file mode 100644 index 0000000..98e4f9c --- /dev/null +++ b/alembic/README @@ -0,0 +1 @@ +Generic single-database configuration. \ No newline at end of file diff --git a/alembic/env.py b/alembic/env.py new file mode 100644 index 0000000..89e442e --- /dev/null +++ b/alembic/env.py @@ -0,0 +1,85 @@ +from logging.config import fileConfig + +from sqlalchemy import engine_from_config +from sqlalchemy import pool + +from alembic import context + +# this is the Alembic Config object, which provides +# access to the values within the .ini file in use. +config = context.config + +# Interpret the config file for Python logging. +# This line sets up loggers basically. +if config.config_file_name is not None: + fileConfig(config.config_file_name) + +# add your model's MetaData object here +# for 'autogenerate' support +from sqlmodel import SQLModel +import app.models.auth +import app.models.watchlist +import app.models.favorites +import app.models.sonarr +from app.database import DATABASE_URL +target_metadata = SQLModel.metadata + +# Set the sqlalchemy.url to the one we use in our app +config.set_main_option("sqlalchemy.url", DATABASE_URL) + +# other values from the config, defined by the needs of env.py, +# can be acquired: +# my_important_option = config.get_main_option("my_important_option") +# ... etc. + + +def run_migrations_offline() -> None: + """Run migrations in 'offline' mode. + + This configures the context with just a URL + and not an Engine, though an Engine is acceptable + here as well. By skipping the Engine creation + we don't even need a DBAPI to be available. + + Calls to context.execute() here emit the given string to the + script output. + + """ + url = config.get_main_option("sqlalchemy.url") + context.configure( + url=url, + target_metadata=target_metadata, + literal_binds=True, + dialect_opts={"paramstyle": "named"}, + ) + + with context.begin_transaction(): + context.run_migrations() + + +def run_migrations_online() -> None: + """Run migrations in 'online' mode. + + In this scenario we need to create an Engine + and associate a connection with the context. + + """ + connectable = engine_from_config( + config.get_section(config.config_ini_section, {}), + prefix="sqlalchemy.", + poolclass=pool.NullPool, + ) + + with connectable.connect() as connection: + context.configure( + connection=connection, target_metadata=target_metadata + ) + + with context.begin_transaction(): + context.run_migrations() + + +if context.is_offline_mode(): + run_migrations_offline() +else: + run_migrations_online() diff --git a/alembic/script.py.mako b/alembic/script.py.mako new file mode 100644 index 0000000..fbc4b07 --- /dev/null +++ b/alembic/script.py.mako @@ -0,0 +1,26 @@ +"""${message} + +Revision ID: ${up_revision} +Revises: ${down_revision | comma,n} +Create Date: ${create_date} + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa +${imports if imports else ""} + +# revision identifiers, used by Alembic. +revision: str = ${repr(up_revision)} +down_revision: Union[str, None] = ${repr(down_revision)} +branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)} +depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)} + + +def upgrade() -> None: + ${upgrades if upgrades else "pass"} + + +def downgrade() -> None: + ${downgrades if downgrades else "pass"} diff --git a/alembic/versions/e0273f326a15_initial_migration.py b/alembic/versions/e0273f326a15_initial_migration.py new file mode 100644 index 0000000..391c491 --- /dev/null +++ b/alembic/versions/e0273f326a15_initial_migration.py @@ -0,0 +1,30 @@ +"""Initial migration + +Revision ID: e0273f326a15 +Revises: +Create Date: 2026-03-24 17:05:50.046027 + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + + +# revision identifiers, used by Alembic. +revision: str = 'e0273f326a15' +down_revision: Union[str, None] = None +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + pass + # ### end Alembic commands ### + + +def downgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + pass + # ### end Alembic commands ### diff --git a/alembic/versions/e88271d11851_add_watchlistsettingstable.py b/alembic/versions/e88271d11851_add_watchlistsettingstable.py new file mode 100644 index 0000000..757a3d3 --- /dev/null +++ b/alembic/versions/e88271d11851_add_watchlistsettingstable.py @@ -0,0 +1,30 @@ +"""Add WatchlistSettingsTable + +Revision ID: e88271d11851 +Revises: e0273f326a15 +Create Date: 2026-03-24 17:07:10.189457 + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + + +# revision identifiers, used by Alembic. +revision: str = 'e88271d11851' +down_revision: Union[str, None] = 'e0273f326a15' +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + pass + # ### end Alembic commands ### + + +def downgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + pass + # ### end Alembic commands ### diff --git a/app/auth.py b/app/auth.py index 8df8da6..3872dc7 100644 --- a/app/auth.py +++ b/app/auth.py @@ -32,6 +32,7 @@ class UserManager: def get_user(self, username: str) -> Optional[UserTable]: """Get user by username""" + from app.models.watchlist import WatchlistItemTable # Force registration with Session(engine) as session: statement = select(UserTable).where(UserTable.username == username) return session.exec(statement).first() @@ -210,6 +211,14 @@ def _save_refresh_tokens(tokens: Dict[str, dict]): logger.error(f"Error saving refresh tokens: {e}") +def _get_jwt_config() -> dict: + return { + "SECRET_KEY": settings.jwt_secret_key, + "ALGORITHM": settings.jwt_algorithm, + "ACCESS_TOKEN_EXPIRE_MINUTES": settings.access_token_expire_minutes, + "REFRESH_TOKEN_EXPIRE_DAYS": 30 + } + def create_access_refresh_tokens(data: dict) -> tuple[str, str]: """ Create both access and refresh tokens. diff --git a/app/database.py b/app/database.py index 82f7430..50cdf26 100644 --- a/app/database.py +++ b/app/database.py @@ -17,10 +17,11 @@ engine = create_engine(DATABASE_URL, connect_args={"check_same_thread": False}) def create_db_and_tables(): """Create the database and tables based on the models""" - # Import all models here to ensure they are registered with SQLModel.metadata + # CRITICAL: Import ALL models here to ensure they are registered with SQLModel.metadata from app.models.auth import UserTable - from app.models.watchlist import WatchlistItemTable - # Add other models as they are migrated + from app.models.watchlist import WatchlistItemTable, WatchlistSettingsTable + from app.models.favorites import FavoriteTable + from app.models.sonarr import SonarrMappingTable, SonarrConfigTable SQLModel.metadata.create_all(engine) diff --git a/app/favorites.py b/app/favorites.py index b78be94..cfd80be 100644 --- a/app/favorites.py +++ b/app/favorites.py @@ -1,52 +1,24 @@ """ Favorites management system for Ohm Stream Downloader -Stores user's favorite anime with metadata in a local JSON file +Stores user's favorite anime with metadata using SQLModel """ -import json -import asyncio import logging -from pathlib import Path from typing import List, Dict, Optional from datetime import datetime -import aiofiles + +from sqlmodel import Session, select +from app.database import engine +from app.models.favorites import FavoriteTable logger = logging.getLogger(__name__) class FavoritesManager: - """Manages user's favorite anime list""" + """Manages user's favorite anime list using SQL database""" - def __init__(self, storage_path: str = "data/favorites.json"): - self.storage_path = Path(storage_path) - self.storage_path.parent.mkdir(parents=True, exist_ok=True) - self._favorites: Dict[str, Dict] = {} - self._lock = asyncio.Lock() - - async def _load(self): - """Load favorites from disk""" - async with self._lock: - await self._load_for_operation() - - async def _load_for_operation(self): - """Load favorites from disk without acquiring lock (lock must already be held)""" - if self.storage_path.exists(): - try: - async with aiofiles.open(self.storage_path, 'r', encoding='utf-8') as f: - content = await f.read() - self._favorites = json.loads(content) if content.strip() else {} - except Exception as e: - logger.error(f"Error loading favorites: {e}") - self._favorites = {} - else: - self._favorites = {} - - async def _save(self): - """Save favorites to disk (assumes lock is already held)""" - try: - async with aiofiles.open(self.storage_path, 'w', encoding='utf-8') as f: - await f.write(json.dumps(self._favorites, indent=2, ensure_ascii=False)) - except Exception as e: - logger.error(f"Error saving favorites: {e}") + def __init__(self, storage_path: str = None): + # Database connection is managed via engine and sessions + pass async def add_favorite( self, @@ -58,48 +30,55 @@ class FavoritesManager: poster_url: Optional[str] = None ) -> Dict: """Add an anime to favorites""" - async with self._lock: - await self._load_for_operation() + with Session(engine) as session: + statement = select(FavoriteTable).where(FavoriteTable.anime_id == anime_id) + existing = session.exec(statement).first() - if anime_id in self._favorites: + if existing: # Update existing favorite - self._favorites[anime_id]["updated_at"] = datetime.now().isoformat() + existing.updated_at = datetime.now() if metadata: - self._favorites[anime_id]["metadata"] = metadata + existing.anime_metadata = metadata if poster_url: - self._favorites[anime_id]["poster_url"] = poster_url + existing.poster_url = poster_url + session.add(existing) + session.commit() + session.refresh(existing) + return self._to_dict(existing) else: # Add new favorite - self._favorites[anime_id] = { - "id": anime_id, - "title": title, - "url": url, - "provider": provider, - "metadata": metadata or {}, - "poster_url": poster_url, - "created_at": datetime.now().isoformat(), - "updated_at": datetime.now().isoformat() - } - - await self._save() - return self._favorites[anime_id] + fav = FavoriteTable( + anime_id=anime_id, + title=title, + url=url, + provider=provider, + anime_metadata=metadata or {}, + poster_url=poster_url + ) + session.add(fav) + session.commit() + session.refresh(fav) + return self._to_dict(fav) async def remove_favorite(self, anime_id: str) -> bool: """Remove an anime from favorites""" - async with self._lock: - await self._load_for_operation() - - if anime_id in self._favorites: - del self._favorites[anime_id] - await self._save() + with Session(engine) as session: + statement = select(FavoriteTable).where(FavoriteTable.anime_id == anime_id) + existing = session.exec(statement).first() + if existing: + session.delete(existing) + session.commit() return True - return False async def get_favorite(self, anime_id: str) -> Optional[Dict]: """Get a specific favorite by ID""" - await self._load() - return self._favorites.get(anime_id) + with Session(engine) as session: + statement = select(FavoriteTable).where(FavoriteTable.anime_id == anime_id) + existing = session.exec(statement).first() + if existing: + return self._to_dict(existing) + return None async def list_favorites( self, @@ -109,13 +88,15 @@ class FavoritesManager: filter_genre: Optional[str] = None ) -> List[Dict]: """List all favorites with optional sorting and filtering""" - await self._load() - - favorites = list(self._favorites.values()) - - # Apply filters - if filter_provider: - favorites = [f for f in favorites if f["provider"] == filter_provider] + with Session(engine) as session: + statement = select(FavoriteTable) + + if filter_provider: + statement = statement.where(FavoriteTable.provider == filter_provider) + + # SQLite JSON filtering for genres is complex, handle it in Python + results = session.exec(statement).all() + favorites = [self._to_dict(fav) for fav in results] if filter_genre: favorites = [ @@ -144,8 +125,9 @@ class FavoritesManager: async def is_favorite(self, anime_id: str) -> bool: """Check if an anime is in favorites""" - await self._load() - return anime_id in self._favorites + with Session(engine) as session: + statement = select(FavoriteTable).where(FavoriteTable.anime_id == anime_id) + return session.exec(statement).first() is not None async def toggle_favorite( self, @@ -168,19 +150,18 @@ class FavoritesManager: async def get_stats(self) -> Dict: """Get statistics about favorites""" - await self._load() - - total = len(self._favorites) + favorites = await self.list_favorites() + total = len(favorites) # Count by provider by_provider = {} - for fav in self._favorites.values(): + for fav in favorites: provider = fav["provider"] by_provider[provider] = by_provider.get(provider, 0) + 1 # Count by genre by_genre = {} - for fav in self._favorites.values(): + for fav in favorites: for genre in fav.get("metadata", {}).get("genres", []): by_genre[genre] = by_genre.get(genre, 0) + 1 @@ -190,6 +171,19 @@ class FavoritesManager: "by_genre": by_genre } + def _to_dict(self, fav: FavoriteTable) -> Dict: + """Convert a FavoriteTable instance to a dictionary for API compatibility""" + return { + "id": fav.anime_id, + "title": fav.title, + "url": fav.url, + "provider": fav.provider, + "metadata": fav.anime_metadata, + "poster_url": fav.poster_url, + "created_at": fav.created_at.isoformat() if fav.created_at else None, + "updated_at": fav.updated_at.isoformat() if fav.updated_at else None + } + # Global favorites manager instance _favorites_manager: Optional[FavoritesManager] = None diff --git a/app/models/__init__.py b/app/models/__init__.py index c0e79b7..3363b09 100644 --- a/app/models/__init__.py +++ b/app/models/__init__.py @@ -63,3 +63,9 @@ class AnimeSearchResult(BaseModel): cover_image: Optional[str] = None type: str # "search_result" or "direct" metadata: Optional[AnimeMetadata] = None + +# Import all SQLModel tables here to ensure they are registered together +from .auth import UserTable +from .watchlist import WatchlistItemTable, WatchlistSettingsTable +from .favorites import FavoriteTable +from .sonarr import SonarrMappingTable, SonarrConfigTable diff --git a/app/models/auth.py b/app/models/auth.py index f65aac0..75e3d93 100644 --- a/app/models/auth.py +++ b/app/models/auth.py @@ -28,7 +28,7 @@ class UserTable(UserBase, table=True): created_at: datetime = Field(default_factory=datetime.now) last_login: Optional[datetime] = None - # Relationships + # Relationships - Using string reference to avoid circular import errors watchlist_items: List["WatchlistItemTable"] = Relationship(back_populates="user") @@ -60,3 +60,6 @@ class Token(BaseModel): class UserInDB(User): """Schema for user stored in database (with hashed password)""" hashed_password: str + +# Import WatchlistItemTable here to resolve SQLModel Relationship mappings +from .watchlist import WatchlistItemTable diff --git a/app/models/favorites.py b/app/models/favorites.py new file mode 100644 index 0000000..5ca5834 --- /dev/null +++ b/app/models/favorites.py @@ -0,0 +1,44 @@ +"""Models for Favorites system with SQLModel support""" +import uuid +import json +from typing import Optional, Dict, List +from datetime import datetime +from sqlmodel import SQLModel, Field, Column, String + +class FavoriteBase(SQLModel): + """Base schema for favorite anime""" + anime_id: str = Field(index=True) + title: str = Field(index=True) + url: str + provider: str + poster_url: Optional[str] = None + + # Timestamps + created_at: datetime = Field(default_factory=datetime.now) + updated_at: datetime = Field(default_factory=datetime.now) + +class FavoriteTable(FavoriteBase, table=True): + """Database table for favorites""" + __tablename__ = "favorites" + + id: str = Field( + default_factory=lambda: str(uuid.uuid4()), + primary_key=True, + index=True, + nullable=False + ) + user_id: str = Field(foreign_key="users.id", index=True, default="default") + + # Store metadata dictionary as JSON string in SQLite + metadata_json: Optional[str] = Field(default="{}", sa_column=Column(String)) + + @property + def anime_metadata(self) -> Dict: + try: + return json.loads(self.metadata_json or "{}") + except json.JSONDecodeError: + return {} + + @anime_metadata.setter + def anime_metadata(self, value: Dict): + self.metadata_json = json.dumps(value or {}) diff --git a/app/models/sonarr.py b/app/models/sonarr.py index 6fd425b..e603f44 100644 --- a/app/models/sonarr.py +++ b/app/models/sonarr.py @@ -1,8 +1,10 @@ """Pydantic models for Sonarr webhook integration""" -from pydantic import BaseModel, Field, validator +from pydantic import BaseModel, Field as PydanticField, validator from typing import Optional, Dict, Any, List from datetime import datetime from enum import Enum +from sqlmodel import SQLModel, Field +import uuid class SonarrEventType(str, Enum): @@ -45,7 +47,7 @@ class SonarrEpisodeFile(BaseModel): class SonarrSeries(BaseModel): """Series information from Sonarr""" - tvdbId: int = Field(..., alias="tvdbId") + tvdbId: int = PydanticField(..., alias="tvdbId") title: str sortTitle: str status: str @@ -129,8 +131,33 @@ class SonarrWebhookPayload(BaseModel): return v +class SonarrMappingBase(SQLModel): + sonarr_series_id: int = Field(index=True, unique=True) + sonarr_title: str + anime_provider: str + anime_url: str + anime_title: str + lang: str = Field(default="vostfr") + quality_preference: Optional[str] = None + auto_download: bool = Field(default=True) + created_at: datetime = Field(default_factory=datetime.now) + updated_at: datetime = Field(default_factory=datetime.now) + + +class SonarrMappingTable(SonarrMappingBase, table=True): + """Database table for Sonarr mappings""" + __tablename__ = "sonarr_mappings" + id: str = Field( + default_factory=lambda: str(uuid.uuid4()), + primary_key=True, + index=True, + nullable=False + ) + user_id: str = Field(foreign_key="users.id", index=True, default="default") + + class SonarrMapping(BaseModel): - """Mapping between Sonarr series and anime providers""" + """Mapping between Sonarr series and anime providers (API model)""" sonarr_series_id: int sonarr_title: str anime_provider: str # 'anime-sama', 'neko-sama', etc. @@ -139,8 +166,8 @@ class SonarrMapping(BaseModel): lang: str = "vostfr" quality_preference: Optional[str] = None # '1080p', '720p', etc. auto_download: bool = True - created_at: datetime = Field(default_factory=datetime.now) - updated_at: datetime = Field(default_factory=datetime.now) + created_at: datetime = PydanticField(default_factory=datetime.now) + updated_at: datetime = PydanticField(default_factory=datetime.now) class Config: json_encoders = { @@ -148,8 +175,30 @@ class SonarrMapping(BaseModel): } +class SonarrConfigBase(SQLModel): + webhook_enabled: bool = Field(default=False) + webhook_secret: Optional[str] = None + auto_download_enabled: bool = Field(default=True) + default_language: str = Field(default="vostfr") + default_quality: Optional[str] = None + default_provider: str = Field(default="anime-sama") + verify_hmac: bool = Field(default=False) + log_webhooks: bool = Field(default=True) + + +class SonarrConfigTable(SonarrConfigBase, table=True): + """Database table for Sonarr configuration (singleton)""" + __tablename__ = "sonarr_config" + id: str = Field( + default_factory=lambda: str(uuid.uuid4()), + primary_key=True, + index=True, + nullable=False + ) + + class SonarrConfig(BaseModel): - """Sonarr webhook configuration""" + """Sonarr webhook configuration (API Model)""" webhook_enabled: bool = False webhook_secret: Optional[str] = None # HMAC SHA256 secret auto_download_enabled: bool = True diff --git a/app/models/watchlist.py b/app/models/watchlist.py index 8de77e5..1a0fec1 100644 --- a/app/models/watchlist.py +++ b/app/models/watchlist.py @@ -74,7 +74,7 @@ class WatchlistItemTable(WatchlistItemBase, table=True): def genres(self, value: List[str]): self.genres_json = json.dumps(value or []) - # Relationships + # Relationships - Using string reference user: Optional["UserTable"] = Relationship(back_populates="watchlist_items") @@ -148,6 +148,24 @@ class AutoDownloadResult(BaseModel): checked_at: datetime = PydanticField(default_factory=datetime.now) +class WatchlistSettingsBase(SQLModel): + check_interval_hours: int = Field(default=6) + auto_download_enabled: bool = Field(default=True) + max_concurrent_auto_downloads: int = Field(default=2) + notify_on_new_episodes: bool = Field(default=False) + include_completed_anime: bool = Field(default=False) + +class WatchlistSettingsTable(WatchlistSettingsBase, table=True): + """Database table for global watchlist settings""" + __tablename__ = "watchlist_settings" + id: str = Field( + default_factory=lambda: str(uuid.uuid4()), + primary_key=True, + index=True, + nullable=False + ) + user_id: str = Field(foreign_key="users.id", index=True, default="default") + class WatchlistSettings(BaseModel): """Global watchlist settings""" check_interval_hours: int = PydanticField(default=6, ge=1, le=168) @@ -155,3 +173,6 @@ class WatchlistSettings(BaseModel): max_concurrent_auto_downloads: int = PydanticField(default=2, ge=1, le=10) notify_on_new_episodes: bool = PydanticField(default=False) include_completed_anime: bool = PydanticField(default=False) + +# Import UserTable here to resolve SQLModel Relationship mappings +from .auth import UserTable diff --git a/app/sonarr_handler.py b/app/sonarr_handler.py index 317c9b8..3f3d32d 100644 --- a/app/sonarr_handler.py +++ b/app/sonarr_handler.py @@ -1,18 +1,19 @@ -"""Sonarr webhook handler and integration logic""" +"""Sonarr webhook handler and integration logic using SQLModel""" import hmac import hashlib -import json import logging -import os -from typing import Optional, Dict, List, Tuple, Any -from pathlib import Path +from typing import Optional, Dict, List, Any from datetime import datetime +from sqlmodel import Session, select +from app.database import engine from app.models.sonarr import ( SonarrWebhookPayload, SonarrEventType, SonarrMapping, + SonarrMappingTable, SonarrConfig, + SonarrConfigTable, SonarrDownloadRequest ) from app.models import DownloadRequest @@ -23,69 +24,150 @@ logger = logging.getLogger(__name__) class SonarrHandler: - """Handles Sonarr webhooks and manages series mappings""" + """Handles Sonarr webhooks and manages series mappings using SQL database""" - def __init__(self, config_path: str = "config/sonarr.json", mappings_path: str = "config/sonarr_mappings.json"): - self.config_path = Path(config_path) - self.mappings_path = Path(mappings_path) - self.config = self._load_config() - self.mappings = self._load_mappings() + def __init__(self, config_path: str = None, mappings_path: str = None): self.download_manager = None - - # Create config directories if they don't exist - self.config_path.parent.mkdir(exist_ok=True) - self.mappings_path.parent.mkdir(exist_ok=True) + self._ensure_default_config() def set_download_manager(self, download_manager): self.download_manager = download_manager - def _load_config(self) -> SonarrConfig: - """Load Sonarr configuration from file""" - if self.config_path.exists(): - try: - with open(self.config_path, 'r') as f: - data = json.load(f) - return SonarrConfig(**data) - except Exception as e: - logger.warning(f"Failed to load Sonarr config: {e}") - return SonarrConfig() + def _ensure_default_config(self): + """Ensure a default config exists in the database""" + with Session(engine) as session: + statement = select(SonarrConfigTable) + if not session.exec(statement).first(): + session.add(SonarrConfigTable()) + session.commit() - def _save_config(self): - try: - temp_file = f"{self.config_path}.tmp" - with open(temp_file, 'w') as f: - json.dump(self.config.model_dump(mode='json'), f, indent=2) - os.replace(temp_file, self.config_path) - except Exception as e: - logger.error(f"Failed to save Sonarr config: {e}") - raise + def get_config(self) -> SonarrConfig: + """Get current configuration""" + with Session(engine) as session: + statement = select(SonarrConfigTable) + db_config = session.exec(statement).first() + if db_config: + return SonarrConfig( + webhook_enabled=db_config.webhook_enabled, + webhook_secret=db_config.webhook_secret, + auto_download_enabled=db_config.auto_download_enabled, + default_language=db_config.default_language, + default_quality=db_config.default_quality, + default_provider=db_config.default_provider, + verify_hmac=db_config.verify_hmac, + log_webhooks=db_config.log_webhooks + ) + return SonarrConfig() - def _load_mappings(self) -> List[SonarrMapping]: - """Load Sonarr to anime mappings from file""" - if self.mappings_path.exists(): - try: - with open(self.mappings_path, 'r') as f: - data = json.load(f) - return [SonarrMapping(**item) for item in data] - except Exception as e: - logger.warning(f"Failed to load Sonarr mappings: {e}") - return [] + def update_config(self, config: SonarrConfig) -> SonarrConfig: + """Update configuration""" + with Session(engine) as session: + statement = select(SonarrConfigTable) + db_config = session.exec(statement).first() + + if not db_config: + db_config = SonarrConfigTable() + + db_config.webhook_enabled = config.webhook_enabled + db_config.webhook_secret = config.webhook_secret + db_config.auto_download_enabled = config.auto_download_enabled + db_config.default_language = config.default_language + db_config.default_quality = config.default_quality + db_config.default_provider = config.default_provider + db_config.verify_hmac = config.verify_hmac + db_config.log_webhooks = config.log_webhooks + + session.add(db_config) + session.commit() + + logger.info("Sonarr configuration updated in database") + return config - def _save_mappings(self): - try: - os.makedirs(os.path.dirname(self.mappings_path), exist_ok=True) - temp_file = f"{self.mappings_path}.tmp" - with open(temp_file, 'w') as f: - mappings_data = [m.model_dump(mode='json') for m in self.mappings] - json.dump(mappings_data, f, indent=2) - os.replace(temp_file, self.mappings_path) - except Exception as e: - logger.error(f"Failed to save mappings: {e}") - raise + def _to_pydantic(self, db_mapping: SonarrMappingTable) -> SonarrMapping: + return SonarrMapping( + sonarr_series_id=db_mapping.sonarr_series_id, + sonarr_title=db_mapping.sonarr_title, + anime_provider=db_mapping.anime_provider, + anime_url=db_mapping.anime_url, + anime_title=db_mapping.anime_title, + lang=db_mapping.lang, + quality_preference=db_mapping.quality_preference, + auto_download=db_mapping.auto_download, + created_at=db_mapping.created_at, + updated_at=db_mapping.updated_at + ) + + def get_mappings(self) -> List[SonarrMapping]: + """Get all mappings""" + with Session(engine) as session: + statement = select(SonarrMappingTable) + db_mappings = session.exec(statement).all() + return [self._to_pydantic(m) for m in db_mappings] + + def get_mapping(self, sonarr_series_id: int) -> Optional[SonarrMapping]: + """Get mapping for specific series""" + with Session(engine) as session: + statement = select(SonarrMappingTable).where(SonarrMappingTable.sonarr_series_id == sonarr_series_id) + db_mapping = session.exec(statement).first() + if db_mapping: + return self._to_pydantic(db_mapping) + return None + + def add_mapping(self, mapping: SonarrMapping) -> SonarrMapping: + """Add or update a mapping""" + with Session(engine) as session: + statement = select(SonarrMappingTable).where(SonarrMappingTable.sonarr_series_id == mapping.sonarr_series_id) + db_mapping = session.exec(statement).first() + + if db_mapping: + # Update existing + db_mapping.sonarr_title = mapping.sonarr_title + db_mapping.anime_provider = mapping.anime_provider + db_mapping.anime_url = mapping.anime_url + db_mapping.anime_title = mapping.anime_title + db_mapping.lang = mapping.lang + db_mapping.quality_preference = mapping.quality_preference + db_mapping.auto_download = mapping.auto_download + db_mapping.updated_at = datetime.now() + logger.info(f"Updated mapping for series {mapping.sonarr_title}") + else: + # Create new + db_mapping = SonarrMappingTable( + user_id="default", + sonarr_series_id=mapping.sonarr_series_id, + sonarr_title=mapping.sonarr_title, + anime_provider=mapping.anime_provider, + anime_url=mapping.anime_url, + anime_title=mapping.anime_title, + lang=mapping.lang, + quality_preference=mapping.quality_preference, + auto_download=mapping.auto_download, + created_at=datetime.now(), + updated_at=datetime.now() + ) + logger.info(f"Added mapping for series {mapping.sonarr_title}") + + session.add(db_mapping) + session.commit() + session.refresh(db_mapping) + return self._to_pydantic(db_mapping) + + def delete_mapping(self, sonarr_series_id: int) -> bool: + """Delete a mapping""" + with Session(engine) as session: + statement = select(SonarrMappingTable).where(SonarrMappingTable.sonarr_series_id == sonarr_series_id) + db_mapping = session.exec(statement).first() + if db_mapping: + session.delete(db_mapping) + session.commit() + logger.info(f"Deleted mapping for series ID {sonarr_series_id}") + return True + return False def verify_hmac(self, payload: bytes, signature: str) -> bool: """Verify HMAC SHA256 signature""" - if not self.config.verify_hmac or not self.config.webhook_secret: + config = self.get_config() + if not config.verify_hmac or not config.webhook_secret: return True try: @@ -94,7 +176,7 @@ class SonarrHandler: signature = signature[7:] computed_hmac = hmac.new( - self.config.webhook_secret.encode(), + config.webhook_secret.encode(), payload, hashlib.sha256 ).hexdigest() @@ -104,57 +186,6 @@ class SonarrHandler: logger.error(f"HMAC verification failed: {e}") return False - def get_config(self) -> SonarrConfig: - """Get current configuration""" - return self.config - - def update_config(self, config: SonarrConfig) -> SonarrConfig: - """Update configuration""" - self.config = config - self._save_config() - logger.info("Sonarr configuration updated") - return self.config - - def get_mappings(self) -> List[SonarrMapping]: - """Get all mappings""" - return self.mappings - - def get_mapping(self, sonarr_series_id: int) -> Optional[SonarrMapping]: - """Get mapping for specific series""" - for mapping in self.mappings: - if mapping.sonarr_series_id == sonarr_series_id: - return mapping - return None - - def add_mapping(self, mapping: SonarrMapping) -> SonarrMapping: - """Add or update a mapping""" - # Check if mapping already exists - for i, existing in enumerate(self.mappings): - if existing.sonarr_series_id == mapping.sonarr_series_id: - mapping.updated_at = datetime.now() - self.mappings[i] = mapping - self._save_mappings() - logger.info(f"Updated mapping for series {mapping.sonarr_title}") - return mapping - - # Add new mapping - mapping.created_at = datetime.now() - mapping.updated_at = datetime.now() - self.mappings.append(mapping) - self._save_mappings() - logger.info(f"Added mapping for series {mapping.sonarr_title}") - return mapping - - def delete_mapping(self, sonarr_series_id: int) -> bool: - """Delete a mapping""" - for i, mapping in enumerate(self.mappings): - if mapping.sonarr_series_id == sonarr_series_id: - del self.mappings[i] - self._save_mappings() - logger.info(f"Deleted mapping for series ID {sonarr_series_id}") - return True - return False - async def search_anime_by_title(self, title: str, provider: str = "anime-sama", lang: str = "vostfr") -> List[Dict]: """Search for anime by title using specified provider""" try: @@ -197,15 +228,16 @@ class SonarrHandler: async def process_webhook(self, payload: SonarrWebhookPayload) -> Dict[str, Any]: """Process Sonarr webhook payload""" - if not self.config.webhook_enabled: + config = self.get_config() + if not config.webhook_enabled: return {"status": "ignored", "reason": "Webhook not enabled"} - if self.config.log_webhooks: + if config.log_webhooks: logger.info(f"Received Sonarr webhook: {payload.eventType.value}") # Handle different event types if payload.eventType == SonarrEventType.GRAB: - return await self._handle_grab(payload) + return await self._handle_grab(payload, config) elif payload.eventType == SonarrEventType.DOWNLOAD: return await self._handle_download(payload) elif payload.eventType == SonarrEventType.RENAME: @@ -217,9 +249,9 @@ class SonarrHandler: else: return {"status": "ignored", "reason": f"Unhandled event type: {payload.eventType}"} - async def _handle_grab(self, payload: SonarrWebhookPayload) -> Dict: + async def _handle_grab(self, payload: SonarrWebhookPayload, config: SonarrConfig) -> Dict: """Handle Grab event (when Sonarr downloads a release)""" - if not self.config.auto_download_enabled: + if not config.auto_download_enabled: return {"status": "ignored", "reason": "Auto-download disabled"} if not payload.series or not payload.episodes: diff --git a/app/watchlist.py b/app/watchlist.py index 2227359..e1480a4 100644 --- a/app/watchlist.py +++ b/app/watchlist.py @@ -16,50 +16,70 @@ from app.models.watchlist import ( WatchlistItemUpdate, WatchlistStatus, WatchlistSettings, + WatchlistSettingsTable, NewEpisodeInfo, AutoDownloadResult ) logger = logging.getLogger(__name__) -# Settings file remains JSON for simplicity for now -WATCHLIST_SETTINGS_FILE = "config/watchlist_settings.json" - - class WatchlistManager: """Manages user watchlist for automatic episode downloads using SQL database""" def __init__(self): - self.settings_file = WATCHLIST_SETTINGS_FILE self.settings: Optional[WatchlistSettings] = None self._load_settings() def _load_settings(self): - """Load watchlist settings from JSON file""" + """Load watchlist settings from database""" try: - if os.path.exists(self.settings_file): - with open(self.settings_file, 'r', encoding='utf-8') as f: - data = json.load(f) - self.settings = WatchlistSettings(**data) - logger.info(f"Loaded watchlist settings") - else: - self.settings = WatchlistSettings() - self._save_settings() - logger.info("Settings file not found, using defaults") + with Session(engine) as session: + statement = select(WatchlistSettingsTable).where(WatchlistSettingsTable.user_id == "default") + db_settings = session.exec(statement).first() + if db_settings: + self.settings = WatchlistSettings( + check_interval_hours=db_settings.check_interval_hours, + auto_download_enabled=db_settings.auto_download_enabled, + max_concurrent_auto_downloads=db_settings.max_concurrent_auto_downloads, + notify_on_new_episodes=db_settings.notify_on_new_episodes, + include_completed_anime=db_settings.include_completed_anime + ) + logger.info(f"Loaded watchlist settings from database") + else: + self.settings = WatchlistSettings() + self._save_settings() + logger.info("Settings not found in database, created defaults") except Exception as e: - logger.error(f"Error loading settings: {e}") + logger.error(f"Error loading settings from database: {e}") self.settings = WatchlistSettings() def _save_settings(self): try: - os.makedirs(os.path.dirname(self.settings_file), exist_ok=True) - temp_file = f"{self.settings_file}.tmp" - with open(temp_file, 'w', encoding='utf-8') as f: - json.dump(self.settings.model_dump(mode='json'), f, indent=2, ensure_ascii=False) - os.replace(temp_file, self.settings_file) - logger.debug("Saved watchlist settings") + with Session(engine) as session: + statement = select(WatchlistSettingsTable).where(WatchlistSettingsTable.user_id == "default") + db_settings = session.exec(statement).first() + + if db_settings: + db_settings.check_interval_hours = self.settings.check_interval_hours + db_settings.auto_download_enabled = self.settings.auto_download_enabled + db_settings.max_concurrent_auto_downloads = self.settings.max_concurrent_auto_downloads + db_settings.notify_on_new_episodes = self.settings.notify_on_new_episodes + db_settings.include_completed_anime = self.settings.include_completed_anime + else: + db_settings = WatchlistSettingsTable( + user_id="default", + check_interval_hours=self.settings.check_interval_hours, + auto_download_enabled=self.settings.auto_download_enabled, + max_concurrent_auto_downloads=self.settings.max_concurrent_auto_downloads, + notify_on_new_episodes=self.settings.notify_on_new_episodes, + include_completed_anime=self.settings.include_completed_anime + ) + session.add(db_settings) + + session.commit() + logger.debug("Saved watchlist settings to database") except Exception as e: - logger.error(f"Error saving settings: {e}") + logger.error(f"Error saving settings to database: {e}") def _to_api_model(self, db_item: WatchlistItemTable) -> WatchlistItem: """Convert database table model to API response model""" diff --git a/scripts/audit_database.py b/scripts/audit_database.py new file mode 100644 index 0000000..941c2bd --- /dev/null +++ b/scripts/audit_database.py @@ -0,0 +1,59 @@ +import os +import sys +from sqlmodel import Session, select, create_engine, inspect + +# Add root directory to sys.path +sys.path.append(os.getcwd()) + +from app.database import engine +from app.models.auth import UserTable +from app.models.watchlist import WatchlistItemTable, WatchlistSettingsTable +from app.models.favorites import FavoriteTable +from app.models.sonarr import SonarrMappingTable, SonarrConfigTable + +def audit_db(): + print("--- Ohm Stream Downloader: SQL Database Audit ---") + + if not os.path.exists("ohm_streaming.db"): + print("ERROR: ohm_streaming.db not found!") + return + + inspector = inspect(engine) + tables = inspector.get_table_names() + print(f"Tables found: {', '.join(tables)}") + + expected_tables = ["users", "watchlist_items", "watchlist_settings", "favorites", "sonarr_mappings", "sonarr_config", "alembic_version"] + missing = [t for t in expected_tables if t not in tables] + if missing: + print(f"WARNING: Missing tables: {', '.join(missing)}") + else: + print("SUCCESS: All core tables are present.") + + with Session(engine) as session: + # Check users + users_count = len(session.exec(select(UserTable)).all()) + print(f"Users: {users_count}") + + # Check watchlist + watchlist_count = len(session.exec(select(WatchlistItemTable)).all()) + print(f"Watchlist Items: {watchlist_count}") + + # Check settings + settings_count = len(session.exec(select(WatchlistSettingsTable)).all()) + print(f"Watchlist Settings entries: {settings_count}") + + # Check favorites + fav_count = len(session.exec(select(FavoriteTable)).all()) + print(f"Favorites: {fav_count}") + + # Check Sonarr + sonarr_map_count = len(session.exec(select(SonarrMappingTable)).all()) + print(f"Sonarr Mappings: {sonarr_map_count}") + + # Sample data check + if fav_count > 0: + sample_fav = session.exec(select(FavoriteTable).limit(1)).first() + print(f"Sample Favorite: {sample_fav.title} ({sample_fav.provider})") + +if __name__ == "__main__": + audit_db() diff --git a/scripts/migrate_json_to_sql.py b/scripts/migrate_json_to_sql.py new file mode 100644 index 0000000..cd1cd29 --- /dev/null +++ b/scripts/migrate_json_to_sql.py @@ -0,0 +1,276 @@ +import json +import os +from pathlib import Path +from datetime import datetime +from sqlmodel import Session, select +from sqlalchemy.exc import IntegrityError +import sys + +# Add the root directory to sys.path to import app modules +sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +from app.database import engine, create_db_and_tables +from app.models.auth import UserTable +from app.models.watchlist import WatchlistItemTable, WatchlistSettingsTable, WatchlistStatus, QualityPreference +from app.models.favorites import FavoriteTable +from app.models.sonarr import SonarrMappingTable, SonarrConfigTable + +def migrate_users(session: Session): + path = Path("config/users.json") + if not path.exists(): + print("No users.json found.") + return + + with open(path, "r", encoding="utf-8") as f: + data = json.load(f) + + users_data = data.get("users", {}) + count = 0 + for user_id, user_info in users_data.items(): + existing = session.get(UserTable, user_id) + if existing: + continue + + # Parse dates + created_at = datetime.now() + if "created_at" in user_info: + try: + created_at = datetime.fromisoformat(user_info["created_at"]) + except ValueError: + pass + + last_login = None + if "last_login" in user_info and user_info["last_login"]: + try: + last_login = datetime.fromisoformat(user_info["last_login"]) + except ValueError: + pass + + user = UserTable( + id=user_id, + username=user_info.get("username", "unknown"), + email=user_info.get("email"), + hashed_password=user_info.get("hashed_password", ""), + is_active=user_info.get("is_active", True), + created_at=created_at, + last_login=last_login + ) + session.add(user) + count += 1 + session.commit() + print(f"Migrated {count} users.") + +def migrate_watchlist(session: Session): + path = Path("config/watchlist.json") + if not path.exists(): + print("No watchlist.json found.") + return + + with open(path, "r", encoding="utf-8") as f: + data = json.load(f) + + count = 0 + for user_id, items in data.items(): + for item in items: + existing = session.get(WatchlistItemTable, item.get("id")) + if existing: + continue + + last_checked = None + if "last_checked" in item and item["last_checked"]: + try: + last_checked = datetime.fromisoformat(item["last_checked"]) + except ValueError: + pass + + added_at = datetime.now() + if "added_at" in item: + try: + added_at = datetime.fromisoformat(item["added_at"]) + except ValueError: + pass + + updated_at = datetime.now() + if "updated_at" in item: + try: + updated_at = datetime.fromisoformat(item["updated_at"]) + except ValueError: + pass + + wl_item = WatchlistItemTable( + id=item["id"], + user_id=user_id, + anime_title=item["anime_title"], + anime_url=item["anime_url"], + provider_id=item["provider_id"], + lang=item.get("lang", "vostfr"), + last_checked=last_checked, + last_episode_downloaded=item.get("last_episode_downloaded", 0), + total_episodes=item.get("total_episodes"), + auto_download=item.get("auto_download", True), + quality_preference=item.get("quality_preference", QualityPreference.AUTO), + status=item.get("status", WatchlistStatus.ACTIVE), + poster_image=item.get("poster_image"), + cover_image=item.get("cover_image"), + synopsis=item.get("synopsis"), + genres=item.get("genres", []), + added_at=added_at, + updated_at=updated_at + ) + session.add(wl_item) + count += 1 + session.commit() + print(f"Migrated {count} watchlist items.") + +def migrate_watchlist_settings(session: Session): + path = Path("config/watchlist_settings.json") + if not path.exists(): + print("No watchlist_settings.json found.") + return + + with open(path, "r", encoding="utf-8") as f: + settings = json.load(f) + + count = 0 + # Treat as global settings for "default" user + user_id = "default" + existing = session.exec(select(WatchlistSettingsTable).where(WatchlistSettingsTable.user_id == user_id)).first() + if not existing: + setting_row = WatchlistSettingsTable( + user_id=user_id, + check_interval_hours=settings.get("check_interval_hours", 6), + auto_download_enabled=settings.get("auto_download_enabled", True), + max_concurrent_auto_downloads=settings.get("max_concurrent_auto_downloads", 2), + notify_on_new_episodes=settings.get("notify_on_new_episodes", False), + include_completed_anime=settings.get("include_completed_anime", False) + ) + session.add(setting_row) + count += 1 + session.commit() + print(f"Migrated {count} watchlist settings.") + +def migrate_favorites(session: Session): + path = Path("data/favorites.json") + if not path.exists(): + print("No favorites.json found.") + return + + with open(path, "r", encoding="utf-8") as f: + try: + data = json.load(f) + except json.JSONDecodeError: + print("Invalid favorites.json.") + return + + count = 0 + for fav_id, fav in data.items(): + existing = session.exec(select(FavoriteTable).where(FavoriteTable.anime_id == fav_id)).first() + if existing: + continue + + created_at = datetime.now() + if "created_at" in fav: + try: + created_at = datetime.fromisoformat(fav["created_at"]) + except ValueError: + pass + + updated_at = datetime.now() + if "updated_at" in fav: + try: + updated_at = datetime.fromisoformat(fav["updated_at"]) + except ValueError: + pass + + fav_row = FavoriteTable( + anime_id=fav_id, + user_id="default", # Favorites were global + title=fav.get("title", ""), + url=fav.get("url", ""), + provider=fav.get("provider", ""), + poster_url=fav.get("poster_url"), + anime_metadata=fav.get("metadata", {}), + created_at=created_at, + updated_at=updated_at + ) + session.add(fav_row) + count += 1 + session.commit() + print(f"Migrated {count} favorites.") + +def migrate_sonarr(session: Session): + # Config + path_config = Path("config/sonarr.json") + if path_config.exists(): + with open(path_config, "r", encoding="utf-8") as f: + data = json.load(f) + + existing = session.exec(select(SonarrConfigTable)).first() + if not existing: + conf = SonarrConfigTable( + webhook_enabled=data.get("webhook_enabled", False), + webhook_secret=data.get("webhook_secret"), + auto_download_enabled=data.get("auto_download_enabled", True), + default_language=data.get("default_language", "vostfr"), + default_quality=data.get("default_quality"), + default_provider=data.get("default_provider", "anime-sama"), + verify_hmac=data.get("verify_hmac", False), + log_webhooks=data.get("log_webhooks", True) + ) + session.add(conf) + session.commit() + print("Migrated Sonarr config.") + + # Mappings + path_maps = Path("config/sonarr_mappings.json") + if path_maps.exists(): + with open(path_maps, "r", encoding="utf-8") as f: + data = json.load(f) + + count = 0 + for map_id, mapping in data.items(): + existing = session.exec(select(SonarrMappingTable).where(SonarrMappingTable.sonarr_series_id == int(map_id))).first() + if existing: + continue + + created_at = datetime.now() + if "created_at" in mapping: + try: + created_at = datetime.fromisoformat(mapping["created_at"]) + except ValueError: + pass + + updated_at = datetime.now() + if "updated_at" in mapping: + try: + updated_at = datetime.fromisoformat(mapping["updated_at"]) + except ValueError: + pass + + map_row = SonarrMappingTable( + user_id="default", + sonarr_series_id=mapping.get("sonarr_series_id", int(map_id)), + sonarr_title=mapping.get("sonarr_title", ""), + anime_provider=mapping.get("anime_provider", ""), + anime_url=mapping.get("anime_url", ""), + anime_title=mapping.get("anime_title", ""), + lang=mapping.get("lang", "vostfr"), + quality_preference=mapping.get("quality_preference"), + auto_download=mapping.get("auto_download", True), + created_at=created_at, + updated_at=updated_at + ) + session.add(map_row) + count += 1 + session.commit() + print(f"Migrated {count} Sonarr mappings.") + +if __name__ == "__main__": + create_db_and_tables() + with Session(engine) as session: + migrate_users(session) + migrate_watchlist(session) + migrate_watchlist_settings(session) + migrate_favorites(session) + migrate_sonarr(session) + print("Data migration complete.") diff --git a/templates/base.html b/templates/base.html index c313b0d..6bedbec 100644 --- a/templates/base.html +++ b/templates/base.html @@ -11,7 +11,7 @@ - +