Initial commit: GitHub Release Monitor CLI tool
CLI tool that tracks GitHub releases or tags from user-defined repositories and exposes them as a subscribable RSS 2.0 feed. Features: - Track releases or tags per repository - SQLite storage with automatic deduplication - RSS 2.0 feed with markdown-stripped release notes - Built-in HTTP server for local feed access - Rate-limit-aware GitHub API client - 64 passing tests
This commit is contained in:
+16
@@ -0,0 +1,16 @@
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.so
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
.eggs/
|
||||
*.egg
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
*.db
|
||||
.env
|
||||
venv/
|
||||
.venv/
|
||||
@@ -0,0 +1,181 @@
|
||||
# GitHub Release Monitor
|
||||
|
||||
A CLI tool that tracks GitHub releases or tags from user-defined repositories and exposes them as a subscribable RSS feed.
|
||||
|
||||
Some repositories (e.g., Zammad) only use git tags without creating GitHub releases — both modes are supported.
|
||||
|
||||
## Features
|
||||
|
||||
- Track **releases** or **tags** per repository
|
||||
- SQLite storage with automatic deduplication
|
||||
- RSS 2.0 feed with markdown-stripped release notes
|
||||
- Built-in HTTP server for local feed access
|
||||
- Rate-limit-aware GitHub API client
|
||||
- Configurable via environment variables and CLI flags
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install .
|
||||
```
|
||||
|
||||
Or install in development mode:
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Set your GitHub token (optional, increases rate limit from 60/h to 5000/h)
|
||||
export GH_TOKEN=your_token
|
||||
|
||||
# Add repositories to track
|
||||
ghrel add golang/go # Track releases (default)
|
||||
ghrel add zammad/zammad --tags # Track tags
|
||||
|
||||
# Fetch latest releases/tags
|
||||
ghrel check
|
||||
|
||||
# List tracked repositories
|
||||
ghrel list
|
||||
|
||||
# Start the RSS feed server
|
||||
ghrel serve
|
||||
```
|
||||
|
||||
The RSS feed is available at `http://127.0.0.1:8080/feed.xml`.
|
||||
|
||||
## CLI Commands
|
||||
|
||||
```
|
||||
ghrel # Main command
|
||||
├── --version # Show version
|
||||
├── --verbose / -v # Enable verbose/debug output
|
||||
├── --quiet / -q # Suppress non-essential output
|
||||
├── --db-path <path> # Override database path
|
||||
├── add <owner/repo> [--tags] # Add a repository to track
|
||||
├── remove <owner/repo> # Remove a repository
|
||||
├── update <owner/repo> [--tags] [--release] # Update tracking mode
|
||||
├── list # List all tracked repositories
|
||||
├── check [--since N] # Fetch latest releases/tags
|
||||
└── serve [--port 8080] [--host 127.0.0.1] # Start HTTP server
|
||||
```
|
||||
|
||||
### `add <owner/repo> [--tags]`
|
||||
|
||||
Add a repository to track. Validates the format and pings the GitHub API to verify the repository exists.
|
||||
|
||||
```bash
|
||||
ghrel add golang/go # Track releases
|
||||
ghrel add zammad/zammad --tags # Track tags
|
||||
```
|
||||
|
||||
### `remove <owner/repo>`
|
||||
|
||||
Remove a repository and its associated entries.
|
||||
|
||||
```bash
|
||||
ghrel remove golang/go
|
||||
```
|
||||
|
||||
### `update <owner/repo> [--tags] [--release]`
|
||||
|
||||
Update the tracking mode for an existing repository.
|
||||
|
||||
```bash
|
||||
ghrel update zammad/zammad --tags
|
||||
ghrel update zammad/zammad --release
|
||||
```
|
||||
|
||||
### `list`
|
||||
|
||||
Display tracked repositories with entry counts and last check time.
|
||||
|
||||
```
|
||||
REPO MODE ENTRIES LAST CHECKED
|
||||
golang/go release 12 2025-01-15 10:30
|
||||
zammad/zammad tag 8 2025-01-15 10:30
|
||||
```
|
||||
|
||||
### `check [--since N]`
|
||||
|
||||
Fetch latest releases or tags for all tracked repositories. With `--since`, only fetch entries published in the last N hours (e.g., `--since 24h`).
|
||||
|
||||
```bash
|
||||
ghrel check # Fetch all new entries
|
||||
ghrel check --since 24h # Only entries from the last 24 hours
|
||||
```
|
||||
|
||||
### `serve [--port PORT] [--host HOST]`
|
||||
|
||||
Start the HTTP server.
|
||||
|
||||
```bash
|
||||
ghrel serve # Default: 127.0.0.1:8080
|
||||
ghrel serve --port 9090 --host 0.0.0.0 # Custom port and host
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
| Setting | Default | Override |
|
||||
|---------------|----------------------------|-------------------|
|
||||
| `db_path` | `~/.config/ghrel/repos.db` | `--db-path` flag |
|
||||
| `gh_token` | `GH_TOKEN` env var | — |
|
||||
| `server_host` | `127.0.0.1` | `--host` flag |
|
||||
| `server_port` | `8080` | `--port` flag |
|
||||
|
||||
## RSS Feed
|
||||
|
||||
The feed follows RSS 2.0 specification:
|
||||
|
||||
- **Title**: `GitHub Release Monitor`
|
||||
- **Items**: Sorted by `published_at` (newest first), capped at 50 items
|
||||
- **Release entries**: Title from release name, description from markdown-stripped body (truncated at 300 characters)
|
||||
- **Tag entries**: Title from tag name, description as `Tag {tag_name}`
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ CLI Interface │
|
||||
│ (add, remove, update, list, check, serve) │
|
||||
├──────────────────────────────────────────────────┤
|
||||
│ Core Logic │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌────────┐ │
|
||||
│ │ config.py │ │ db.py │ │github.py│ │
|
||||
│ │ (defaults, │ │ (sqlite3) │ │(httpx) │ │
|
||||
│ │ env vars) │ │ │ │ │ │
|
||||
│ └──────────────┘ └──────┬───────┘ └───┬────┘ │
|
||||
│ │ │ │
|
||||
│ ┌────────▼──────────────▼────┐ │
|
||||
│ │ rss.py │ │
|
||||
│ │ (xml.etree + markdown) │ │
|
||||
│ └──────────────┬─────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────────────▼─────────────┐ │
|
||||
│ │ server.py │ │
|
||||
│ │ (http.server, stdlib) │ │
|
||||
│ │ (/feed.xml endpoint) │ │
|
||||
│ └─────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
pytest
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Purpose |
|
||||
|--------------|------------------------------------------|
|
||||
| `httpx` | HTTP client for GitHub API |
|
||||
| `markdown` | Convert markdown to HTML |
|
||||
| `bleach` | Strip HTML tags for plain text |
|
||||
| `pytest` | Testing framework |
|
||||
| `respx` | Mocking HTTP requests for tests |
|
||||
|
||||
No heavy frameworks. `http.server` and `xml.etree.ElementTree` are stdlib.
|
||||
@@ -0,0 +1,274 @@
|
||||
# GitHub Release Monitor — Project Plan
|
||||
|
||||
## 1. Overview
|
||||
|
||||
A CLI tool that tracks GitHub releases or tags from user-defined repositories and exposes them as a subscribable RSS feed. Some repos (e.g., Zammad) only use git tags without creating GitHub releases — both modes are supported.
|
||||
|
||||
- **Language**: Python 3.12
|
||||
- **Storage**: SQLite (single file, embedded via `sqlite3` stdlib)
|
||||
- **API**: GitHub REST API via `httpx`
|
||||
- **Output**: RSS 2.0 XML feed served over HTTP (stdlib `http.server`)
|
||||
|
||||
## 2. Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ CLI Interface │
|
||||
│ (add, remove, update, list, check, serve) │
|
||||
├──────────────────────────────────────────────────┤
|
||||
│ Core Logic │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌────────┐ │
|
||||
│ │ config.py │ │ db.py │ │github.py│ │
|
||||
│ │ (defaults, │ │ (sqlite3) │ │(httpx) │ │
|
||||
│ │ env vars) │ │ │ │ │ │
|
||||
│ └──────────────┘ └──────┬───────┘ └───┬────┘ │
|
||||
│ │ │ │
|
||||
│ ┌────────▼──────────────▼────┐ │
|
||||
│ │ rss.py │ │
|
||||
│ │ (xml.etree + markdown) │ │
|
||||
│ └──────────────┬─────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────────────▼─────────────┐ │
|
||||
│ │ server.py │ │
|
||||
│ │ (http.server, stdlib) │ │
|
||||
│ │ (/feed.xml endpoint) │ │
|
||||
│ └─────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 3. File Structure
|
||||
|
||||
```
|
||||
github_release_monitor/
|
||||
├── plan.md # This file
|
||||
├── pyproject.toml # Project metadata, dependencies
|
||||
├── README.md
|
||||
├── src/
|
||||
│ ├── __init__.py
|
||||
│ ├── cli.py # CLI entry point (argparse)
|
||||
│ ├── config.py # Centralized config (defaults, env vars, paths)
|
||||
│ ├── db.py # SQLite schema & queries
|
||||
│ ├── github.py # GitHub API interaction
|
||||
│ ├── rss.py # RSS feed generation
|
||||
│ └── server.py # HTTP server for feed
|
||||
├── tests/
|
||||
│ ├── __init__.py
|
||||
│ ├── test_db.py
|
||||
│ ├── test_github.py
|
||||
│ ├── test_rss.py
|
||||
│ ├── test_server.py
|
||||
│ └── test_integration.py # End-to-end tests
|
||||
└── .gitignore
|
||||
```
|
||||
|
||||
## 4. Database Schema
|
||||
|
||||
### `repos` table
|
||||
| Column | Type | Constraints | Description |
|
||||
|------------|--------------|--------------------------------------|--------------------------------------|
|
||||
| id | INTEGER | PRIMARY KEY AUTO | Internal identifier |
|
||||
| owner_repo | TEXT | NOT NULL UNIQUE | GitHub repo identifier (`owner/repo`)|
|
||||
| mode | TEXT | NOT NULL DEFAULT 'release' CHECK(mode IN ('release', 'tag')) | Track releases or tags |
|
||||
| created_at | DATETIME | DEFAULT NOW() | When the repo was added |
|
||||
|
||||
The `owner_repo` field serves as both the unique identifier and the display name. The `mode` field determines whether to fetch GitHub releases (`release`) or git tags (`tag`). Default is `release`.
|
||||
|
||||
### `entries` table
|
||||
| Column | Type | Constraints | Description |
|
||||
|--------------|--------------|--------------------|----------------------------------------|
|
||||
| id | INTEGER | PRIMARY KEY AUTO | Internal identifier |
|
||||
| repo_id | INTEGER | FK → repos.id | Which repository this entry belongs to |
|
||||
| kind | TEXT | NOT NULL CHECK(kind IN ('release', 'tag')) | Whether this is a release or tag |
|
||||
| tag_name | TEXT | NOT NULL | Git tag / release tag |
|
||||
| title | TEXT | NOT NULL | Release title or tag name |
|
||||
| body | TEXT | DEFAULT '' | Release notes (empty for tags) |
|
||||
| published_at | DATETIME | NOT NULL | Release date or commit date |
|
||||
| html_url | TEXT | NOT NULL | Link to release or commit on GitHub |
|
||||
| fetched_at | DATETIME | DEFAULT NOW() | When we fetched this entry |
|
||||
|
||||
**Unique constraint**: `(repo_id, tag_name)` to avoid duplicates.
|
||||
|
||||
## 5. CLI Commands
|
||||
|
||||
```
|
||||
ghrel # Main command
|
||||
├── --version # Show version
|
||||
├── --verbose / -v # Enable verbose/debug output
|
||||
├── --quiet / -q # Suppress non-essential output
|
||||
├── --db-path <path> # Override database path (default: ~/.config/ghrel/repos.db)
|
||||
├── add <owner/repo> [--tags] # Add a repository to track
|
||||
├── remove <owner/repo> # Remove a repository
|
||||
├── update <owner/repo> [--tags] [--release] # Update tracking mode
|
||||
├── list # List all tracked repositories
|
||||
├── check [--since N] # Fetch latest releases/tags for all repos
|
||||
└── serve [--port 8080] [--host 127.0.0.1] # Start HTTP server
|
||||
```
|
||||
|
||||
### `add <owner/repo> [--tags]`
|
||||
- Accepts a single `owner/repo` argument (e.g., `golang/go`)
|
||||
- `--tags` flag: track git tags instead of releases (default: releases)
|
||||
- Validates the format with a regex: `^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$`
|
||||
- Pings the GitHub API to verify the repo exists (with token validation)
|
||||
- Inserts into `repos` table; errors on duplicate `owner_repo`
|
||||
- On success: `Added golang/go (releases)` or `Added zammad/zammad (tags)`
|
||||
|
||||
### `remove <owner/repo>`
|
||||
- Deletes from `repos` table by `owner_repo`
|
||||
- Cascading delete removes associated entries
|
||||
- On success: `Removed golang/go`
|
||||
|
||||
### `update <owner/repo> [--tags] [--release]`
|
||||
- Updates the tracking mode for an existing repo
|
||||
- `--tags`: switch to tracking tags
|
||||
- `--release`: switch to tracking releases
|
||||
- On success: `Updated zammad/zammad to track tags`
|
||||
|
||||
### `list`
|
||||
- Displays a table of tracked repos:
|
||||
```
|
||||
REPO MODE ENTRIES LAST CHECKED
|
||||
golang/go release 12 2025-01-15 10:30
|
||||
zammad/zammad tag 8 2025-01-15 10:30
|
||||
```
|
||||
|
||||
### `check [--since N]`
|
||||
- Iterates over all repos in the database
|
||||
- For each repo, fetches latest releases or tags depending on the repo's `mode`
|
||||
- If `--since N` is provided (e.g., `--since 24h`), only fetches entries published in the last N hours
|
||||
- Upserts into `entries` table (insert if new, skip if already known by `tag_name`)
|
||||
- Reports results: `Checked 5 repos: 3 new entries found`
|
||||
- Global rate-limit-aware queue: if one repo hits the rate limit, remaining repos wait
|
||||
|
||||
### `serve [--port 8080] [--host 127.0.0.1]`
|
||||
- Starts an HTTP server using stdlib `http.server`
|
||||
- Default host: `127.0.0.1` (localhost only, not exposed to network)
|
||||
- Serves RSS feed at `/feed.xml`
|
||||
- Serves a small index page at `/` with a link to the feed
|
||||
- Health check endpoint at `/health` returns `200 OK`
|
||||
|
||||
### Error format
|
||||
All errors go to stderr with non-zero exit code:
|
||||
```
|
||||
Error: <message>
|
||||
```
|
||||
|
||||
## 6. GitHub API Integration
|
||||
|
||||
### Approach: Direct HTTP calls via `httpx`
|
||||
|
||||
- **Authentication**: Read `GH_TOKEN` from environment variable. If set, use it for authenticated requests (5000/h vs 60/h). If not set, fall back to unauthenticated.
|
||||
- **Token validation**: On first API call after startup, check `GET /rate_limit` to verify the token is valid. If `401`, print a clear error: `Error: Invalid or missing GH_TOKEN. Set it with: export GH_TOKEN=your_token`
|
||||
- **Pagination**: Fetch first page only (30 items). Sufficient for tracking recent entries.
|
||||
- **Rate limiting**: Track `X-RateLimit-Remaining` header globally across all repos. If rate limited, pause all requests until `X-RateLimit-Reset` time. No per-repo retries — a single global wait prevents cascading 403s.
|
||||
|
||||
### Two API endpoints depending on `mode`
|
||||
|
||||
**Releases** (`mode = 'release'`):
|
||||
- `GET https://api.github.com/repos/{owner}/{repo}/releases?per_page=30`
|
||||
- Data extracted: `tag_name`, `name` (title), `body` (markdown), `published_at`, `html_url`
|
||||
|
||||
**Tags** (`mode = 'tag'`):
|
||||
- `GET https://api.github.com/repos/{owner}/{repo}/tags?per_page=30`
|
||||
- Data extracted: `name` (used as both `tag_name` and `title`), `zipball_url` → derive `html_url` as `https://github.com/{owner}/{repo}/tags/{name}`
|
||||
- Tags have no `body` — store empty string
|
||||
- Tags have no `published_at` — fetch commit date via `GET /repos/{owner}/{repo}/git/ref/tags/{name}` and use `object.commit.author.date` from the resolved commit object
|
||||
|
||||
## 7. RSS Feed Specification
|
||||
|
||||
RSS 2.0 generated with stdlib `xml.etree.ElementTree`. RSS 2.0 is simple enough that `feedgen` is unnecessary.
|
||||
|
||||
### Feed-level
|
||||
- **title**: "GitHub Release Monitor"
|
||||
- **description**: "Latest releases from tracked GitHub repositories"
|
||||
- **link**: `http://{host}:{port}/` (base URL of the serving instance)
|
||||
- **lastBuildDate**: Current time in RFC 822 format
|
||||
|
||||
### Per-item
|
||||
- **title**: `[{owner/repo}] {title}`
|
||||
- **description**: First 300 characters of `body`, with markdown stripped to plain text. Append `…` if truncated. For tags (no body), use: `Tag {tag_name}`
|
||||
- **link**: `html_url` from the entry
|
||||
- **pubDate**: `published_at` in RFC 822 format
|
||||
- **guid**: `github.com/{owner}/{repo}/releases/tag/{tag_name}` (unique identifier, same format for both releases and tags)
|
||||
|
||||
Items sorted by `published_at` descending (newest first). **Capped at 50 items** to prevent feed bloat.
|
||||
|
||||
### Markdown stripping
|
||||
Use `markdown` library to convert markdown to HTML, then `bleach` to strip all HTML tags and produce plain text.
|
||||
|
||||
## 8. Configuration (`config.py`)
|
||||
|
||||
Centralized configuration module:
|
||||
|
||||
| Setting | Default | Override |
|
||||
|------------------|----------------------------|-------------------|
|
||||
| `db_path` | `~/.config/ghrel/repos.db` | `--db-path` flag |
|
||||
| `gh_token` | `GH_TOKEN` env var | — |
|
||||
| `server_host` | `127.0.0.1` | `--host` flag |
|
||||
| `server_port` | `8080` | `--port` flag |
|
||||
| `rss_max_items` | `50` | — |
|
||||
| `api_per_page` | `30` | — |
|
||||
|
||||
On database creation, set file permissions to `0600` (owner read/write only) to avoid leaking tracked repo list.
|
||||
|
||||
## 9. Error Handling
|
||||
|
||||
- **GitHub API errors** (404, 403, etc.): Log warning, skip the repo, continue with others
|
||||
- **Network errors**: Retry with exponential backoff (3 attempts, 1s / 2s / 4s), then skip
|
||||
- **Rate limit exceeded**: Log warning with reset time, pause all requests globally
|
||||
- **Invalid repo URL**: Validate format on `add`, show clear error: `Error: Invalid repo format 'foo'. Expected 'owner/repo'`
|
||||
- **Invalid token**: Clear error on first API call: `Error: GH_TOKEN is invalid (401 Unauthorized)`
|
||||
- **Database errors**: Wrap in user-friendly messages: `Error: Failed to open database: <reason>`
|
||||
- **Port in use**: Clear error: `Error: Port 8080 is already in use`
|
||||
|
||||
All errors go to stderr. Exit codes: `0` success, `1` general error, `2` usage error.
|
||||
|
||||
## 10. Testing Strategy
|
||||
|
||||
Write tests **alongside implementation**, not after. Each module gets tests before moving to the next.
|
||||
|
||||
| Test file | What it covers |
|
||||
|----------------------|----------------------------------------------------------|
|
||||
| `test_db.py` | Schema creation, CRUD, constraints, `mode` CHECK, `0600` permissions |
|
||||
| `test_github.py` | API mocking for both releases and tags, parsing, rate limiting, token validation |
|
||||
| `test_rss.py` | Feed generation, markdown stripping, truncation, XML escaping, item cap, tag entries (no body) |
|
||||
| `test_server.py` | HTTP server, feed endpoint, health check, response headers |
|
||||
| `test_integration.py`| Full flow: `add` → `check` → `serve` → fetch feed → verify XML (both modes) |
|
||||
| `test_cli.py` | CLI smoke tests: exit codes, stderr output, flag parsing, `--tags` flag |
|
||||
|
||||
Use `pytest` with `respx` for mocking `httpx`. In-memory SQLite for DB tests. `pytest-httpserver` or `http.server` in a thread for integration tests.
|
||||
|
||||
## 11. Implementation Order
|
||||
|
||||
1. **Project scaffolding**: `pyproject.toml`, directory structure, `.gitignore`
|
||||
2. **Config module** (`config.py`): Defaults, env vars, DB path management
|
||||
3. **Database layer** (`db.py`): Schema, connection, CRUD functions + `test_db.py`
|
||||
4. **CLI foundation** (`cli.py`): `add`, `remove`, `list` commands + `test_cli.py`
|
||||
5. **GitHub fetcher** (`github.py`): API calls for releases and tags, parsing, rate limiting, token validation + `test_github.py`
|
||||
6. **CLI `check` command**: Integration of fetcher with DB upsert
|
||||
7. **RSS generator** (`rss.py`): xml.etree generation, markdown stripping + `test_rss.py`
|
||||
8. **HTTP server** (`server.py`): stdlib http.server, `/feed.xml`, `/health` + `test_server.py`
|
||||
9. **CLI `serve` command**: Wire up server
|
||||
10. **Integration tests** (`test_integration.py`): End-to-end flow
|
||||
11. **README**: Usage documentation
|
||||
|
||||
## 12. Dependencies
|
||||
|
||||
| Package | Purpose |
|
||||
|--------------|----------------------------------------------|
|
||||
| `httpx` | HTTP client for GitHub API |
|
||||
| `markdown` | Convert markdown to HTML |
|
||||
| `bleach` | Strip HTML tags for plain text |
|
||||
| `pytest` | Testing framework |
|
||||
| `respx` | Mocking HTTP requests for tests |
|
||||
|
||||
No heavy frameworks. `http.server` and `xml.etree.ElementTree` are stdlib.
|
||||
|
||||
## 13. Future Considerations (Out of Scope for MVP)
|
||||
|
||||
- Webhook-based real-time updates instead of polling
|
||||
- Docker container for easy deployment
|
||||
- Configuration file (`ghrel.toml`) instead of CLI-only management
|
||||
- Support for GitLab releases
|
||||
- Email notifications for new releases
|
||||
- Web UI for managing tracked repos
|
||||
- Scheduled background checks (cron / systemd timer integration)
|
||||
@@ -0,0 +1,22 @@
|
||||
[project]
|
||||
name = "ghrel"
|
||||
version = "0.1.0"
|
||||
description = "Track GitHub releases and tags as an RSS feed"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"httpx",
|
||||
"markdown",
|
||||
"bleach",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
ghrel = "src.cli:main"
|
||||
|
||||
[project.optional-dependencies]
|
||||
test = [
|
||||
"pytest",
|
||||
"respx",
|
||||
]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
@@ -0,0 +1,3 @@
|
||||
"""GitHub Release Monitor - track GitHub releases and tags as an RSS feed."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
+268
@@ -0,0 +1,268 @@
|
||||
"""CLI entry point."""
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Literal
|
||||
|
||||
from src import __version__
|
||||
from src.config import (
|
||||
DEFAULT_SERVER_HOST,
|
||||
DEFAULT_SERVER_PORT,
|
||||
get_db_path,
|
||||
)
|
||||
from src.db import get_connection
|
||||
from src.github import GitHubClient, AuthenticationError, NotFoundError
|
||||
|
||||
|
||||
REPO_PATTERN = re.compile(r"^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$")
|
||||
|
||||
|
||||
def error(msg: str, code: int = 1) -> None:
|
||||
print(f"Error: {msg}", file=sys.stderr)
|
||||
sys.exit(code)
|
||||
|
||||
|
||||
def parse_since(value: str) -> timedelta:
|
||||
"""Parse a duration string like '24h', '2h', '1d' into a timedelta."""
|
||||
m = re.match(r"^(\d+)([hdm])$", value.strip().lower())
|
||||
if not m:
|
||||
return timedelta(hours=24) # default
|
||||
num, unit = int(m.group(1)), m.group(2)
|
||||
if unit == "h":
|
||||
return timedelta(hours=num)
|
||||
elif unit == "d":
|
||||
return timedelta(days=num)
|
||||
else: # 'm' for minutes
|
||||
return timedelta(minutes=num)
|
||||
|
||||
|
||||
def cmd_add(args: argparse.Namespace, db_path: str) -> None:
|
||||
"""Add a repository to track."""
|
||||
owner_repo = args.repo
|
||||
if not REPO_PATTERN.match(owner_repo):
|
||||
error(f"Invalid repo format '{owner_repo}'. Expected 'owner/repo'")
|
||||
|
||||
mode: Literal["release", "tag"] = "tag" if args.tags else "release"
|
||||
|
||||
conn = get_connection(db_path)
|
||||
try:
|
||||
# Validate repo exists via GitHub API
|
||||
with GitHubClient() as gh:
|
||||
if not gh.repo_exists(owner_repo):
|
||||
error(f"Repository '{owner_repo}' not found on GitHub")
|
||||
|
||||
from src.db import add_repo
|
||||
add_repo(conn, owner_repo, mode)
|
||||
conn.commit()
|
||||
mode_label = "tags" if mode == "tag" else "releases"
|
||||
print(f"Added {owner_repo} ({mode_label})")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def cmd_remove(args: argparse.Namespace, db_path: str) -> None:
|
||||
"""Remove a repository."""
|
||||
owner_repo = args.repo
|
||||
conn = get_connection(db_path)
|
||||
try:
|
||||
from src.db import remove_repo
|
||||
if remove_repo(conn, owner_repo):
|
||||
print(f"Removed {owner_repo}")
|
||||
else:
|
||||
error(f"Repository '{owner_repo}' not found")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def cmd_update(args: argparse.Namespace, db_path: str) -> None:
|
||||
"""Update the tracking mode for a repository."""
|
||||
owner_repo = args.repo
|
||||
if args.tags and args.release:
|
||||
error("Cannot specify both --tags and --release")
|
||||
|
||||
if args.tags:
|
||||
mode: Literal["release", "tag"] = "tag"
|
||||
elif args.release:
|
||||
mode = "release"
|
||||
else:
|
||||
error("Specify --tags or --release")
|
||||
|
||||
conn = get_connection(db_path)
|
||||
try:
|
||||
from src.db import update_repo_mode
|
||||
if update_repo_mode(conn, owner_repo, mode):
|
||||
mode_label = "tags" if mode == "tag" else "releases"
|
||||
print(f"Updated {owner_repo} to track {mode_label}")
|
||||
else:
|
||||
error(f"Repository '{owner_repo}' not found")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def cmd_list(args: argparse.Namespace, db_path: str) -> None:
|
||||
"""List all tracked repositories."""
|
||||
conn = get_connection(db_path)
|
||||
try:
|
||||
from src.db import list_repos
|
||||
repos = list_repos(conn)
|
||||
if not repos:
|
||||
print("No repositories tracked.")
|
||||
return
|
||||
# Format table
|
||||
header = f"{'REPO':<20} {'MODE':<10} {'ENTRIES':<8} {'LAST CHECKED'}"
|
||||
print(header)
|
||||
print("-" * len(header))
|
||||
for r in repos:
|
||||
last = r["last_checked"] or "never"
|
||||
if "T" in str(last):
|
||||
last = str(last)[:16].replace("T", " ")
|
||||
print(f"{r['owner_repo']:<20} {r['mode']:<10} {r['entries']:<8} {last}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def cmd_check(args: argparse.Namespace, db_path: str) -> None:
|
||||
"""Fetch latest releases/tags for all tracked repos."""
|
||||
conn = get_connection(db_path)
|
||||
try:
|
||||
from src.db import get_repos_with_mode, upsert_entry, get_repo_id
|
||||
|
||||
repos = get_repos_with_mode(conn)
|
||||
if not repos:
|
||||
print("No repositories tracked. Use 'ghrel add <owner/repo>' first.")
|
||||
return
|
||||
|
||||
since = getattr(args, "since", None)
|
||||
since_delta = parse_since(since) if since else None
|
||||
|
||||
with GitHubClient() as gh:
|
||||
total_new = 0
|
||||
for repo_id, owner_repo, mode in repos:
|
||||
try:
|
||||
if mode == "tag":
|
||||
items = gh.fetch_tags(owner_repo)
|
||||
kind: Literal["release", "tag"] = "tag"
|
||||
else:
|
||||
items = gh.fetch_releases(owner_repo)
|
||||
kind = "release"
|
||||
|
||||
for item in items:
|
||||
if since_delta:
|
||||
published = datetime.fromisoformat(item["published_at"].replace("Z", "+00:00"))
|
||||
cutoff = datetime.now(timezone.utc) - since_delta
|
||||
if published < cutoff:
|
||||
continue
|
||||
|
||||
inserted = upsert_entry(
|
||||
conn, repo_id, kind,
|
||||
item["tag_name"], item["title"], item["body"],
|
||||
item["published_at"], item["html_url"],
|
||||
)
|
||||
if inserted:
|
||||
total_new += 1
|
||||
except Exception as e:
|
||||
print(f"Warning: Failed to check {owner_repo}: {e}", file=sys.stderr)
|
||||
|
||||
print(f"Checked {len(repos)} repos: {total_new} new entries found")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def cmd_serve(args: argparse.Namespace, db_path: str) -> None:
|
||||
"""Start the HTTP server."""
|
||||
host = args.host or DEFAULT_SERVER_HOST
|
||||
port = args.port or DEFAULT_SERVER_PORT
|
||||
|
||||
from src.server import create_handler
|
||||
from http.server import HTTPServer
|
||||
|
||||
handler = create_handler(db_path)
|
||||
try:
|
||||
server = HTTPServer((host, port), handler)
|
||||
except OSError as e:
|
||||
if e.errno == 98: # Address already in use
|
||||
error(f"Port {port} is already in use")
|
||||
raise
|
||||
|
||||
print(f"Serving RSS feed at http://{host}:{port}/feed.xml")
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
print("\nShutting down server.")
|
||||
server.server_close()
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="ghrel",
|
||||
description="Track GitHub releases and tags as an RSS feed",
|
||||
)
|
||||
parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
|
||||
parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose output")
|
||||
parser.add_argument("--quiet", "-q", action="store_true", help="Suppress non-essential output")
|
||||
parser.add_argument("--db-path", default=None, help="Override database path")
|
||||
|
||||
subparsers = parser.add_subparsers(dest="command")
|
||||
|
||||
# add
|
||||
p_add = subparsers.add_parser("add", help="Add a repository to track")
|
||||
p_add.add_argument("repo", help="Repository in 'owner/repo' format")
|
||||
p_add.add_argument("--tags", action="store_true", help="Track git tags instead of releases")
|
||||
|
||||
# remove
|
||||
p_remove = subparsers.add_parser("remove", help="Remove a repository")
|
||||
p_remove.add_argument("repo", help="Repository in 'owner/repo' format")
|
||||
|
||||
# update
|
||||
p_update = subparsers.add_parser("update", help="Update tracking mode")
|
||||
p_update.add_argument("repo", help="Repository in 'owner/repo' format")
|
||||
p_update.add_argument("--tags", action="store_true", help="Switch to tracking tags")
|
||||
p_update.add_argument("--release", action="store_true", help="Switch to tracking releases")
|
||||
|
||||
# list
|
||||
subparsers.add_parser("list", help="List tracked repositories")
|
||||
|
||||
# check
|
||||
p_check = subparsers.add_parser("check", help="Fetch latest releases/tags")
|
||||
p_check.add_argument("--since", nargs="?", const="24h", default=None,
|
||||
help="Only fetch entries since N hours/days/minutes ago (e.g., 24h, 2d)")
|
||||
|
||||
# serve
|
||||
p_serve = subparsers.add_parser("serve", help="Start HTTP server")
|
||||
p_serve.add_argument("--port", type=int, default=DEFAULT_SERVER_PORT, help="Server port")
|
||||
p_serve.add_argument("--host", default=DEFAULT_SERVER_HOST, help="Server host")
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = build_parser()
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.command:
|
||||
parser.print_help()
|
||||
sys.exit(0)
|
||||
|
||||
db_path = args.db_path or str(get_db_path())
|
||||
|
||||
commands = {
|
||||
"add": cmd_add,
|
||||
"remove": cmd_remove,
|
||||
"update": cmd_update,
|
||||
"list": cmd_list,
|
||||
"check": cmd_check,
|
||||
"serve": cmd_serve,
|
||||
}
|
||||
|
||||
cmd = commands.get(args.command)
|
||||
if cmd:
|
||||
cmd(args, db_path)
|
||||
else:
|
||||
parser.print_help()
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Centralized configuration."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def get_db_path(override: str | None = None) -> Path:
|
||||
"""Return the database file path.
|
||||
|
||||
Priority: override argument > env var GHREL_DB_PATH > default.
|
||||
"""
|
||||
if override:
|
||||
return Path(override)
|
||||
env_path = os.environ.get("GHREL_DB_PATH")
|
||||
if env_path:
|
||||
return Path(env_path)
|
||||
return Path.home() / ".config" / "ghrel" / "repos.db"
|
||||
|
||||
|
||||
def get_gh_token() -> str | None:
|
||||
"""Return the GitHub token from the environment, or None."""
|
||||
return os.environ.get("GH_TOKEN")
|
||||
|
||||
|
||||
# Defaults
|
||||
DEFAULT_SERVER_HOST = "127.0.0.1"
|
||||
DEFAULT_SERVER_PORT = 8080
|
||||
RSS_MAX_ITEMS = 50
|
||||
API_PER_PAGE = 30
|
||||
@@ -0,0 +1,177 @@
|
||||
"""SQLite database layer."""
|
||||
|
||||
import sqlite3
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from src.config import get_db_path
|
||||
|
||||
|
||||
def get_connection(db_path: str | Path | None = None) -> sqlite3.Connection:
|
||||
"""Open a database connection, creating the DB and schema if needed."""
|
||||
path = Path(db_path) if db_path else get_db_path()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
conn = sqlite3.connect(str(path))
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
conn.execute("PRAGMA foreign_keys=ON")
|
||||
_create_tables(conn)
|
||||
# Set restrictive permissions on the database file
|
||||
try:
|
||||
path.chmod(0o600)
|
||||
except OSError:
|
||||
pass
|
||||
return conn
|
||||
|
||||
|
||||
def _create_tables(conn: sqlite3.Connection) -> None:
|
||||
"""Create the database tables if they don't exist."""
|
||||
conn.executescript("""
|
||||
CREATE TABLE IF NOT EXISTS repos (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
owner_repo TEXT NOT NULL UNIQUE,
|
||||
mode TEXT NOT NULL DEFAULT 'release' CHECK(mode IN ('release', 'tag')),
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS entries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
repo_id INTEGER NOT NULL,
|
||||
kind TEXT NOT NULL CHECK(kind IN ('release', 'tag')),
|
||||
tag_name TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
body TEXT DEFAULT '',
|
||||
published_at DATETIME NOT NULL,
|
||||
html_url TEXT NOT NULL,
|
||||
fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
FOREIGN KEY (repo_id) REFERENCES repos(id) ON DELETE CASCADE,
|
||||
UNIQUE(repo_id, tag_name)
|
||||
);
|
||||
""")
|
||||
conn.commit()
|
||||
|
||||
|
||||
# ── Repos ──
|
||||
|
||||
|
||||
def add_repo(conn: sqlite3.Connection, owner_repo: str, mode: Literal["release", "tag"] = "release") -> int:
|
||||
"""Add a repository to track. Returns the repo id."""
|
||||
cursor = conn.execute(
|
||||
"INSERT INTO repos (owner_repo, mode) VALUES (?, ?)", (owner_repo, mode)
|
||||
)
|
||||
conn.commit()
|
||||
return cursor.lastrowid
|
||||
|
||||
|
||||
def remove_repo(conn: sqlite3.Connection, owner_repo: str) -> bool:
|
||||
"""Remove a repository. Returns True if a row was deleted."""
|
||||
cursor = conn.execute("DELETE FROM repos WHERE owner_repo = ?", (owner_repo,))
|
||||
conn.commit()
|
||||
return cursor.rowcount > 0
|
||||
|
||||
|
||||
def update_repo_mode(conn: sqlite3.Connection, owner_repo: str, mode: Literal["release", "tag"]) -> bool:
|
||||
"""Update the tracking mode for a repo. Returns True if a row was updated."""
|
||||
cursor = conn.execute(
|
||||
"UPDATE repos SET mode = ? WHERE owner_repo = ?", (mode, owner_repo)
|
||||
)
|
||||
conn.commit()
|
||||
return cursor.rowcount > 0
|
||||
|
||||
|
||||
def list_repos(conn: sqlite3.Connection) -> list[dict]:
|
||||
"""List all tracked repositories with entry counts and last checked time."""
|
||||
rows = conn.execute("""
|
||||
SELECT
|
||||
r.owner_repo,
|
||||
r.mode,
|
||||
COUNT(e.id) AS entries,
|
||||
MAX(e.fetched_at) AS last_checked
|
||||
FROM repos r
|
||||
LEFT JOIN entries e ON e.repo_id = r.id
|
||||
GROUP BY r.id
|
||||
ORDER BY r.owner_repo
|
||||
""").fetchall()
|
||||
return [
|
||||
{
|
||||
"owner_repo": row[0],
|
||||
"mode": row[1],
|
||||
"entries": row[2],
|
||||
"last_checked": row[3],
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
|
||||
def get_repo_id(conn: sqlite3.Connection, owner_repo: str) -> int | None:
|
||||
"""Get the internal id for a repo, or None if not found."""
|
||||
row = conn.execute("SELECT id FROM repos WHERE owner_repo = ?", (owner_repo,)).fetchone()
|
||||
return row[0] if row else None
|
||||
|
||||
|
||||
def get_repos_with_mode(conn: sqlite3.Connection) -> list[tuple[int, str, str]]:
|
||||
"""Return list of (id, owner_repo, mode) for all repos."""
|
||||
return conn.execute("SELECT id, owner_repo, mode FROM repos").fetchall()
|
||||
|
||||
|
||||
# ── Entries ──
|
||||
|
||||
|
||||
def upsert_entry(
|
||||
conn: sqlite3.Connection,
|
||||
repo_id: int,
|
||||
kind: Literal["release", "tag"],
|
||||
tag_name: str,
|
||||
title: str,
|
||||
body: str,
|
||||
published_at: str,
|
||||
html_url: str,
|
||||
) -> bool:
|
||||
"""Insert an entry, skip if already exists (by repo_id + tag_name).
|
||||
|
||||
Returns True if a new entry was inserted.
|
||||
"""
|
||||
try:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO entries (repo_id, kind, tag_name, title, body, published_at, html_url)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(repo_id, kind, tag_name, title, body, published_at, html_url),
|
||||
)
|
||||
conn.commit()
|
||||
return True
|
||||
except sqlite3.IntegrityError:
|
||||
return False
|
||||
|
||||
|
||||
def get_all_entries(conn: sqlite3.Connection, limit: int = 50) -> list[dict]:
|
||||
"""Get all entries sorted by published_at descending, capped at *limit*."""
|
||||
rows = conn.execute("""
|
||||
SELECT e.tag_name, e.title, e.body, e.published_at, e.html_url,
|
||||
e.kind, r.owner_repo
|
||||
FROM entries e
|
||||
JOIN repos r ON r.id = e.repo_id
|
||||
ORDER BY e.published_at DESC
|
||||
LIMIT ?
|
||||
""", (limit,)).fetchall()
|
||||
return [
|
||||
{
|
||||
"tag_name": row[0],
|
||||
"title": row[1],
|
||||
"body": row[2],
|
||||
"published_at": row[3],
|
||||
"html_url": row[4],
|
||||
"kind": row[5],
|
||||
"owner_repo": row[6],
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
|
||||
def get_entry_count(conn: sqlite3.Connection, repo_id: int) -> int:
|
||||
"""Get the number of entries for a repo."""
|
||||
row = conn.execute(
|
||||
"SELECT COUNT(*) FROM entries WHERE repo_id = ?", (repo_id,)
|
||||
).fetchone()
|
||||
return row[0]
|
||||
+172
@@ -0,0 +1,172 @@
|
||||
"""GitHub API interaction."""
|
||||
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from typing import Literal
|
||||
|
||||
import httpx
|
||||
|
||||
from src.config import API_PER_PAGE, get_gh_token
|
||||
|
||||
|
||||
class GitHubClient:
|
||||
"""Client for the GitHub REST API with rate-limit awareness."""
|
||||
|
||||
BASE_URL = "https://api.github.com"
|
||||
|
||||
def __init__(self, token: str | None = None):
|
||||
self._client = httpx.Client()
|
||||
headers = {}
|
||||
self._token = token or get_gh_token()
|
||||
if self._token:
|
||||
headers["Authorization"] = f"token {self._token}"
|
||||
self._client.headers.update(headers)
|
||||
self._rate_limit_remaining = None
|
||||
self._rate_limit_reset = None
|
||||
|
||||
def close(self):
|
||||
self._client.close()
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *args):
|
||||
self.close()
|
||||
|
||||
def _check_rate_limit(self, response: httpx.Response) -> None:
|
||||
"""Update rate limit state from response headers."""
|
||||
remaining = response.headers.get("X-RateLimit-Remaining")
|
||||
reset = response.headers.get("X-RateLimit-Reset")
|
||||
if remaining is not None:
|
||||
self._rate_limit_remaining = int(remaining)
|
||||
if reset is not None:
|
||||
self._rate_limit_reset = int(reset)
|
||||
|
||||
def _handle_rate_limit(self, response: httpx.Response) -> None:
|
||||
"""Wait if rate limited."""
|
||||
if response.status_code == 403 and self._rate_limit_remaining is not None and self._rate_limit_remaining == 0:
|
||||
if self._rate_limit_reset:
|
||||
wait = max(1, self._rate_limit_reset - int(time.time()))
|
||||
import sys
|
||||
print(f"Warning: Rate limited. Waiting {wait}s...", file=sys.stderr)
|
||||
time.sleep(wait)
|
||||
|
||||
def _request(self, method: str, url: str, **kwargs) -> httpx.Response:
|
||||
"""Make an HTTP request with retry and rate-limit handling."""
|
||||
last_error = None
|
||||
for attempt in range(3):
|
||||
try:
|
||||
response = self._client.request(method, url, **kwargs)
|
||||
self._check_rate_limit(response)
|
||||
if response.status_code == 401:
|
||||
raise AuthenticationError("GH_TOKEN is invalid (401 Unauthorized)")
|
||||
if response.status_code == 404:
|
||||
raise NotFoundError(f"Resource not found: {url}")
|
||||
self._handle_rate_limit(response)
|
||||
response.raise_for_status()
|
||||
return response
|
||||
except (NotFoundError, AuthenticationError):
|
||||
raise
|
||||
except httpx.HTTPStatusError:
|
||||
raise
|
||||
except httpx.TransportError as e:
|
||||
last_error = e
|
||||
if attempt < 2:
|
||||
time.sleep(2 ** attempt)
|
||||
raise last_error # type: ignore[misc]
|
||||
|
||||
def validate_token(self) -> None:
|
||||
"""Check that the token is valid by hitting the rate_limit endpoint."""
|
||||
if not self._token:
|
||||
return # No token, allow unauthenticated
|
||||
resp = self._request("GET", f"{self.BASE_URL}/rate_limit")
|
||||
|
||||
def fetch_releases(self, owner_repo: str) -> list[dict]:
|
||||
"""Fetch the latest releases for a repository.
|
||||
|
||||
Returns a list of dicts with keys: tag_name, title, body, published_at, html_url.
|
||||
"""
|
||||
resp = self._request(
|
||||
"GET",
|
||||
f"{self.BASE_URL}/repos/{owner_repo}/releases",
|
||||
params={"per_page": API_PER_PAGE},
|
||||
)
|
||||
data = resp.json()
|
||||
result = []
|
||||
for release in data:
|
||||
result.append({
|
||||
"tag_name": release["tag_name"],
|
||||
"title": release.get("name") or release["tag_name"],
|
||||
"body": release.get("body") or "",
|
||||
"published_at": release.get("published_at") or datetime.now(timezone.utc).isoformat(),
|
||||
"html_url": release.get("html_url") or f"https://github.com/{owner_repo}/releases/tag/{release['tag_name']}",
|
||||
})
|
||||
return result
|
||||
|
||||
def fetch_tags(self, owner_repo: str) -> list[dict]:
|
||||
"""Fetch the latest tags for a repository.
|
||||
|
||||
Returns a list of dicts with keys: tag_name, title, body, published_at, html_url.
|
||||
"""
|
||||
resp = self._request(
|
||||
"GET",
|
||||
f"{self.BASE_URL}/repos/{owner_repo}/tags",
|
||||
params={"per_page": API_PER_PAGE},
|
||||
)
|
||||
data = resp.json()
|
||||
result = []
|
||||
for tag in data:
|
||||
tag_name = tag["name"]
|
||||
published_at = self._get_tag_date(owner_repo, tag_name)
|
||||
result.append({
|
||||
"tag_name": tag_name,
|
||||
"title": tag_name,
|
||||
"body": "",
|
||||
"published_at": published_at,
|
||||
"html_url": f"https://github.com/{owner_repo}/tags/{tag_name}",
|
||||
})
|
||||
return result
|
||||
|
||||
def _get_tag_date(self, owner_repo: str, tag_name: str) -> str:
|
||||
"""Get the commit date for a tag by resolving the git ref."""
|
||||
try:
|
||||
resp = self._request(
|
||||
"GET",
|
||||
f"{self.BASE_URL}/repos/{owner_repo}/git/ref/tags/{tag_name}",
|
||||
)
|
||||
data = resp.json()
|
||||
# The ref may point to a commit directly or to a tag object
|
||||
obj = data.get("object", {})
|
||||
obj_type = obj.get("type")
|
||||
if obj_type == "commit":
|
||||
author_date = obj.get("author", {}).get("date")
|
||||
if author_date:
|
||||
return author_date
|
||||
elif obj_type == "tag":
|
||||
# Follow the tag object to get the commit
|
||||
commit_sha = obj.get("sha")
|
||||
if commit_sha:
|
||||
commit_resp = self._request("GET", f"{self.BASE_URL}/repos/{owner_repo}/git/commits/{commit_sha}")
|
||||
commit_data = commit_resp.json()
|
||||
author_date = commit_data.get("author", {}).get("date")
|
||||
if author_date:
|
||||
return author_date
|
||||
except Exception:
|
||||
pass
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
def repo_exists(self, owner_repo: str) -> bool:
|
||||
"""Check if a repository exists on GitHub."""
|
||||
try:
|
||||
resp = self._request("GET", f"{self.BASE_URL}/repos/{owner_repo}")
|
||||
return resp.status_code == 200
|
||||
except (NotFoundError, httpx.HTTPStatusError):
|
||||
return False
|
||||
|
||||
|
||||
class AuthenticationError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class NotFoundError(Exception):
|
||||
pass
|
||||
+79
@@ -0,0 +1,79 @@
|
||||
"""RSS feed generation."""
|
||||
|
||||
import re
|
||||
import xml.etree.ElementTree as ET
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import bleach
|
||||
import markdown
|
||||
|
||||
from src.config import RSS_MAX_ITEMS
|
||||
|
||||
|
||||
def strip_markdown(text: str) -> str:
|
||||
"""Convert markdown to HTML, then strip tags to produce plain text."""
|
||||
if not text:
|
||||
return ""
|
||||
html = markdown.markdown(text)
|
||||
plain = bleach.clean(html, tags=[], strip=True)
|
||||
return plain.strip()
|
||||
|
||||
|
||||
def format_rfc822(dt_str: str) -> str:
|
||||
"""Convert an ISO 8601 datetime string to RFC 822 format for RSS."""
|
||||
dt = datetime.fromisoformat(dt_str.replace("Z", "+00:00"))
|
||||
return dt.strftime("%a, %d %b %Y %H:%M:%S %z")
|
||||
|
||||
|
||||
def generate_feed(entries: list[dict], base_url: str = "http://127.0.0.1:8080") -> str:
|
||||
"""Generate an RSS 2.0 XML feed from a list of entry dicts.
|
||||
|
||||
Each entry should have: tag_name, title, body, published_at, html_url, owner_repo.
|
||||
"""
|
||||
root = ET.Element("rss")
|
||||
root.set("version", "2.0")
|
||||
|
||||
channel = ET.SubElement(root, "channel")
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
ET.SubElement(channel, "title").text = "GitHub Release Monitor"
|
||||
ET.SubElement(channel, "description").text = "Latest releases from tracked GitHub repositories"
|
||||
ET.SubElement(channel, "link").text = base_url.rstrip("/") + "/"
|
||||
ET.SubElement(channel, "lastBuildDate").text = now.strftime("%a, %d %b %Y %H:%M:%S %z")
|
||||
|
||||
for entry in entries[:RSS_MAX_ITEMS]:
|
||||
item = ET.SubElement(channel, "item")
|
||||
|
||||
owner_repo = entry.get("owner_repo", "")
|
||||
ET.SubElement(item, "title").text = f"[{owner_repo}] {entry['title']}"
|
||||
|
||||
body = entry.get("body", "")
|
||||
if body:
|
||||
plain = strip_markdown(body)
|
||||
if len(plain) > 300:
|
||||
plain = plain[:300] + "\u2026"
|
||||
ET.SubElement(item, "description").text = plain
|
||||
else:
|
||||
ET.SubElement(item, "description").text = f"Tag {entry['tag_name']}"
|
||||
|
||||
ET.SubElement(item, "link").text = entry["html_url"]
|
||||
ET.SubElement(item, "pubDate").text = format_rfc822(entry["published_at"])
|
||||
ET.SubElement(item, "guid").text = (
|
||||
f"github.com/{owner_repo}/releases/tag/{entry['tag_name']}"
|
||||
)
|
||||
|
||||
return ET.tostring(root, encoding="unicode", xml_declaration=False)
|
||||
|
||||
|
||||
def generate_index_html(base_url: str = "http://127.0.0.1:8080") -> str:
|
||||
"""Generate a simple index page with a link to the RSS feed."""
|
||||
return f"""<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>GitHub Release Monitor</title></head>
|
||||
<body>
|
||||
<h1>GitHub Release Monitor</h1>
|
||||
<p><a href="{base_url.rstrip('/')}/feed.xml">RSS Feed</a></p>
|
||||
<p><a href="{base_url.rstrip('/')}/health">Health Check</a></p>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
@@ -0,0 +1,66 @@
|
||||
"""HTTP server for serving the RSS feed."""
|
||||
|
||||
from http.server import BaseHTTPRequestHandler
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from src.db import get_connection
|
||||
from src.rss import generate_feed, generate_index_html
|
||||
|
||||
|
||||
def create_handler(db_path: str):
|
||||
"""Factory that returns an HTTP request handler class bound to a DB path."""
|
||||
|
||||
class FeedHandler(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
parsed = urlparse(self.path)
|
||||
path = parsed.path.rstrip("/") or "/"
|
||||
|
||||
if path == "/feed.xml":
|
||||
self._serve_feed()
|
||||
elif path == "/health":
|
||||
self._serve_health()
|
||||
elif path == "/":
|
||||
self._serve_index()
|
||||
else:
|
||||
self.send_response(404)
|
||||
self.send_header("Content-Type", "text/plain")
|
||||
self.end_headers()
|
||||
self.wfile.write(b"Not Found")
|
||||
|
||||
def _serve_feed(self):
|
||||
conn = get_connection(db_path)
|
||||
try:
|
||||
from src.db import get_all_entries
|
||||
entries = get_all_entries(conn)
|
||||
feed_xml = generate_feed(entries, self._base_url())
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/rss+xml; charset=utf-8")
|
||||
self.end_headers()
|
||||
self.wfile.write(feed_xml.encode("utf-8"))
|
||||
|
||||
def _serve_index(self):
|
||||
html = generate_index_html(self._base_url())
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/html; charset=utf-8")
|
||||
self.end_headers()
|
||||
self.wfile.write(html.encode("utf-8"))
|
||||
|
||||
def _serve_health(self):
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/plain")
|
||||
self.end_headers()
|
||||
self.wfile.write(b"OK")
|
||||
|
||||
def _base_url(self) -> str:
|
||||
host = self.server.server_address[0]
|
||||
port = self.server.server_address[1]
|
||||
return f"http://{host}:{port}"
|
||||
|
||||
def log_message(self, format, *args):
|
||||
"""Suppress default request logging."""
|
||||
pass
|
||||
|
||||
return FeedHandler
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Tests for CLI commands."""
|
||||
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
import pytest
|
||||
|
||||
from src.db import _create_tables, add_repo
|
||||
|
||||
|
||||
class TestRepoFormatValidation:
|
||||
def test_valid_format(self):
|
||||
from src.cli import REPO_PATTERN
|
||||
assert REPO_PATTERN.match("owner/repo")
|
||||
assert REPO_PATTERN.match("my-org/my_repo")
|
||||
assert REPO_PATTERN.match("a.b/c-d_e")
|
||||
|
||||
def test_invalid_format(self):
|
||||
from src.cli import REPO_PATTERN
|
||||
assert not REPO_PATTERN.match("owner")
|
||||
assert not REPO_PATTERN.match("repo")
|
||||
assert not REPO_PATTERN.match("owner/repo/extra")
|
||||
|
||||
|
||||
class TestParseSince:
|
||||
def test_hours(self):
|
||||
from src.cli import parse_since
|
||||
from datetime import timedelta
|
||||
assert parse_since("24h") == timedelta(hours=24)
|
||||
|
||||
def test_days(self):
|
||||
from src.cli import parse_since
|
||||
from datetime import timedelta
|
||||
assert parse_since("2d") == timedelta(days=2)
|
||||
|
||||
def test_minutes(self):
|
||||
from src.cli import parse_since
|
||||
from datetime import timedelta
|
||||
assert parse_since("30m") == timedelta(minutes=30)
|
||||
|
||||
def test_invalid_defaults(self):
|
||||
from src.cli import parse_since
|
||||
from datetime import timedelta
|
||||
assert parse_since("invalid") == timedelta(hours=24)
|
||||
|
||||
|
||||
class TestCliHelp:
|
||||
def test_help_flag(self):
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "src.cli", "--help"],
|
||||
capture_output=True, text=True
|
||||
)
|
||||
assert result.returncode == 0
|
||||
assert "ghrel" in result.stdout.lower() or "track" in result.stdout.lower()
|
||||
|
||||
def test_version_flag(self):
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "src.cli", "--version"],
|
||||
capture_output=True, text=True
|
||||
)
|
||||
assert result.returncode == 0
|
||||
assert "0.1.0" in result.stdout
|
||||
|
||||
|
||||
class TestCliList:
|
||||
def test_list_empty(self, tmp_path):
|
||||
db_path = str(tmp_path / "test.db")
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "src.cli", "--db-path", db_path, "list"],
|
||||
capture_output=True, text=True
|
||||
)
|
||||
assert result.returncode == 0
|
||||
assert "No repositories" in result.stdout
|
||||
|
||||
|
||||
class TestCliAddRemove:
|
||||
def test_add_invalid_format(self, tmp_path):
|
||||
db_path = str(tmp_path / "test.db")
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "src.cli", "--db-path", db_path, "add", "invalid"],
|
||||
capture_output=True, text=True
|
||||
)
|
||||
assert result.returncode != 0
|
||||
assert "Invalid repo format" in result.stderr
|
||||
@@ -0,0 +1,143 @@
|
||||
"""Tests for the database layer."""
|
||||
|
||||
import os
|
||||
import stat
|
||||
import sqlite3
|
||||
import tempfile
|
||||
|
||||
import pytest
|
||||
|
||||
from src import db
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def conn():
|
||||
"""Provide an in-memory database connection for testing."""
|
||||
c = sqlite3.connect(":memory:")
|
||||
c.execute("PRAGMA foreign_keys=ON")
|
||||
db._create_tables(c)
|
||||
return c
|
||||
|
||||
|
||||
class TestCreateTables:
|
||||
def test_tables_exist(self, conn):
|
||||
tables = set(r[0] for r in conn.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table'"
|
||||
).fetchall())
|
||||
assert "repos" in tables
|
||||
assert "entries" in tables
|
||||
|
||||
def test_mode_check_constraint(self, conn):
|
||||
with pytest.raises(sqlite3.IntegrityError):
|
||||
conn.execute("INSERT INTO repos (owner_repo, mode) VALUES ('a/b', 'invalid')")
|
||||
conn.commit()
|
||||
|
||||
def test_kind_check_constraint(self, conn):
|
||||
conn.execute("INSERT INTO repos (owner_repo) VALUES ('a/b')")
|
||||
conn.commit()
|
||||
with pytest.raises(sqlite3.IntegrityError):
|
||||
conn.execute(
|
||||
"INSERT INTO entries (repo_id, kind, tag_name, title, published_at, html_url) "
|
||||
"VALUES (1, 'invalid', 'v1', 'v1', '2025-01-01', 'http://x')"
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
|
||||
class TestAddRepo:
|
||||
def test_add_repo_default_mode(self, conn):
|
||||
repo_id = db.add_repo(conn, "owner/repo")
|
||||
assert repo_id == 1
|
||||
row = conn.execute("SELECT owner_repo, mode FROM repos WHERE id = ?", (repo_id,)).fetchone()
|
||||
assert row == ("owner/repo", "release")
|
||||
|
||||
def test_add_repo_tag_mode(self, conn):
|
||||
repo_id = db.add_repo(conn, "owner/repo", "tag")
|
||||
row = conn.execute("SELECT mode FROM repos WHERE id = ?", (repo_id,)).fetchone()
|
||||
assert row[0] == "tag"
|
||||
|
||||
def test_add_repo_duplicate_fails(self, conn):
|
||||
db.add_repo(conn, "owner/repo")
|
||||
with pytest.raises(sqlite3.IntegrityError):
|
||||
db.add_repo(conn, "owner/repo")
|
||||
|
||||
|
||||
class TestRemoveRepo:
|
||||
def test_remove_existing(self, conn):
|
||||
db.add_repo(conn, "owner/repo")
|
||||
assert db.remove_repo(conn, "owner/repo") is True
|
||||
assert db.list_repos(conn) == []
|
||||
|
||||
def test_remove_nonexistent(self, conn):
|
||||
assert db.remove_repo(conn, "no/one") is False
|
||||
|
||||
def test_remove_cascades_to_entries(self, conn):
|
||||
repo_id = db.add_repo(conn, "owner/repo")
|
||||
db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x")
|
||||
db.remove_repo(conn, "owner/repo")
|
||||
assert db.get_entry_count(conn, repo_id) == 0
|
||||
|
||||
|
||||
class TestUpdateRepoMode:
|
||||
def test_update_mode(self, conn):
|
||||
db.add_repo(conn, "owner/repo")
|
||||
assert db.update_repo_mode(conn, "owner/repo", "tag") is True
|
||||
row = conn.execute("SELECT mode FROM repos WHERE owner_repo = 'owner/repo'").fetchone()
|
||||
assert row[0] == "tag"
|
||||
|
||||
def test_update_nonexistent(self, conn):
|
||||
assert db.update_repo_mode(conn, "no/one", "tag") is False
|
||||
|
||||
|
||||
class TestListRepos:
|
||||
def test_empty_list(self, conn):
|
||||
assert db.list_repos(conn) == []
|
||||
|
||||
def test_list_with_entries(self, conn):
|
||||
repo_id = db.add_repo(conn, "a/b")
|
||||
db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x")
|
||||
result = db.list_repos(conn)
|
||||
assert len(result) == 1
|
||||
assert result[0]["owner_repo"] == "a/b"
|
||||
assert result[0]["entries"] == 1
|
||||
assert result[0]["mode"] == "release"
|
||||
|
||||
|
||||
class TestUpsertEntry:
|
||||
def test_insert_new(self, conn):
|
||||
repo_id = db.add_repo(conn, "owner/repo")
|
||||
assert db.upsert_entry(conn, repo_id, "release", "v1", "Release v1", "body text",
|
||||
"2025-01-01T00:00:00Z", "http://example.com") is True
|
||||
|
||||
def test_skip_duplicate(self, conn):
|
||||
repo_id = db.add_repo(conn, "owner/repo")
|
||||
assert db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x") is True
|
||||
assert db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x") is False
|
||||
|
||||
|
||||
class TestGetAllEntries:
|
||||
def test_returns_entries_sorted_desc(self, conn):
|
||||
repo_id = db.add_repo(conn, "owner/repo")
|
||||
db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x")
|
||||
db.upsert_entry(conn, repo_id, "release", "v2", "v2", "", "2025-01-02", "http://y")
|
||||
entries = db.get_all_entries(conn)
|
||||
assert entries[0]["tag_name"] == "v2"
|
||||
assert entries[1]["tag_name"] == "v1"
|
||||
|
||||
def test_respects_limit(self, conn):
|
||||
repo_id = db.add_repo(conn, "owner/repo")
|
||||
for i in range(5):
|
||||
db.upsert_entry(conn, repo_id, "release", f"v{i}", f"v{i}", "", f"2025-01-0{i}", "http://x")
|
||||
entries = db.get_all_entries(conn, limit=3)
|
||||
assert len(entries) == 3
|
||||
|
||||
|
||||
class TestGetConnection:
|
||||
def test_creates_file(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = os.path.join(tmp, "test.db")
|
||||
c = db.get_connection(path)
|
||||
c.close()
|
||||
assert os.path.exists(path)
|
||||
# Check permissions
|
||||
mode = os.stat(path).st_mode
|
||||
assert stat.S_IMODE(mode) == 0o600
|
||||
@@ -0,0 +1,172 @@
|
||||
"""Tests for the GitHub API client."""
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
import respx
|
||||
|
||||
from src.github import GitHubClient, AuthenticationError, NotFoundError
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_api():
|
||||
"""Enable respx mocking for test scope."""
|
||||
with respx.mock:
|
||||
yield respx
|
||||
|
||||
|
||||
class TestFetchReleases:
|
||||
def test_fetch_releases(self, mock_api):
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/releases").mock(
|
||||
return_value=httpx.Response(200, json=[
|
||||
{
|
||||
"tag_name": "v1.0.0",
|
||||
"name": "Release v1.0.0",
|
||||
"body": "# Changelog\n\n- Fixed bug",
|
||||
"published_at": "2025-01-15T10:30:00Z",
|
||||
"html_url": "https://github.com/owner/repo/releases/tag/v1.0.0",
|
||||
},
|
||||
{
|
||||
"tag_name": "v0.9.0",
|
||||
"name": None,
|
||||
"body": "",
|
||||
"published_at": "2025-01-10T08:00:00Z",
|
||||
"html_url": "https://github.com/owner/repo/releases/tag/v0.9.0",
|
||||
},
|
||||
])
|
||||
)
|
||||
|
||||
client = GitHubClient(token="fake_token")
|
||||
releases = client.fetch_releases("owner/repo")
|
||||
client.close()
|
||||
|
||||
assert len(releases) == 2
|
||||
assert releases[0]["tag_name"] == "v1.0.0"
|
||||
assert releases[0]["title"] == "Release v1.0.0"
|
||||
assert releases[0]["body"] == "# Changelog\n\n- Fixed bug"
|
||||
assert releases[1]["title"] == "v0.9.0"
|
||||
assert releases[1]["body"] == ""
|
||||
|
||||
def test_fetch_releases_404(self, mock_api):
|
||||
mock_api.get("https://api.github.com/repos/no/such/releases").mock(
|
||||
return_value=httpx.Response(404)
|
||||
)
|
||||
|
||||
client = GitHubClient(token="fake_token")
|
||||
with pytest.raises(NotFoundError):
|
||||
client.fetch_releases("no/such")
|
||||
client.close()
|
||||
|
||||
|
||||
class TestFetchTags:
|
||||
def test_fetch_tags(self, mock_api):
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/tags").mock(
|
||||
return_value=httpx.Response(200, json=[
|
||||
{"name": "v1.0.0", "zipball_url": "https://api.github.com/repos/owner/repo/zipball/v1.0.0"},
|
||||
{"name": "v0.9.0", "zipball_url": "https://api.github.com/repos/owner/repo/zipball/v0.9.0"},
|
||||
])
|
||||
)
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/git/ref/tags/v1.0.0").mock(
|
||||
return_value=httpx.Response(200, json={
|
||||
"object": {
|
||||
"type": "commit",
|
||||
"author": {"date": "2025-01-15T10:30:00Z"},
|
||||
}
|
||||
})
|
||||
)
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/git/ref/tags/v0.9.0").mock(
|
||||
return_value=httpx.Response(200, json={
|
||||
"object": {
|
||||
"type": "commit",
|
||||
"author": {"date": "2025-01-10T08:00:00Z"},
|
||||
}
|
||||
})
|
||||
)
|
||||
|
||||
client = GitHubClient(token="fake_token")
|
||||
tags = client.fetch_tags("owner/repo")
|
||||
client.close()
|
||||
|
||||
assert len(tags) == 2
|
||||
assert tags[0]["tag_name"] == "v1.0.0"
|
||||
assert tags[0]["title"] == "v1.0.0"
|
||||
assert tags[0]["body"] == ""
|
||||
assert tags[0]["published_at"] == "2025-01-15T10:30:00Z"
|
||||
assert tags[0]["html_url"] == "https://github.com/owner/repo/tags/v1.0.0"
|
||||
|
||||
def test_fetch_tags_with_tag_object(self, mock_api):
|
||||
"""When a tag points to a tag object, follow to the commit."""
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/tags").mock(
|
||||
return_value=httpx.Response(200, json=[
|
||||
{"name": "v2.0.0", "zipball_url": "https://api.github.com/..."},
|
||||
])
|
||||
)
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/git/ref/tags/v2.0.0").mock(
|
||||
return_value=httpx.Response(200, json={
|
||||
"object": {
|
||||
"type": "tag",
|
||||
"sha": "abc123",
|
||||
}
|
||||
})
|
||||
)
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/git/commits/abc123").mock(
|
||||
return_value=httpx.Response(200, json={
|
||||
"author": {"date": "2025-02-01T12:00:00Z"}
|
||||
})
|
||||
)
|
||||
|
||||
client = GitHubClient(token="fake_token")
|
||||
tags = client.fetch_tags("owner/repo")
|
||||
client.close()
|
||||
|
||||
assert tags[0]["published_at"] == "2025-02-01T12:00:00Z"
|
||||
|
||||
def test_fetch_tags_fallback_date(self, mock_api):
|
||||
"""When git ref fails, use current time as fallback."""
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/tags").mock(
|
||||
return_value=httpx.Response(200, json=[
|
||||
{"name": "v1.0.0", "zipball_url": "https://api.github.com/..."},
|
||||
])
|
||||
)
|
||||
mock_api.get("https://api.github.com/repos/owner/repo/git/ref/tags/v1.0.0").mock(
|
||||
return_value=httpx.Response(404)
|
||||
)
|
||||
|
||||
client = GitHubClient(token="fake_token")
|
||||
tags = client.fetch_tags("owner/repo")
|
||||
client.close()
|
||||
|
||||
assert tags[0]["published_at"] is not None
|
||||
|
||||
|
||||
class TestRepoExists:
|
||||
def test_repo_exists_true(self, mock_api):
|
||||
mock_api.get("https://api.github.com/repos/owner/repo").mock(
|
||||
return_value=httpx.Response(200, json={"full_name": "owner/repo"})
|
||||
)
|
||||
client = GitHubClient(token="fake_token")
|
||||
assert client.repo_exists("owner/repo") is True
|
||||
client.close()
|
||||
|
||||
def test_repo_exists_false(self, mock_api):
|
||||
mock_api.get("https://api.github.com/repos/no/such").mock(
|
||||
return_value=httpx.Response(404)
|
||||
)
|
||||
client = GitHubClient(token="fake_token")
|
||||
assert client.repo_exists("no/such") is False
|
||||
client.close()
|
||||
|
||||
|
||||
class TestAuthentication:
|
||||
def test_invalid_token(self, mock_api):
|
||||
mock_api.get("https://api.github.com/rate_limit").mock(
|
||||
return_value=httpx.Response(401)
|
||||
)
|
||||
client = GitHubClient(token="bad_token")
|
||||
with pytest.raises(AuthenticationError, match="401 Unauthorized"):
|
||||
client.validate_token()
|
||||
client.close()
|
||||
|
||||
def test_no_token_skips_validation(self):
|
||||
client = GitHubClient(token=None)
|
||||
client.validate_token()
|
||||
client.close()
|
||||
@@ -0,0 +1,205 @@
|
||||
"""End-to-end integration tests."""
|
||||
|
||||
import sqlite3
|
||||
import threading
|
||||
import tempfile
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
import respx
|
||||
|
||||
from src.db import _create_tables, add_repo, upsert_entry, get_connection
|
||||
from src.server import create_handler
|
||||
from http.server import HTTPServer
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_github():
|
||||
"""Mock only GitHub API calls."""
|
||||
with respx.mock:
|
||||
yield respx
|
||||
|
||||
|
||||
def fetch_feed(port: int) -> str:
|
||||
"""Fetch feed from localhost using urllib (bypasses respx)."""
|
||||
url = f"http://127.0.0.1:{port}/feed.xml"
|
||||
with urllib.request.urlopen(url) as resp:
|
||||
return resp.read().decode("utf-8")
|
||||
|
||||
|
||||
class TestFullFlowWithMockedAPI:
|
||||
"""Test the full flow: add repo, check (with mocked API), serve, fetch feed."""
|
||||
|
||||
def test_release_mode_full_flow(self, mock_github):
|
||||
mock_github.get("https://api.github.com/repos/signoz/signoz/releases").mock(
|
||||
return_value=httpx.Response(200, json=[
|
||||
{
|
||||
"tag_name": "v1.0.0",
|
||||
"name": "Signoz v1.0.0",
|
||||
"body": "# Release v1.0.0\n\n- New dashboard features",
|
||||
"published_at": "2025-01-15T10:30:00Z",
|
||||
"html_url": "https://github.com/signoz/signoz/releases/tag/v1.0.0",
|
||||
}
|
||||
])
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
db_path = Path(tmp) / "test.db"
|
||||
conn = get_connection(db_path)
|
||||
add_repo(conn, "signoz/signoz", "release")
|
||||
conn.close()
|
||||
|
||||
from src.github import GitHubClient
|
||||
from src.db import get_repos_with_mode, upsert_entry as db_upsert
|
||||
|
||||
conn = get_connection(db_path)
|
||||
repos = get_repos_with_mode(conn)
|
||||
assert len(repos) == 1
|
||||
|
||||
with GitHubClient(token="fake_token") as gh:
|
||||
for repo_id, owner_repo, mode in repos:
|
||||
items = gh.fetch_releases(owner_repo)
|
||||
for item in items:
|
||||
db_upsert(conn, repo_id, mode,
|
||||
item["tag_name"], item["title"], item["body"],
|
||||
item["published_at"], item["html_url"])
|
||||
|
||||
conn.close()
|
||||
|
||||
handler = create_handler(str(db_path))
|
||||
server = HTTPServer(("127.0.0.1", 0), handler)
|
||||
port = server.server_address[1]
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
|
||||
try:
|
||||
text = fetch_feed(port)
|
||||
assert "signoz/signoz" in text
|
||||
assert "v1.0.0" in text
|
||||
assert "Signoz v1.0.0" in text
|
||||
finally:
|
||||
server.shutdown()
|
||||
|
||||
def test_tag_mode_full_flow(self, mock_github):
|
||||
mock_github.get("https://api.github.com/repos/zammad/zammad/tags").mock(
|
||||
return_value=httpx.Response(200, json=[
|
||||
{"name": "6.3.0", "zipball_url": "https://api.github.com/..."},
|
||||
])
|
||||
)
|
||||
mock_github.get("https://api.github.com/repos/zammad/zammad/git/ref/tags/6.3.0").mock(
|
||||
return_value=httpx.Response(200, json={
|
||||
"object": {
|
||||
"type": "commit",
|
||||
"author": {"date": "2025-02-01T12:00:00Z"},
|
||||
}
|
||||
})
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
db_path = Path(tmp) / "test.db"
|
||||
conn = get_connection(db_path)
|
||||
add_repo(conn, "zammad/zammad", "tag")
|
||||
conn.close()
|
||||
|
||||
from src.github import GitHubClient
|
||||
from src.db import get_repos_with_mode, upsert_entry as db_upsert
|
||||
|
||||
conn = get_connection(db_path)
|
||||
repos = get_repos_with_mode(conn)
|
||||
|
||||
with GitHubClient(token="fake_token") as gh:
|
||||
for repo_id, owner_repo, mode in repos:
|
||||
items = gh.fetch_tags(owner_repo)
|
||||
for item in items:
|
||||
db_upsert(conn, repo_id, "tag",
|
||||
item["tag_name"], item["title"], item["body"],
|
||||
item["published_at"], item["html_url"])
|
||||
|
||||
conn.close()
|
||||
|
||||
handler = create_handler(str(db_path))
|
||||
server = HTTPServer(("127.0.0.1", 0), handler)
|
||||
port = server.server_address[1]
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
|
||||
try:
|
||||
text = fetch_feed(port)
|
||||
assert "zammad/zammad" in text
|
||||
assert "6.3.0" in text
|
||||
assert "Tag 6.3.0" in text
|
||||
finally:
|
||||
server.shutdown()
|
||||
|
||||
|
||||
class TestMultipleRepos:
|
||||
def test_mixed_repos_in_feed(self, mock_github):
|
||||
"""Feed should contain entries from both release and tag repos, sorted by date."""
|
||||
mock_github.get("https://api.github.com/repos/a/b/releases").mock(
|
||||
return_value=httpx.Response(200, json=[
|
||||
{
|
||||
"tag_name": "v1.0.0",
|
||||
"name": "v1.0.0",
|
||||
"body": "Release body",
|
||||
"published_at": "2025-01-10T00:00:00Z",
|
||||
"html_url": "https://github.com/a/b/releases/tag/v1.0.0",
|
||||
}
|
||||
])
|
||||
)
|
||||
mock_github.get("https://api.github.com/repos/c/d/tags").mock(
|
||||
return_value=httpx.Response(200, json=[
|
||||
{"name": "v2.0.0", "zipball_url": "https://api.github.com/..."},
|
||||
])
|
||||
)
|
||||
mock_github.get("https://api.github.com/repos/c/d/git/ref/tags/v2.0.0").mock(
|
||||
return_value=httpx.Response(200, json={
|
||||
"object": {
|
||||
"type": "commit",
|
||||
"author": {"date": "2025-01-20T00:00:00Z"},
|
||||
}
|
||||
})
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
db_path = Path(tmp) / "test.db"
|
||||
conn = get_connection(db_path)
|
||||
add_repo(conn, "a/b", "release")
|
||||
add_repo(conn, "c/d", "tag")
|
||||
conn.close()
|
||||
|
||||
from src.github import GitHubClient
|
||||
from src.db import get_repos_with_mode, upsert_entry as db_upsert
|
||||
|
||||
conn = get_connection(db_path)
|
||||
repos = get_repos_with_mode(conn)
|
||||
|
||||
with GitHubClient(token="fake_token") as gh:
|
||||
for repo_id, owner_repo, mode in repos:
|
||||
if mode == "tag":
|
||||
items = gh.fetch_tags(owner_repo)
|
||||
kind = "tag"
|
||||
else:
|
||||
items = gh.fetch_releases(owner_repo)
|
||||
kind = "release"
|
||||
for item in items:
|
||||
db_upsert(conn, repo_id, kind,
|
||||
item["tag_name"], item["title"], item["body"],
|
||||
item["published_at"], item["html_url"])
|
||||
|
||||
conn.close()
|
||||
|
||||
handler = create_handler(str(db_path))
|
||||
server = HTTPServer(("127.0.0.1", 0), handler)
|
||||
port = server.server_address[1]
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
|
||||
try:
|
||||
text = fetch_feed(port)
|
||||
c_d_pos = text.find("c/d")
|
||||
a_b_pos = text.find("a/b")
|
||||
assert c_d_pos < a_b_pos
|
||||
finally:
|
||||
server.shutdown()
|
||||
@@ -0,0 +1,134 @@
|
||||
"""Tests for RSS feed generation."""
|
||||
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from src.rss import generate_feed, strip_markdown, format_rfc822
|
||||
|
||||
|
||||
class TestStripMarkdown:
|
||||
def test_plain_text(self):
|
||||
assert strip_markdown("Hello world") == "Hello world"
|
||||
|
||||
def test_bold_and_lists(self):
|
||||
result = strip_markdown("**bold** and - item1\n- item2")
|
||||
assert "bold" in result
|
||||
assert "item1" in result
|
||||
|
||||
def test_empty_string(self):
|
||||
assert strip_markdown("") == ""
|
||||
|
||||
def test_none(self):
|
||||
assert strip_markdown("") == ""
|
||||
|
||||
def test_code_blocks(self):
|
||||
result = strip_markdown("```python\nprint('hello')\n```")
|
||||
assert "print" in result or "hello" in result
|
||||
|
||||
|
||||
class TestFormatRfc822:
|
||||
def test_iso_to_rfc822(self):
|
||||
result = format_rfc822("2025-01-15T10:30:00Z")
|
||||
assert "Wed" in result or "15" in result
|
||||
assert "2025" in result
|
||||
|
||||
def test_with_tz_offset(self):
|
||||
result = format_rfc822("2025-01-15T10:30:00+00:00")
|
||||
assert "2025" in result
|
||||
|
||||
|
||||
class TestGenerateFeed:
|
||||
def _entries(self, count=1):
|
||||
return [
|
||||
{
|
||||
"tag_name": f"v1.{i}.0",
|
||||
"title": f"Release v1.{i}.0",
|
||||
"body": f"# Release notes\n\n- Feature {i}",
|
||||
"published_at": "2025-01-15T10:30:00Z",
|
||||
"html_url": f"https://github.com/owner/repo/releases/tag/v1.{i}.0",
|
||||
"owner_repo": "owner/repo",
|
||||
}
|
||||
for i in range(count)
|
||||
]
|
||||
|
||||
def test_basic_feed(self):
|
||||
entries = self._entries(2)
|
||||
xml_str = generate_feed(entries)
|
||||
root = ET.fromstring(xml_str)
|
||||
assert root.tag == "rss"
|
||||
assert root.get("version") == "2.0"
|
||||
|
||||
channel = root[0]
|
||||
assert channel.tag == "channel"
|
||||
assert channel.find("title").text == "GitHub Release Monitor"
|
||||
assert channel.find("description").text == "Latest releases from tracked GitHub repositories"
|
||||
|
||||
items = channel.findall("item")
|
||||
assert len(items) == 2
|
||||
|
||||
def test_item_structure(self):
|
||||
entries = self._entries(1)
|
||||
xml_str = generate_feed(entries)
|
||||
root = ET.fromstring(xml_str)
|
||||
channel = root[0]
|
||||
item = channel.find("item")
|
||||
|
||||
assert item.find("title").text == "[owner/repo] Release v1.0.0"
|
||||
assert "Feature 0" in item.find("description").text
|
||||
assert item.find("link").text == "https://github.com/owner/repo/releases/tag/v1.0.0"
|
||||
assert item.find("pubDate").text is not None
|
||||
assert item.find("guid").text == "github.com/owner/repo/releases/tag/v1.0.0"
|
||||
|
||||
def test_description_truncation(self):
|
||||
entry = {
|
||||
"tag_name": "v1.0.0",
|
||||
"title": "Release",
|
||||
"body": "A" * 400,
|
||||
"published_at": "2025-01-15T10:30:00Z",
|
||||
"html_url": "https://example.com",
|
||||
"owner_repo": "owner/repo",
|
||||
}
|
||||
xml_str = generate_feed([entry])
|
||||
root = ET.fromstring(xml_str)
|
||||
desc = root[0].find("item").find("description").text
|
||||
assert len(desc) == 301 # 300 chars + 1 ellipsis char
|
||||
|
||||
def test_tag_entry_no_body(self):
|
||||
entry = {
|
||||
"tag_name": "v1.0.0",
|
||||
"title": "v1.0.0",
|
||||
"body": "",
|
||||
"published_at": "2025-01-15T10:30:00Z",
|
||||
"html_url": "https://github.com/owner/repo/tags/v1.0.0",
|
||||
"owner_repo": "owner/repo",
|
||||
}
|
||||
xml_str = generate_feed([entry])
|
||||
root = ET.fromstring(xml_str)
|
||||
desc = root[0].find("item").find("description").text
|
||||
assert desc == "Tag v1.0.0"
|
||||
|
||||
def test_item_cap(self):
|
||||
entries = self._entries(100)
|
||||
xml_str = generate_feed(entries)
|
||||
root = ET.fromstring(xml_str)
|
||||
items = root[0].findall("item")
|
||||
assert len(items) == 50 # RSS_MAX_ITEMS
|
||||
|
||||
def test_xml_escaping(self):
|
||||
entry = {
|
||||
"tag_name": "v1.0.0",
|
||||
"title": "Release with <special> & 'chars'",
|
||||
"body": "Some **bold** text",
|
||||
"published_at": "2025-01-15T10:30:00Z",
|
||||
"html_url": "https://example.com?a=1&b=2",
|
||||
"owner_repo": "owner/repo",
|
||||
}
|
||||
xml_str = generate_feed([entry])
|
||||
# Should parse without error
|
||||
root = ET.fromstring(xml_str)
|
||||
assert root is not None
|
||||
|
||||
def test_base_url(self):
|
||||
entries = self._entries(1)
|
||||
xml_str = generate_feed(entries, base_url="http://example.com:9000")
|
||||
root = ET.fromstring(xml_str)
|
||||
assert root[0].find("link").text == "http://example.com:9000/"
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Tests for the HTTP server."""
|
||||
|
||||
import sqlite3
|
||||
import threading
|
||||
import tempfile
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from src.db import _create_tables, add_repo, upsert_entry
|
||||
from src.server import create_handler
|
||||
from http.server import HTTPServer
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db_path(tmp_path):
|
||||
"""Create a temporary database with sample data."""
|
||||
path = str(tmp_path / "test.db")
|
||||
conn = __import__("sqlite3").connect(path)
|
||||
_create_tables(conn)
|
||||
repo_id = add_repo(conn, "owner/repo")
|
||||
upsert_entry(conn, repo_id, "release", "v1.0.0", "Release v1.0.0",
|
||||
"# Changelog\n\n- New feature",
|
||||
"2025-01-15T10:30:00Z",
|
||||
"https://github.com/owner/repo/releases/tag/v1.0.0")
|
||||
conn.close()
|
||||
return path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def server(db_path):
|
||||
"""Start a test HTTP server in a background thread."""
|
||||
handler = create_handler(db_path)
|
||||
server = HTTPServer(("127.0.0.1", 0), handler)
|
||||
port = server.server_address[1]
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
yield f"http://127.0.0.1:{port}"
|
||||
server.shutdown()
|
||||
|
||||
|
||||
class TestFeedEndpoint:
|
||||
def test_feed_returns_200(self, server):
|
||||
resp = httpx.get(f"{server}/feed.xml")
|
||||
assert resp.status_code == 200
|
||||
|
||||
def test_feed_content_type(self, server):
|
||||
resp = httpx.get(f"{server}/feed.xml")
|
||||
assert "application/rss+xml" in resp.headers["content-type"]
|
||||
|
||||
def test_feed_contains_entry(self, server):
|
||||
resp = httpx.get(f"{server}/feed.xml")
|
||||
assert "owner/repo" in resp.text
|
||||
assert "v1.0.0" in resp.text
|
||||
|
||||
def test_feed_is_valid_xml(self, server):
|
||||
import xml.etree.ElementTree as ET
|
||||
resp = httpx.get(f"{server}/feed.xml")
|
||||
root = ET.fromstring(resp.text)
|
||||
assert root.tag == "rss"
|
||||
|
||||
|
||||
class TestIndexEndpoint:
|
||||
def test_index_returns_200(self, server):
|
||||
resp = httpx.get(server + "/")
|
||||
assert resp.status_code == 200
|
||||
|
||||
def test_index_content_type(self, server):
|
||||
resp = httpx.get(server + "/")
|
||||
assert "text/html" in resp.headers["content-type"]
|
||||
|
||||
def test_index_links_to_feed(self, server):
|
||||
resp = httpx.get(server + "/")
|
||||
assert "/feed.xml" in resp.text
|
||||
|
||||
|
||||
class TestHealthEndpoint:
|
||||
def test_health_returns_200(self, server):
|
||||
resp = httpx.get(f"{server}/health")
|
||||
assert resp.status_code == 200
|
||||
|
||||
def test_health_body(self, server):
|
||||
resp = httpx.get(f"{server}/health")
|
||||
assert resp.text == "OK"
|
||||
|
||||
|
||||
class TestNotFound:
|
||||
def test_unknown_path(self, server):
|
||||
resp = httpx.get(f"{server}/unknown")
|
||||
assert resp.status_code == 404
|
||||
Reference in New Issue
Block a user