Initial commit: GitHub Release Monitor CLI tool

CLI tool that tracks GitHub releases or tags from user-defined repositories
and exposes them as a subscribable RSS 2.0 feed.

Features:
- Track releases or tags per repository
- SQLite storage with automatic deduplication
- RSS 2.0 feed with markdown-stripped release notes
- Built-in HTTP server for local feed access
- Rate-limit-aware GitHub API client
- 64 passing tests
This commit is contained in:
2026-07-28 11:56:10 +02:00
commit 2a5f11bd9b
18 changed files with 2117 additions and 0 deletions
+16
View File
@@ -0,0 +1,16 @@
__pycache__/
*.py[cod]
*.so
*.egg-info/
dist/
build/
.eggs/
*.egg
.pytest_cache/
.mypy_cache/
.coverage
htmlcov/
*.db
.env
venv/
.venv/
+181
View File
@@ -0,0 +1,181 @@
# GitHub Release Monitor
A CLI tool that tracks GitHub releases or tags from user-defined repositories and exposes them as a subscribable RSS feed.
Some repositories (e.g., Zammad) only use git tags without creating GitHub releases — both modes are supported.
## Features
- Track **releases** or **tags** per repository
- SQLite storage with automatic deduplication
- RSS 2.0 feed with markdown-stripped release notes
- Built-in HTTP server for local feed access
- Rate-limit-aware GitHub API client
- Configurable via environment variables and CLI flags
## Installation
```bash
pip install .
```
Or install in development mode:
```bash
pip install -e .
```
## Quick Start
```bash
# Set your GitHub token (optional, increases rate limit from 60/h to 5000/h)
export GH_TOKEN=your_token
# Add repositories to track
ghrel add golang/go # Track releases (default)
ghrel add zammad/zammad --tags # Track tags
# Fetch latest releases/tags
ghrel check
# List tracked repositories
ghrel list
# Start the RSS feed server
ghrel serve
```
The RSS feed is available at `http://127.0.0.1:8080/feed.xml`.
## CLI Commands
```
ghrel # Main command
├── --version # Show version
├── --verbose / -v # Enable verbose/debug output
├── --quiet / -q # Suppress non-essential output
├── --db-path <path> # Override database path
├── add <owner/repo> [--tags] # Add a repository to track
├── remove <owner/repo> # Remove a repository
├── update <owner/repo> [--tags] [--release] # Update tracking mode
├── list # List all tracked repositories
├── check [--since N] # Fetch latest releases/tags
└── serve [--port 8080] [--host 127.0.0.1] # Start HTTP server
```
### `add <owner/repo> [--tags]`
Add a repository to track. Validates the format and pings the GitHub API to verify the repository exists.
```bash
ghrel add golang/go # Track releases
ghrel add zammad/zammad --tags # Track tags
```
### `remove <owner/repo>`
Remove a repository and its associated entries.
```bash
ghrel remove golang/go
```
### `update <owner/repo> [--tags] [--release]`
Update the tracking mode for an existing repository.
```bash
ghrel update zammad/zammad --tags
ghrel update zammad/zammad --release
```
### `list`
Display tracked repositories with entry counts and last check time.
```
REPO MODE ENTRIES LAST CHECKED
golang/go release 12 2025-01-15 10:30
zammad/zammad tag 8 2025-01-15 10:30
```
### `check [--since N]`
Fetch latest releases or tags for all tracked repositories. With `--since`, only fetch entries published in the last N hours (e.g., `--since 24h`).
```bash
ghrel check # Fetch all new entries
ghrel check --since 24h # Only entries from the last 24 hours
```
### `serve [--port PORT] [--host HOST]`
Start the HTTP server.
```bash
ghrel serve # Default: 127.0.0.1:8080
ghrel serve --port 9090 --host 0.0.0.0 # Custom port and host
```
## Configuration
| Setting | Default | Override |
|---------------|----------------------------|-------------------|
| `db_path` | `~/.config/ghrel/repos.db` | `--db-path` flag |
| `gh_token` | `GH_TOKEN` env var | — |
| `server_host` | `127.0.0.1` | `--host` flag |
| `server_port` | `8080` | `--port` flag |
## RSS Feed
The feed follows RSS 2.0 specification:
- **Title**: `GitHub Release Monitor`
- **Items**: Sorted by `published_at` (newest first), capped at 50 items
- **Release entries**: Title from release name, description from markdown-stripped body (truncated at 300 characters)
- **Tag entries**: Title from tag name, description as `Tag {tag_name}`
## Architecture
```
┌──────────────────────────────────────────────────┐
│ CLI Interface │
│ (add, remove, update, list, check, serve) │
├──────────────────────────────────────────────────┤
│ Core Logic │
│ ┌──────────────┐ ┌──────────────┐ ┌────────┐ │
│ │ config.py │ │ db.py │ │github.py│ │
│ │ (defaults, │ │ (sqlite3) │ │(httpx) │ │
│ │ env vars) │ │ │ │ │ │
│ └──────────────┘ └──────┬───────┘ └───┬────┘ │
│ │ │ │
│ ┌────────▼──────────────▼────┐ │
│ │ rss.py │ │
│ │ (xml.etree + markdown) │ │
│ └──────────────┬─────────────┘ │
│ │ │
│ ┌──────────────▼─────────────┐ │
│ │ server.py │ │
│ │ (http.server, stdlib) │ │
│ │ (/feed.xml endpoint) │ │
│ └─────────────────────────────┘ │
└──────────────────────────────────────────────────┘
```
## Testing
```bash
pytest
```
## Dependencies
| Package | Purpose |
|--------------|------------------------------------------|
| `httpx` | HTTP client for GitHub API |
| `markdown` | Convert markdown to HTML |
| `bleach` | Strip HTML tags for plain text |
| `pytest` | Testing framework |
| `respx` | Mocking HTTP requests for tests |
No heavy frameworks. `http.server` and `xml.etree.ElementTree` are stdlib.
+274
View File
@@ -0,0 +1,274 @@
# GitHub Release Monitor — Project Plan
## 1. Overview
A CLI tool that tracks GitHub releases or tags from user-defined repositories and exposes them as a subscribable RSS feed. Some repos (e.g., Zammad) only use git tags without creating GitHub releases — both modes are supported.
- **Language**: Python 3.12
- **Storage**: SQLite (single file, embedded via `sqlite3` stdlib)
- **API**: GitHub REST API via `httpx`
- **Output**: RSS 2.0 XML feed served over HTTP (stdlib `http.server`)
## 2. Architecture
```
┌──────────────────────────────────────────────────┐
│ CLI Interface │
│ (add, remove, update, list, check, serve) │
├──────────────────────────────────────────────────┤
│ Core Logic │
│ ┌──────────────┐ ┌──────────────┐ ┌────────┐ │
│ │ config.py │ │ db.py │ │github.py│ │
│ │ (defaults, │ │ (sqlite3) │ │(httpx) │ │
│ │ env vars) │ │ │ │ │ │
│ └──────────────┘ └──────┬───────┘ └───┬────┘ │
│ │ │ │
│ ┌────────▼──────────────▼────┐ │
│ │ rss.py │ │
│ │ (xml.etree + markdown) │ │
│ └──────────────┬─────────────┘ │
│ │ │
│ ┌──────────────▼─────────────┐ │
│ │ server.py │ │
│ │ (http.server, stdlib) │ │
│ │ (/feed.xml endpoint) │ │
│ └─────────────────────────────┘ │
└──────────────────────────────────────────────────┘
```
## 3. File Structure
```
github_release_monitor/
├── plan.md # This file
├── pyproject.toml # Project metadata, dependencies
├── README.md
├── src/
│ ├── __init__.py
│ ├── cli.py # CLI entry point (argparse)
│ ├── config.py # Centralized config (defaults, env vars, paths)
│ ├── db.py # SQLite schema & queries
│ ├── github.py # GitHub API interaction
│ ├── rss.py # RSS feed generation
│ └── server.py # HTTP server for feed
├── tests/
│ ├── __init__.py
│ ├── test_db.py
│ ├── test_github.py
│ ├── test_rss.py
│ ├── test_server.py
│ └── test_integration.py # End-to-end tests
└── .gitignore
```
## 4. Database Schema
### `repos` table
| Column | Type | Constraints | Description |
|------------|--------------|--------------------------------------|--------------------------------------|
| id | INTEGER | PRIMARY KEY AUTO | Internal identifier |
| owner_repo | TEXT | NOT NULL UNIQUE | GitHub repo identifier (`owner/repo`)|
| mode | TEXT | NOT NULL DEFAULT 'release' CHECK(mode IN ('release', 'tag')) | Track releases or tags |
| created_at | DATETIME | DEFAULT NOW() | When the repo was added |
The `owner_repo` field serves as both the unique identifier and the display name. The `mode` field determines whether to fetch GitHub releases (`release`) or git tags (`tag`). Default is `release`.
### `entries` table
| Column | Type | Constraints | Description |
|--------------|--------------|--------------------|----------------------------------------|
| id | INTEGER | PRIMARY KEY AUTO | Internal identifier |
| repo_id | INTEGER | FK → repos.id | Which repository this entry belongs to |
| kind | TEXT | NOT NULL CHECK(kind IN ('release', 'tag')) | Whether this is a release or tag |
| tag_name | TEXT | NOT NULL | Git tag / release tag |
| title | TEXT | NOT NULL | Release title or tag name |
| body | TEXT | DEFAULT '' | Release notes (empty for tags) |
| published_at | DATETIME | NOT NULL | Release date or commit date |
| html_url | TEXT | NOT NULL | Link to release or commit on GitHub |
| fetched_at | DATETIME | DEFAULT NOW() | When we fetched this entry |
**Unique constraint**: `(repo_id, tag_name)` to avoid duplicates.
## 5. CLI Commands
```
ghrel # Main command
├── --version # Show version
├── --verbose / -v # Enable verbose/debug output
├── --quiet / -q # Suppress non-essential output
├── --db-path <path> # Override database path (default: ~/.config/ghrel/repos.db)
├── add <owner/repo> [--tags] # Add a repository to track
├── remove <owner/repo> # Remove a repository
├── update <owner/repo> [--tags] [--release] # Update tracking mode
├── list # List all tracked repositories
├── check [--since N] # Fetch latest releases/tags for all repos
└── serve [--port 8080] [--host 127.0.0.1] # Start HTTP server
```
### `add <owner/repo> [--tags]`
- Accepts a single `owner/repo` argument (e.g., `golang/go`)
- `--tags` flag: track git tags instead of releases (default: releases)
- Validates the format with a regex: `^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$`
- Pings the GitHub API to verify the repo exists (with token validation)
- Inserts into `repos` table; errors on duplicate `owner_repo`
- On success: `Added golang/go (releases)` or `Added zammad/zammad (tags)`
### `remove <owner/repo>`
- Deletes from `repos` table by `owner_repo`
- Cascading delete removes associated entries
- On success: `Removed golang/go`
### `update <owner/repo> [--tags] [--release]`
- Updates the tracking mode for an existing repo
- `--tags`: switch to tracking tags
- `--release`: switch to tracking releases
- On success: `Updated zammad/zammad to track tags`
### `list`
- Displays a table of tracked repos:
```
REPO MODE ENTRIES LAST CHECKED
golang/go release 12 2025-01-15 10:30
zammad/zammad tag 8 2025-01-15 10:30
```
### `check [--since N]`
- Iterates over all repos in the database
- For each repo, fetches latest releases or tags depending on the repo's `mode`
- If `--since N` is provided (e.g., `--since 24h`), only fetches entries published in the last N hours
- Upserts into `entries` table (insert if new, skip if already known by `tag_name`)
- Reports results: `Checked 5 repos: 3 new entries found`
- Global rate-limit-aware queue: if one repo hits the rate limit, remaining repos wait
### `serve [--port 8080] [--host 127.0.0.1]`
- Starts an HTTP server using stdlib `http.server`
- Default host: `127.0.0.1` (localhost only, not exposed to network)
- Serves RSS feed at `/feed.xml`
- Serves a small index page at `/` with a link to the feed
- Health check endpoint at `/health` returns `200 OK`
### Error format
All errors go to stderr with non-zero exit code:
```
Error: <message>
```
## 6. GitHub API Integration
### Approach: Direct HTTP calls via `httpx`
- **Authentication**: Read `GH_TOKEN` from environment variable. If set, use it for authenticated requests (5000/h vs 60/h). If not set, fall back to unauthenticated.
- **Token validation**: On first API call after startup, check `GET /rate_limit` to verify the token is valid. If `401`, print a clear error: `Error: Invalid or missing GH_TOKEN. Set it with: export GH_TOKEN=your_token`
- **Pagination**: Fetch first page only (30 items). Sufficient for tracking recent entries.
- **Rate limiting**: Track `X-RateLimit-Remaining` header globally across all repos. If rate limited, pause all requests until `X-RateLimit-Reset` time. No per-repo retries — a single global wait prevents cascading 403s.
### Two API endpoints depending on `mode`
**Releases** (`mode = 'release'`):
- `GET https://api.github.com/repos/{owner}/{repo}/releases?per_page=30`
- Data extracted: `tag_name`, `name` (title), `body` (markdown), `published_at`, `html_url`
**Tags** (`mode = 'tag'`):
- `GET https://api.github.com/repos/{owner}/{repo}/tags?per_page=30`
- Data extracted: `name` (used as both `tag_name` and `title`), `zipball_url` → derive `html_url` as `https://github.com/{owner}/{repo}/tags/{name}`
- Tags have no `body` — store empty string
- Tags have no `published_at` — fetch commit date via `GET /repos/{owner}/{repo}/git/ref/tags/{name}` and use `object.commit.author.date` from the resolved commit object
## 7. RSS Feed Specification
RSS 2.0 generated with stdlib `xml.etree.ElementTree`. RSS 2.0 is simple enough that `feedgen` is unnecessary.
### Feed-level
- **title**: "GitHub Release Monitor"
- **description**: "Latest releases from tracked GitHub repositories"
- **link**: `http://{host}:{port}/` (base URL of the serving instance)
- **lastBuildDate**: Current time in RFC 822 format
### Per-item
- **title**: `[{owner/repo}] {title}`
- **description**: First 300 characters of `body`, with markdown stripped to plain text. Append `` if truncated. For tags (no body), use: `Tag {tag_name}`
- **link**: `html_url` from the entry
- **pubDate**: `published_at` in RFC 822 format
- **guid**: `github.com/{owner}/{repo}/releases/tag/{tag_name}` (unique identifier, same format for both releases and tags)
Items sorted by `published_at` descending (newest first). **Capped at 50 items** to prevent feed bloat.
### Markdown stripping
Use `markdown` library to convert markdown to HTML, then `bleach` to strip all HTML tags and produce plain text.
## 8. Configuration (`config.py`)
Centralized configuration module:
| Setting | Default | Override |
|------------------|----------------------------|-------------------|
| `db_path` | `~/.config/ghrel/repos.db` | `--db-path` flag |
| `gh_token` | `GH_TOKEN` env var | — |
| `server_host` | `127.0.0.1` | `--host` flag |
| `server_port` | `8080` | `--port` flag |
| `rss_max_items` | `50` | — |
| `api_per_page` | `30` | — |
On database creation, set file permissions to `0600` (owner read/write only) to avoid leaking tracked repo list.
## 9. Error Handling
- **GitHub API errors** (404, 403, etc.): Log warning, skip the repo, continue with others
- **Network errors**: Retry with exponential backoff (3 attempts, 1s / 2s / 4s), then skip
- **Rate limit exceeded**: Log warning with reset time, pause all requests globally
- **Invalid repo URL**: Validate format on `add`, show clear error: `Error: Invalid repo format 'foo'. Expected 'owner/repo'`
- **Invalid token**: Clear error on first API call: `Error: GH_TOKEN is invalid (401 Unauthorized)`
- **Database errors**: Wrap in user-friendly messages: `Error: Failed to open database: <reason>`
- **Port in use**: Clear error: `Error: Port 8080 is already in use`
All errors go to stderr. Exit codes: `0` success, `1` general error, `2` usage error.
## 10. Testing Strategy
Write tests **alongside implementation**, not after. Each module gets tests before moving to the next.
| Test file | What it covers |
|----------------------|----------------------------------------------------------|
| `test_db.py` | Schema creation, CRUD, constraints, `mode` CHECK, `0600` permissions |
| `test_github.py` | API mocking for both releases and tags, parsing, rate limiting, token validation |
| `test_rss.py` | Feed generation, markdown stripping, truncation, XML escaping, item cap, tag entries (no body) |
| `test_server.py` | HTTP server, feed endpoint, health check, response headers |
| `test_integration.py`| Full flow: `add` → `check` → `serve` → fetch feed → verify XML (both modes) |
| `test_cli.py` | CLI smoke tests: exit codes, stderr output, flag parsing, `--tags` flag |
Use `pytest` with `respx` for mocking `httpx`. In-memory SQLite for DB tests. `pytest-httpserver` or `http.server` in a thread for integration tests.
## 11. Implementation Order
1. **Project scaffolding**: `pyproject.toml`, directory structure, `.gitignore`
2. **Config module** (`config.py`): Defaults, env vars, DB path management
3. **Database layer** (`db.py`): Schema, connection, CRUD functions + `test_db.py`
4. **CLI foundation** (`cli.py`): `add`, `remove`, `list` commands + `test_cli.py`
5. **GitHub fetcher** (`github.py`): API calls for releases and tags, parsing, rate limiting, token validation + `test_github.py`
6. **CLI `check` command**: Integration of fetcher with DB upsert
7. **RSS generator** (`rss.py`): xml.etree generation, markdown stripping + `test_rss.py`
8. **HTTP server** (`server.py`): stdlib http.server, `/feed.xml`, `/health` + `test_server.py`
9. **CLI `serve` command**: Wire up server
10. **Integration tests** (`test_integration.py`): End-to-end flow
11. **README**: Usage documentation
## 12. Dependencies
| Package | Purpose |
|--------------|----------------------------------------------|
| `httpx` | HTTP client for GitHub API |
| `markdown` | Convert markdown to HTML |
| `bleach` | Strip HTML tags for plain text |
| `pytest` | Testing framework |
| `respx` | Mocking HTTP requests for tests |
No heavy frameworks. `http.server` and `xml.etree.ElementTree` are stdlib.
## 13. Future Considerations (Out of Scope for MVP)
- Webhook-based real-time updates instead of polling
- Docker container for easy deployment
- Configuration file (`ghrel.toml`) instead of CLI-only management
- Support for GitLab releases
- Email notifications for new releases
- Web UI for managing tracked repos
- Scheduled background checks (cron / systemd timer integration)
+22
View File
@@ -0,0 +1,22 @@
[project]
name = "ghrel"
version = "0.1.0"
description = "Track GitHub releases and tags as an RSS feed"
requires-python = ">=3.12"
dependencies = [
"httpx",
"markdown",
"bleach",
]
[project.scripts]
ghrel = "src.cli:main"
[project.optional-dependencies]
test = [
"pytest",
"respx",
]
[tool.pytest.ini_options]
testpaths = ["tests"]
+3
View File
@@ -0,0 +1,3 @@
"""GitHub Release Monitor - track GitHub releases and tags as an RSS feed."""
__version__ = "0.1.0"
+268
View File
@@ -0,0 +1,268 @@
"""CLI entry point."""
import argparse
import re
import sys
from datetime import datetime, timedelta, timezone
from typing import Literal
from src import __version__
from src.config import (
DEFAULT_SERVER_HOST,
DEFAULT_SERVER_PORT,
get_db_path,
)
from src.db import get_connection
from src.github import GitHubClient, AuthenticationError, NotFoundError
REPO_PATTERN = re.compile(r"^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$")
def error(msg: str, code: int = 1) -> None:
print(f"Error: {msg}", file=sys.stderr)
sys.exit(code)
def parse_since(value: str) -> timedelta:
"""Parse a duration string like '24h', '2h', '1d' into a timedelta."""
m = re.match(r"^(\d+)([hdm])$", value.strip().lower())
if not m:
return timedelta(hours=24) # default
num, unit = int(m.group(1)), m.group(2)
if unit == "h":
return timedelta(hours=num)
elif unit == "d":
return timedelta(days=num)
else: # 'm' for minutes
return timedelta(minutes=num)
def cmd_add(args: argparse.Namespace, db_path: str) -> None:
"""Add a repository to track."""
owner_repo = args.repo
if not REPO_PATTERN.match(owner_repo):
error(f"Invalid repo format '{owner_repo}'. Expected 'owner/repo'")
mode: Literal["release", "tag"] = "tag" if args.tags else "release"
conn = get_connection(db_path)
try:
# Validate repo exists via GitHub API
with GitHubClient() as gh:
if not gh.repo_exists(owner_repo):
error(f"Repository '{owner_repo}' not found on GitHub")
from src.db import add_repo
add_repo(conn, owner_repo, mode)
conn.commit()
mode_label = "tags" if mode == "tag" else "releases"
print(f"Added {owner_repo} ({mode_label})")
finally:
conn.close()
def cmd_remove(args: argparse.Namespace, db_path: str) -> None:
"""Remove a repository."""
owner_repo = args.repo
conn = get_connection(db_path)
try:
from src.db import remove_repo
if remove_repo(conn, owner_repo):
print(f"Removed {owner_repo}")
else:
error(f"Repository '{owner_repo}' not found")
finally:
conn.close()
def cmd_update(args: argparse.Namespace, db_path: str) -> None:
"""Update the tracking mode for a repository."""
owner_repo = args.repo
if args.tags and args.release:
error("Cannot specify both --tags and --release")
if args.tags:
mode: Literal["release", "tag"] = "tag"
elif args.release:
mode = "release"
else:
error("Specify --tags or --release")
conn = get_connection(db_path)
try:
from src.db import update_repo_mode
if update_repo_mode(conn, owner_repo, mode):
mode_label = "tags" if mode == "tag" else "releases"
print(f"Updated {owner_repo} to track {mode_label}")
else:
error(f"Repository '{owner_repo}' not found")
finally:
conn.close()
def cmd_list(args: argparse.Namespace, db_path: str) -> None:
"""List all tracked repositories."""
conn = get_connection(db_path)
try:
from src.db import list_repos
repos = list_repos(conn)
if not repos:
print("No repositories tracked.")
return
# Format table
header = f"{'REPO':<20} {'MODE':<10} {'ENTRIES':<8} {'LAST CHECKED'}"
print(header)
print("-" * len(header))
for r in repos:
last = r["last_checked"] or "never"
if "T" in str(last):
last = str(last)[:16].replace("T", " ")
print(f"{r['owner_repo']:<20} {r['mode']:<10} {r['entries']:<8} {last}")
finally:
conn.close()
def cmd_check(args: argparse.Namespace, db_path: str) -> None:
"""Fetch latest releases/tags for all tracked repos."""
conn = get_connection(db_path)
try:
from src.db import get_repos_with_mode, upsert_entry, get_repo_id
repos = get_repos_with_mode(conn)
if not repos:
print("No repositories tracked. Use 'ghrel add <owner/repo>' first.")
return
since = getattr(args, "since", None)
since_delta = parse_since(since) if since else None
with GitHubClient() as gh:
total_new = 0
for repo_id, owner_repo, mode in repos:
try:
if mode == "tag":
items = gh.fetch_tags(owner_repo)
kind: Literal["release", "tag"] = "tag"
else:
items = gh.fetch_releases(owner_repo)
kind = "release"
for item in items:
if since_delta:
published = datetime.fromisoformat(item["published_at"].replace("Z", "+00:00"))
cutoff = datetime.now(timezone.utc) - since_delta
if published < cutoff:
continue
inserted = upsert_entry(
conn, repo_id, kind,
item["tag_name"], item["title"], item["body"],
item["published_at"], item["html_url"],
)
if inserted:
total_new += 1
except Exception as e:
print(f"Warning: Failed to check {owner_repo}: {e}", file=sys.stderr)
print(f"Checked {len(repos)} repos: {total_new} new entries found")
finally:
conn.close()
def cmd_serve(args: argparse.Namespace, db_path: str) -> None:
"""Start the HTTP server."""
host = args.host or DEFAULT_SERVER_HOST
port = args.port or DEFAULT_SERVER_PORT
from src.server import create_handler
from http.server import HTTPServer
handler = create_handler(db_path)
try:
server = HTTPServer((host, port), handler)
except OSError as e:
if e.errno == 98: # Address already in use
error(f"Port {port} is already in use")
raise
print(f"Serving RSS feed at http://{host}:{port}/feed.xml")
try:
server.serve_forever()
except KeyboardInterrupt:
print("\nShutting down server.")
server.server_close()
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="ghrel",
description="Track GitHub releases and tags as an RSS feed",
)
parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose output")
parser.add_argument("--quiet", "-q", action="store_true", help="Suppress non-essential output")
parser.add_argument("--db-path", default=None, help="Override database path")
subparsers = parser.add_subparsers(dest="command")
# add
p_add = subparsers.add_parser("add", help="Add a repository to track")
p_add.add_argument("repo", help="Repository in 'owner/repo' format")
p_add.add_argument("--tags", action="store_true", help="Track git tags instead of releases")
# remove
p_remove = subparsers.add_parser("remove", help="Remove a repository")
p_remove.add_argument("repo", help="Repository in 'owner/repo' format")
# update
p_update = subparsers.add_parser("update", help="Update tracking mode")
p_update.add_argument("repo", help="Repository in 'owner/repo' format")
p_update.add_argument("--tags", action="store_true", help="Switch to tracking tags")
p_update.add_argument("--release", action="store_true", help="Switch to tracking releases")
# list
subparsers.add_parser("list", help="List tracked repositories")
# check
p_check = subparsers.add_parser("check", help="Fetch latest releases/tags")
p_check.add_argument("--since", nargs="?", const="24h", default=None,
help="Only fetch entries since N hours/days/minutes ago (e.g., 24h, 2d)")
# serve
p_serve = subparsers.add_parser("serve", help="Start HTTP server")
p_serve.add_argument("--port", type=int, default=DEFAULT_SERVER_PORT, help="Server port")
p_serve.add_argument("--host", default=DEFAULT_SERVER_HOST, help="Server host")
return parser
def main() -> None:
parser = build_parser()
args = parser.parse_args()
if not args.command:
parser.print_help()
sys.exit(0)
db_path = args.db_path or str(get_db_path())
commands = {
"add": cmd_add,
"remove": cmd_remove,
"update": cmd_update,
"list": cmd_list,
"check": cmd_check,
"serve": cmd_serve,
}
cmd = commands.get(args.command)
if cmd:
cmd(args, db_path)
else:
parser.print_help()
sys.exit(2)
if __name__ == "__main__":
main()
+29
View File
@@ -0,0 +1,29 @@
"""Centralized configuration."""
import os
from pathlib import Path
def get_db_path(override: str | None = None) -> Path:
"""Return the database file path.
Priority: override argument > env var GHREL_DB_PATH > default.
"""
if override:
return Path(override)
env_path = os.environ.get("GHREL_DB_PATH")
if env_path:
return Path(env_path)
return Path.home() / ".config" / "ghrel" / "repos.db"
def get_gh_token() -> str | None:
"""Return the GitHub token from the environment, or None."""
return os.environ.get("GH_TOKEN")
# Defaults
DEFAULT_SERVER_HOST = "127.0.0.1"
DEFAULT_SERVER_PORT = 8080
RSS_MAX_ITEMS = 50
API_PER_PAGE = 30
+177
View File
@@ -0,0 +1,177 @@
"""SQLite database layer."""
import sqlite3
from datetime import datetime
from pathlib import Path
from typing import Literal
from src.config import get_db_path
def get_connection(db_path: str | Path | None = None) -> sqlite3.Connection:
"""Open a database connection, creating the DB and schema if needed."""
path = Path(db_path) if db_path else get_db_path()
path.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(str(path))
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON")
_create_tables(conn)
# Set restrictive permissions on the database file
try:
path.chmod(0o600)
except OSError:
pass
return conn
def _create_tables(conn: sqlite3.Connection) -> None:
"""Create the database tables if they don't exist."""
conn.executescript("""
CREATE TABLE IF NOT EXISTS repos (
id INTEGER PRIMARY KEY AUTOINCREMENT,
owner_repo TEXT NOT NULL UNIQUE,
mode TEXT NOT NULL DEFAULT 'release' CHECK(mode IN ('release', 'tag')),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS entries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
repo_id INTEGER NOT NULL,
kind TEXT NOT NULL CHECK(kind IN ('release', 'tag')),
tag_name TEXT NOT NULL,
title TEXT NOT NULL,
body TEXT DEFAULT '',
published_at DATETIME NOT NULL,
html_url TEXT NOT NULL,
fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (repo_id) REFERENCES repos(id) ON DELETE CASCADE,
UNIQUE(repo_id, tag_name)
);
""")
conn.commit()
# ── Repos ──
def add_repo(conn: sqlite3.Connection, owner_repo: str, mode: Literal["release", "tag"] = "release") -> int:
"""Add a repository to track. Returns the repo id."""
cursor = conn.execute(
"INSERT INTO repos (owner_repo, mode) VALUES (?, ?)", (owner_repo, mode)
)
conn.commit()
return cursor.lastrowid
def remove_repo(conn: sqlite3.Connection, owner_repo: str) -> bool:
"""Remove a repository. Returns True if a row was deleted."""
cursor = conn.execute("DELETE FROM repos WHERE owner_repo = ?", (owner_repo,))
conn.commit()
return cursor.rowcount > 0
def update_repo_mode(conn: sqlite3.Connection, owner_repo: str, mode: Literal["release", "tag"]) -> bool:
"""Update the tracking mode for a repo. Returns True if a row was updated."""
cursor = conn.execute(
"UPDATE repos SET mode = ? WHERE owner_repo = ?", (mode, owner_repo)
)
conn.commit()
return cursor.rowcount > 0
def list_repos(conn: sqlite3.Connection) -> list[dict]:
"""List all tracked repositories with entry counts and last checked time."""
rows = conn.execute("""
SELECT
r.owner_repo,
r.mode,
COUNT(e.id) AS entries,
MAX(e.fetched_at) AS last_checked
FROM repos r
LEFT JOIN entries e ON e.repo_id = r.id
GROUP BY r.id
ORDER BY r.owner_repo
""").fetchall()
return [
{
"owner_repo": row[0],
"mode": row[1],
"entries": row[2],
"last_checked": row[3],
}
for row in rows
]
def get_repo_id(conn: sqlite3.Connection, owner_repo: str) -> int | None:
"""Get the internal id for a repo, or None if not found."""
row = conn.execute("SELECT id FROM repos WHERE owner_repo = ?", (owner_repo,)).fetchone()
return row[0] if row else None
def get_repos_with_mode(conn: sqlite3.Connection) -> list[tuple[int, str, str]]:
"""Return list of (id, owner_repo, mode) for all repos."""
return conn.execute("SELECT id, owner_repo, mode FROM repos").fetchall()
# ── Entries ──
def upsert_entry(
conn: sqlite3.Connection,
repo_id: int,
kind: Literal["release", "tag"],
tag_name: str,
title: str,
body: str,
published_at: str,
html_url: str,
) -> bool:
"""Insert an entry, skip if already exists (by repo_id + tag_name).
Returns True if a new entry was inserted.
"""
try:
conn.execute(
"""
INSERT INTO entries (repo_id, kind, tag_name, title, body, published_at, html_url)
VALUES (?, ?, ?, ?, ?, ?, ?)
""",
(repo_id, kind, tag_name, title, body, published_at, html_url),
)
conn.commit()
return True
except sqlite3.IntegrityError:
return False
def get_all_entries(conn: sqlite3.Connection, limit: int = 50) -> list[dict]:
"""Get all entries sorted by published_at descending, capped at *limit*."""
rows = conn.execute("""
SELECT e.tag_name, e.title, e.body, e.published_at, e.html_url,
e.kind, r.owner_repo
FROM entries e
JOIN repos r ON r.id = e.repo_id
ORDER BY e.published_at DESC
LIMIT ?
""", (limit,)).fetchall()
return [
{
"tag_name": row[0],
"title": row[1],
"body": row[2],
"published_at": row[3],
"html_url": row[4],
"kind": row[5],
"owner_repo": row[6],
}
for row in rows
]
def get_entry_count(conn: sqlite3.Connection, repo_id: int) -> int:
"""Get the number of entries for a repo."""
row = conn.execute(
"SELECT COUNT(*) FROM entries WHERE repo_id = ?", (repo_id,)
).fetchone()
return row[0]
+172
View File
@@ -0,0 +1,172 @@
"""GitHub API interaction."""
import time
from datetime import datetime, timezone
from typing import Literal
import httpx
from src.config import API_PER_PAGE, get_gh_token
class GitHubClient:
"""Client for the GitHub REST API with rate-limit awareness."""
BASE_URL = "https://api.github.com"
def __init__(self, token: str | None = None):
self._client = httpx.Client()
headers = {}
self._token = token or get_gh_token()
if self._token:
headers["Authorization"] = f"token {self._token}"
self._client.headers.update(headers)
self._rate_limit_remaining = None
self._rate_limit_reset = None
def close(self):
self._client.close()
def __enter__(self):
return self
def __exit__(self, *args):
self.close()
def _check_rate_limit(self, response: httpx.Response) -> None:
"""Update rate limit state from response headers."""
remaining = response.headers.get("X-RateLimit-Remaining")
reset = response.headers.get("X-RateLimit-Reset")
if remaining is not None:
self._rate_limit_remaining = int(remaining)
if reset is not None:
self._rate_limit_reset = int(reset)
def _handle_rate_limit(self, response: httpx.Response) -> None:
"""Wait if rate limited."""
if response.status_code == 403 and self._rate_limit_remaining is not None and self._rate_limit_remaining == 0:
if self._rate_limit_reset:
wait = max(1, self._rate_limit_reset - int(time.time()))
import sys
print(f"Warning: Rate limited. Waiting {wait}s...", file=sys.stderr)
time.sleep(wait)
def _request(self, method: str, url: str, **kwargs) -> httpx.Response:
"""Make an HTTP request with retry and rate-limit handling."""
last_error = None
for attempt in range(3):
try:
response = self._client.request(method, url, **kwargs)
self._check_rate_limit(response)
if response.status_code == 401:
raise AuthenticationError("GH_TOKEN is invalid (401 Unauthorized)")
if response.status_code == 404:
raise NotFoundError(f"Resource not found: {url}")
self._handle_rate_limit(response)
response.raise_for_status()
return response
except (NotFoundError, AuthenticationError):
raise
except httpx.HTTPStatusError:
raise
except httpx.TransportError as e:
last_error = e
if attempt < 2:
time.sleep(2 ** attempt)
raise last_error # type: ignore[misc]
def validate_token(self) -> None:
"""Check that the token is valid by hitting the rate_limit endpoint."""
if not self._token:
return # No token, allow unauthenticated
resp = self._request("GET", f"{self.BASE_URL}/rate_limit")
def fetch_releases(self, owner_repo: str) -> list[dict]:
"""Fetch the latest releases for a repository.
Returns a list of dicts with keys: tag_name, title, body, published_at, html_url.
"""
resp = self._request(
"GET",
f"{self.BASE_URL}/repos/{owner_repo}/releases",
params={"per_page": API_PER_PAGE},
)
data = resp.json()
result = []
for release in data:
result.append({
"tag_name": release["tag_name"],
"title": release.get("name") or release["tag_name"],
"body": release.get("body") or "",
"published_at": release.get("published_at") or datetime.now(timezone.utc).isoformat(),
"html_url": release.get("html_url") or f"https://github.com/{owner_repo}/releases/tag/{release['tag_name']}",
})
return result
def fetch_tags(self, owner_repo: str) -> list[dict]:
"""Fetch the latest tags for a repository.
Returns a list of dicts with keys: tag_name, title, body, published_at, html_url.
"""
resp = self._request(
"GET",
f"{self.BASE_URL}/repos/{owner_repo}/tags",
params={"per_page": API_PER_PAGE},
)
data = resp.json()
result = []
for tag in data:
tag_name = tag["name"]
published_at = self._get_tag_date(owner_repo, tag_name)
result.append({
"tag_name": tag_name,
"title": tag_name,
"body": "",
"published_at": published_at,
"html_url": f"https://github.com/{owner_repo}/tags/{tag_name}",
})
return result
def _get_tag_date(self, owner_repo: str, tag_name: str) -> str:
"""Get the commit date for a tag by resolving the git ref."""
try:
resp = self._request(
"GET",
f"{self.BASE_URL}/repos/{owner_repo}/git/ref/tags/{tag_name}",
)
data = resp.json()
# The ref may point to a commit directly or to a tag object
obj = data.get("object", {})
obj_type = obj.get("type")
if obj_type == "commit":
author_date = obj.get("author", {}).get("date")
if author_date:
return author_date
elif obj_type == "tag":
# Follow the tag object to get the commit
commit_sha = obj.get("sha")
if commit_sha:
commit_resp = self._request("GET", f"{self.BASE_URL}/repos/{owner_repo}/git/commits/{commit_sha}")
commit_data = commit_resp.json()
author_date = commit_data.get("author", {}).get("date")
if author_date:
return author_date
except Exception:
pass
return datetime.now(timezone.utc).isoformat()
def repo_exists(self, owner_repo: str) -> bool:
"""Check if a repository exists on GitHub."""
try:
resp = self._request("GET", f"{self.BASE_URL}/repos/{owner_repo}")
return resp.status_code == 200
except (NotFoundError, httpx.HTTPStatusError):
return False
class AuthenticationError(Exception):
pass
class NotFoundError(Exception):
pass
+79
View File
@@ -0,0 +1,79 @@
"""RSS feed generation."""
import re
import xml.etree.ElementTree as ET
from datetime import datetime, timezone
import bleach
import markdown
from src.config import RSS_MAX_ITEMS
def strip_markdown(text: str) -> str:
"""Convert markdown to HTML, then strip tags to produce plain text."""
if not text:
return ""
html = markdown.markdown(text)
plain = bleach.clean(html, tags=[], strip=True)
return plain.strip()
def format_rfc822(dt_str: str) -> str:
"""Convert an ISO 8601 datetime string to RFC 822 format for RSS."""
dt = datetime.fromisoformat(dt_str.replace("Z", "+00:00"))
return dt.strftime("%a, %d %b %Y %H:%M:%S %z")
def generate_feed(entries: list[dict], base_url: str = "http://127.0.0.1:8080") -> str:
"""Generate an RSS 2.0 XML feed from a list of entry dicts.
Each entry should have: tag_name, title, body, published_at, html_url, owner_repo.
"""
root = ET.Element("rss")
root.set("version", "2.0")
channel = ET.SubElement(root, "channel")
now = datetime.now(timezone.utc)
ET.SubElement(channel, "title").text = "GitHub Release Monitor"
ET.SubElement(channel, "description").text = "Latest releases from tracked GitHub repositories"
ET.SubElement(channel, "link").text = base_url.rstrip("/") + "/"
ET.SubElement(channel, "lastBuildDate").text = now.strftime("%a, %d %b %Y %H:%M:%S %z")
for entry in entries[:RSS_MAX_ITEMS]:
item = ET.SubElement(channel, "item")
owner_repo = entry.get("owner_repo", "")
ET.SubElement(item, "title").text = f"[{owner_repo}] {entry['title']}"
body = entry.get("body", "")
if body:
plain = strip_markdown(body)
if len(plain) > 300:
plain = plain[:300] + "\u2026"
ET.SubElement(item, "description").text = plain
else:
ET.SubElement(item, "description").text = f"Tag {entry['tag_name']}"
ET.SubElement(item, "link").text = entry["html_url"]
ET.SubElement(item, "pubDate").text = format_rfc822(entry["published_at"])
ET.SubElement(item, "guid").text = (
f"github.com/{owner_repo}/releases/tag/{entry['tag_name']}"
)
return ET.tostring(root, encoding="unicode", xml_declaration=False)
def generate_index_html(base_url: str = "http://127.0.0.1:8080") -> str:
"""Generate a simple index page with a link to the RSS feed."""
return f"""<!DOCTYPE html>
<html>
<head><title>GitHub Release Monitor</title></head>
<body>
<h1>GitHub Release Monitor</h1>
<p><a href="{base_url.rstrip('/')}/feed.xml">RSS Feed</a></p>
<p><a href="{base_url.rstrip('/')}/health">Health Check</a></p>
</body>
</html>
"""
+66
View File
@@ -0,0 +1,66 @@
"""HTTP server for serving the RSS feed."""
from http.server import BaseHTTPRequestHandler
from urllib.parse import urlparse
from src.db import get_connection
from src.rss import generate_feed, generate_index_html
def create_handler(db_path: str):
"""Factory that returns an HTTP request handler class bound to a DB path."""
class FeedHandler(BaseHTTPRequestHandler):
def do_GET(self):
parsed = urlparse(self.path)
path = parsed.path.rstrip("/") or "/"
if path == "/feed.xml":
self._serve_feed()
elif path == "/health":
self._serve_health()
elif path == "/":
self._serve_index()
else:
self.send_response(404)
self.send_header("Content-Type", "text/plain")
self.end_headers()
self.wfile.write(b"Not Found")
def _serve_feed(self):
conn = get_connection(db_path)
try:
from src.db import get_all_entries
entries = get_all_entries(conn)
feed_xml = generate_feed(entries, self._base_url())
finally:
conn.close()
self.send_response(200)
self.send_header("Content-Type", "application/rss+xml; charset=utf-8")
self.end_headers()
self.wfile.write(feed_xml.encode("utf-8"))
def _serve_index(self):
html = generate_index_html(self._base_url())
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.end_headers()
self.wfile.write(html.encode("utf-8"))
def _serve_health(self):
self.send_response(200)
self.send_header("Content-Type", "text/plain")
self.end_headers()
self.wfile.write(b"OK")
def _base_url(self) -> str:
host = self.server.server_address[0]
port = self.server.server_address[1]
return f"http://{host}:{port}"
def log_message(self, format, *args):
"""Suppress default request logging."""
pass
return FeedHandler
View File
+86
View File
@@ -0,0 +1,86 @@
"""Tests for CLI commands."""
import sqlite3
import subprocess
import sys
import tempfile
import pytest
from src.db import _create_tables, add_repo
class TestRepoFormatValidation:
def test_valid_format(self):
from src.cli import REPO_PATTERN
assert REPO_PATTERN.match("owner/repo")
assert REPO_PATTERN.match("my-org/my_repo")
assert REPO_PATTERN.match("a.b/c-d_e")
def test_invalid_format(self):
from src.cli import REPO_PATTERN
assert not REPO_PATTERN.match("owner")
assert not REPO_PATTERN.match("repo")
assert not REPO_PATTERN.match("owner/repo/extra")
class TestParseSince:
def test_hours(self):
from src.cli import parse_since
from datetime import timedelta
assert parse_since("24h") == timedelta(hours=24)
def test_days(self):
from src.cli import parse_since
from datetime import timedelta
assert parse_since("2d") == timedelta(days=2)
def test_minutes(self):
from src.cli import parse_since
from datetime import timedelta
assert parse_since("30m") == timedelta(minutes=30)
def test_invalid_defaults(self):
from src.cli import parse_since
from datetime import timedelta
assert parse_since("invalid") == timedelta(hours=24)
class TestCliHelp:
def test_help_flag(self):
result = subprocess.run(
[sys.executable, "-m", "src.cli", "--help"],
capture_output=True, text=True
)
assert result.returncode == 0
assert "ghrel" in result.stdout.lower() or "track" in result.stdout.lower()
def test_version_flag(self):
result = subprocess.run(
[sys.executable, "-m", "src.cli", "--version"],
capture_output=True, text=True
)
assert result.returncode == 0
assert "0.1.0" in result.stdout
class TestCliList:
def test_list_empty(self, tmp_path):
db_path = str(tmp_path / "test.db")
result = subprocess.run(
[sys.executable, "-m", "src.cli", "--db-path", db_path, "list"],
capture_output=True, text=True
)
assert result.returncode == 0
assert "No repositories" in result.stdout
class TestCliAddRemove:
def test_add_invalid_format(self, tmp_path):
db_path = str(tmp_path / "test.db")
result = subprocess.run(
[sys.executable, "-m", "src.cli", "--db-path", db_path, "add", "invalid"],
capture_output=True, text=True
)
assert result.returncode != 0
assert "Invalid repo format" in result.stderr
+143
View File
@@ -0,0 +1,143 @@
"""Tests for the database layer."""
import os
import stat
import sqlite3
import tempfile
import pytest
from src import db
@pytest.fixture
def conn():
"""Provide an in-memory database connection for testing."""
c = sqlite3.connect(":memory:")
c.execute("PRAGMA foreign_keys=ON")
db._create_tables(c)
return c
class TestCreateTables:
def test_tables_exist(self, conn):
tables = set(r[0] for r in conn.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall())
assert "repos" in tables
assert "entries" in tables
def test_mode_check_constraint(self, conn):
with pytest.raises(sqlite3.IntegrityError):
conn.execute("INSERT INTO repos (owner_repo, mode) VALUES ('a/b', 'invalid')")
conn.commit()
def test_kind_check_constraint(self, conn):
conn.execute("INSERT INTO repos (owner_repo) VALUES ('a/b')")
conn.commit()
with pytest.raises(sqlite3.IntegrityError):
conn.execute(
"INSERT INTO entries (repo_id, kind, tag_name, title, published_at, html_url) "
"VALUES (1, 'invalid', 'v1', 'v1', '2025-01-01', 'http://x')"
)
conn.commit()
class TestAddRepo:
def test_add_repo_default_mode(self, conn):
repo_id = db.add_repo(conn, "owner/repo")
assert repo_id == 1
row = conn.execute("SELECT owner_repo, mode FROM repos WHERE id = ?", (repo_id,)).fetchone()
assert row == ("owner/repo", "release")
def test_add_repo_tag_mode(self, conn):
repo_id = db.add_repo(conn, "owner/repo", "tag")
row = conn.execute("SELECT mode FROM repos WHERE id = ?", (repo_id,)).fetchone()
assert row[0] == "tag"
def test_add_repo_duplicate_fails(self, conn):
db.add_repo(conn, "owner/repo")
with pytest.raises(sqlite3.IntegrityError):
db.add_repo(conn, "owner/repo")
class TestRemoveRepo:
def test_remove_existing(self, conn):
db.add_repo(conn, "owner/repo")
assert db.remove_repo(conn, "owner/repo") is True
assert db.list_repos(conn) == []
def test_remove_nonexistent(self, conn):
assert db.remove_repo(conn, "no/one") is False
def test_remove_cascades_to_entries(self, conn):
repo_id = db.add_repo(conn, "owner/repo")
db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x")
db.remove_repo(conn, "owner/repo")
assert db.get_entry_count(conn, repo_id) == 0
class TestUpdateRepoMode:
def test_update_mode(self, conn):
db.add_repo(conn, "owner/repo")
assert db.update_repo_mode(conn, "owner/repo", "tag") is True
row = conn.execute("SELECT mode FROM repos WHERE owner_repo = 'owner/repo'").fetchone()
assert row[0] == "tag"
def test_update_nonexistent(self, conn):
assert db.update_repo_mode(conn, "no/one", "tag") is False
class TestListRepos:
def test_empty_list(self, conn):
assert db.list_repos(conn) == []
def test_list_with_entries(self, conn):
repo_id = db.add_repo(conn, "a/b")
db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x")
result = db.list_repos(conn)
assert len(result) == 1
assert result[0]["owner_repo"] == "a/b"
assert result[0]["entries"] == 1
assert result[0]["mode"] == "release"
class TestUpsertEntry:
def test_insert_new(self, conn):
repo_id = db.add_repo(conn, "owner/repo")
assert db.upsert_entry(conn, repo_id, "release", "v1", "Release v1", "body text",
"2025-01-01T00:00:00Z", "http://example.com") is True
def test_skip_duplicate(self, conn):
repo_id = db.add_repo(conn, "owner/repo")
assert db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x") is True
assert db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x") is False
class TestGetAllEntries:
def test_returns_entries_sorted_desc(self, conn):
repo_id = db.add_repo(conn, "owner/repo")
db.upsert_entry(conn, repo_id, "release", "v1", "v1", "", "2025-01-01", "http://x")
db.upsert_entry(conn, repo_id, "release", "v2", "v2", "", "2025-01-02", "http://y")
entries = db.get_all_entries(conn)
assert entries[0]["tag_name"] == "v2"
assert entries[1]["tag_name"] == "v1"
def test_respects_limit(self, conn):
repo_id = db.add_repo(conn, "owner/repo")
for i in range(5):
db.upsert_entry(conn, repo_id, "release", f"v{i}", f"v{i}", "", f"2025-01-0{i}", "http://x")
entries = db.get_all_entries(conn, limit=3)
assert len(entries) == 3
class TestGetConnection:
def test_creates_file(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "test.db")
c = db.get_connection(path)
c.close()
assert os.path.exists(path)
# Check permissions
mode = os.stat(path).st_mode
assert stat.S_IMODE(mode) == 0o600
+172
View File
@@ -0,0 +1,172 @@
"""Tests for the GitHub API client."""
import httpx
import pytest
import respx
from src.github import GitHubClient, AuthenticationError, NotFoundError
@pytest.fixture
def mock_api():
"""Enable respx mocking for test scope."""
with respx.mock:
yield respx
class TestFetchReleases:
def test_fetch_releases(self, mock_api):
mock_api.get("https://api.github.com/repos/owner/repo/releases").mock(
return_value=httpx.Response(200, json=[
{
"tag_name": "v1.0.0",
"name": "Release v1.0.0",
"body": "# Changelog\n\n- Fixed bug",
"published_at": "2025-01-15T10:30:00Z",
"html_url": "https://github.com/owner/repo/releases/tag/v1.0.0",
},
{
"tag_name": "v0.9.0",
"name": None,
"body": "",
"published_at": "2025-01-10T08:00:00Z",
"html_url": "https://github.com/owner/repo/releases/tag/v0.9.0",
},
])
)
client = GitHubClient(token="fake_token")
releases = client.fetch_releases("owner/repo")
client.close()
assert len(releases) == 2
assert releases[0]["tag_name"] == "v1.0.0"
assert releases[0]["title"] == "Release v1.0.0"
assert releases[0]["body"] == "# Changelog\n\n- Fixed bug"
assert releases[1]["title"] == "v0.9.0"
assert releases[1]["body"] == ""
def test_fetch_releases_404(self, mock_api):
mock_api.get("https://api.github.com/repos/no/such/releases").mock(
return_value=httpx.Response(404)
)
client = GitHubClient(token="fake_token")
with pytest.raises(NotFoundError):
client.fetch_releases("no/such")
client.close()
class TestFetchTags:
def test_fetch_tags(self, mock_api):
mock_api.get("https://api.github.com/repos/owner/repo/tags").mock(
return_value=httpx.Response(200, json=[
{"name": "v1.0.0", "zipball_url": "https://api.github.com/repos/owner/repo/zipball/v1.0.0"},
{"name": "v0.9.0", "zipball_url": "https://api.github.com/repos/owner/repo/zipball/v0.9.0"},
])
)
mock_api.get("https://api.github.com/repos/owner/repo/git/ref/tags/v1.0.0").mock(
return_value=httpx.Response(200, json={
"object": {
"type": "commit",
"author": {"date": "2025-01-15T10:30:00Z"},
}
})
)
mock_api.get("https://api.github.com/repos/owner/repo/git/ref/tags/v0.9.0").mock(
return_value=httpx.Response(200, json={
"object": {
"type": "commit",
"author": {"date": "2025-01-10T08:00:00Z"},
}
})
)
client = GitHubClient(token="fake_token")
tags = client.fetch_tags("owner/repo")
client.close()
assert len(tags) == 2
assert tags[0]["tag_name"] == "v1.0.0"
assert tags[0]["title"] == "v1.0.0"
assert tags[0]["body"] == ""
assert tags[0]["published_at"] == "2025-01-15T10:30:00Z"
assert tags[0]["html_url"] == "https://github.com/owner/repo/tags/v1.0.0"
def test_fetch_tags_with_tag_object(self, mock_api):
"""When a tag points to a tag object, follow to the commit."""
mock_api.get("https://api.github.com/repos/owner/repo/tags").mock(
return_value=httpx.Response(200, json=[
{"name": "v2.0.0", "zipball_url": "https://api.github.com/..."},
])
)
mock_api.get("https://api.github.com/repos/owner/repo/git/ref/tags/v2.0.0").mock(
return_value=httpx.Response(200, json={
"object": {
"type": "tag",
"sha": "abc123",
}
})
)
mock_api.get("https://api.github.com/repos/owner/repo/git/commits/abc123").mock(
return_value=httpx.Response(200, json={
"author": {"date": "2025-02-01T12:00:00Z"}
})
)
client = GitHubClient(token="fake_token")
tags = client.fetch_tags("owner/repo")
client.close()
assert tags[0]["published_at"] == "2025-02-01T12:00:00Z"
def test_fetch_tags_fallback_date(self, mock_api):
"""When git ref fails, use current time as fallback."""
mock_api.get("https://api.github.com/repos/owner/repo/tags").mock(
return_value=httpx.Response(200, json=[
{"name": "v1.0.0", "zipball_url": "https://api.github.com/..."},
])
)
mock_api.get("https://api.github.com/repos/owner/repo/git/ref/tags/v1.0.0").mock(
return_value=httpx.Response(404)
)
client = GitHubClient(token="fake_token")
tags = client.fetch_tags("owner/repo")
client.close()
assert tags[0]["published_at"] is not None
class TestRepoExists:
def test_repo_exists_true(self, mock_api):
mock_api.get("https://api.github.com/repos/owner/repo").mock(
return_value=httpx.Response(200, json={"full_name": "owner/repo"})
)
client = GitHubClient(token="fake_token")
assert client.repo_exists("owner/repo") is True
client.close()
def test_repo_exists_false(self, mock_api):
mock_api.get("https://api.github.com/repos/no/such").mock(
return_value=httpx.Response(404)
)
client = GitHubClient(token="fake_token")
assert client.repo_exists("no/such") is False
client.close()
class TestAuthentication:
def test_invalid_token(self, mock_api):
mock_api.get("https://api.github.com/rate_limit").mock(
return_value=httpx.Response(401)
)
client = GitHubClient(token="bad_token")
with pytest.raises(AuthenticationError, match="401 Unauthorized"):
client.validate_token()
client.close()
def test_no_token_skips_validation(self):
client = GitHubClient(token=None)
client.validate_token()
client.close()
+205
View File
@@ -0,0 +1,205 @@
"""End-to-end integration tests."""
import sqlite3
import threading
import tempfile
import urllib.request
from pathlib import Path
import httpx
import pytest
import respx
from src.db import _create_tables, add_repo, upsert_entry, get_connection
from src.server import create_handler
from http.server import HTTPServer
@pytest.fixture
def mock_github():
"""Mock only GitHub API calls."""
with respx.mock:
yield respx
def fetch_feed(port: int) -> str:
"""Fetch feed from localhost using urllib (bypasses respx)."""
url = f"http://127.0.0.1:{port}/feed.xml"
with urllib.request.urlopen(url) as resp:
return resp.read().decode("utf-8")
class TestFullFlowWithMockedAPI:
"""Test the full flow: add repo, check (with mocked API), serve, fetch feed."""
def test_release_mode_full_flow(self, mock_github):
mock_github.get("https://api.github.com/repos/signoz/signoz/releases").mock(
return_value=httpx.Response(200, json=[
{
"tag_name": "v1.0.0",
"name": "Signoz v1.0.0",
"body": "# Release v1.0.0\n\n- New dashboard features",
"published_at": "2025-01-15T10:30:00Z",
"html_url": "https://github.com/signoz/signoz/releases/tag/v1.0.0",
}
])
)
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
conn = get_connection(db_path)
add_repo(conn, "signoz/signoz", "release")
conn.close()
from src.github import GitHubClient
from src.db import get_repos_with_mode, upsert_entry as db_upsert
conn = get_connection(db_path)
repos = get_repos_with_mode(conn)
assert len(repos) == 1
with GitHubClient(token="fake_token") as gh:
for repo_id, owner_repo, mode in repos:
items = gh.fetch_releases(owner_repo)
for item in items:
db_upsert(conn, repo_id, mode,
item["tag_name"], item["title"], item["body"],
item["published_at"], item["html_url"])
conn.close()
handler = create_handler(str(db_path))
server = HTTPServer(("127.0.0.1", 0), handler)
port = server.server_address[1]
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
text = fetch_feed(port)
assert "signoz/signoz" in text
assert "v1.0.0" in text
assert "Signoz v1.0.0" in text
finally:
server.shutdown()
def test_tag_mode_full_flow(self, mock_github):
mock_github.get("https://api.github.com/repos/zammad/zammad/tags").mock(
return_value=httpx.Response(200, json=[
{"name": "6.3.0", "zipball_url": "https://api.github.com/..."},
])
)
mock_github.get("https://api.github.com/repos/zammad/zammad/git/ref/tags/6.3.0").mock(
return_value=httpx.Response(200, json={
"object": {
"type": "commit",
"author": {"date": "2025-02-01T12:00:00Z"},
}
})
)
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
conn = get_connection(db_path)
add_repo(conn, "zammad/zammad", "tag")
conn.close()
from src.github import GitHubClient
from src.db import get_repos_with_mode, upsert_entry as db_upsert
conn = get_connection(db_path)
repos = get_repos_with_mode(conn)
with GitHubClient(token="fake_token") as gh:
for repo_id, owner_repo, mode in repos:
items = gh.fetch_tags(owner_repo)
for item in items:
db_upsert(conn, repo_id, "tag",
item["tag_name"], item["title"], item["body"],
item["published_at"], item["html_url"])
conn.close()
handler = create_handler(str(db_path))
server = HTTPServer(("127.0.0.1", 0), handler)
port = server.server_address[1]
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
text = fetch_feed(port)
assert "zammad/zammad" in text
assert "6.3.0" in text
assert "Tag 6.3.0" in text
finally:
server.shutdown()
class TestMultipleRepos:
def test_mixed_repos_in_feed(self, mock_github):
"""Feed should contain entries from both release and tag repos, sorted by date."""
mock_github.get("https://api.github.com/repos/a/b/releases").mock(
return_value=httpx.Response(200, json=[
{
"tag_name": "v1.0.0",
"name": "v1.0.0",
"body": "Release body",
"published_at": "2025-01-10T00:00:00Z",
"html_url": "https://github.com/a/b/releases/tag/v1.0.0",
}
])
)
mock_github.get("https://api.github.com/repos/c/d/tags").mock(
return_value=httpx.Response(200, json=[
{"name": "v2.0.0", "zipball_url": "https://api.github.com/..."},
])
)
mock_github.get("https://api.github.com/repos/c/d/git/ref/tags/v2.0.0").mock(
return_value=httpx.Response(200, json={
"object": {
"type": "commit",
"author": {"date": "2025-01-20T00:00:00Z"},
}
})
)
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
conn = get_connection(db_path)
add_repo(conn, "a/b", "release")
add_repo(conn, "c/d", "tag")
conn.close()
from src.github import GitHubClient
from src.db import get_repos_with_mode, upsert_entry as db_upsert
conn = get_connection(db_path)
repos = get_repos_with_mode(conn)
with GitHubClient(token="fake_token") as gh:
for repo_id, owner_repo, mode in repos:
if mode == "tag":
items = gh.fetch_tags(owner_repo)
kind = "tag"
else:
items = gh.fetch_releases(owner_repo)
kind = "release"
for item in items:
db_upsert(conn, repo_id, kind,
item["tag_name"], item["title"], item["body"],
item["published_at"], item["html_url"])
conn.close()
handler = create_handler(str(db_path))
server = HTTPServer(("127.0.0.1", 0), handler)
port = server.server_address[1]
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
text = fetch_feed(port)
c_d_pos = text.find("c/d")
a_b_pos = text.find("a/b")
assert c_d_pos < a_b_pos
finally:
server.shutdown()
+134
View File
@@ -0,0 +1,134 @@
"""Tests for RSS feed generation."""
import xml.etree.ElementTree as ET
from src.rss import generate_feed, strip_markdown, format_rfc822
class TestStripMarkdown:
def test_plain_text(self):
assert strip_markdown("Hello world") == "Hello world"
def test_bold_and_lists(self):
result = strip_markdown("**bold** and - item1\n- item2")
assert "bold" in result
assert "item1" in result
def test_empty_string(self):
assert strip_markdown("") == ""
def test_none(self):
assert strip_markdown("") == ""
def test_code_blocks(self):
result = strip_markdown("```python\nprint('hello')\n```")
assert "print" in result or "hello" in result
class TestFormatRfc822:
def test_iso_to_rfc822(self):
result = format_rfc822("2025-01-15T10:30:00Z")
assert "Wed" in result or "15" in result
assert "2025" in result
def test_with_tz_offset(self):
result = format_rfc822("2025-01-15T10:30:00+00:00")
assert "2025" in result
class TestGenerateFeed:
def _entries(self, count=1):
return [
{
"tag_name": f"v1.{i}.0",
"title": f"Release v1.{i}.0",
"body": f"# Release notes\n\n- Feature {i}",
"published_at": "2025-01-15T10:30:00Z",
"html_url": f"https://github.com/owner/repo/releases/tag/v1.{i}.0",
"owner_repo": "owner/repo",
}
for i in range(count)
]
def test_basic_feed(self):
entries = self._entries(2)
xml_str = generate_feed(entries)
root = ET.fromstring(xml_str)
assert root.tag == "rss"
assert root.get("version") == "2.0"
channel = root[0]
assert channel.tag == "channel"
assert channel.find("title").text == "GitHub Release Monitor"
assert channel.find("description").text == "Latest releases from tracked GitHub repositories"
items = channel.findall("item")
assert len(items) == 2
def test_item_structure(self):
entries = self._entries(1)
xml_str = generate_feed(entries)
root = ET.fromstring(xml_str)
channel = root[0]
item = channel.find("item")
assert item.find("title").text == "[owner/repo] Release v1.0.0"
assert "Feature 0" in item.find("description").text
assert item.find("link").text == "https://github.com/owner/repo/releases/tag/v1.0.0"
assert item.find("pubDate").text is not None
assert item.find("guid").text == "github.com/owner/repo/releases/tag/v1.0.0"
def test_description_truncation(self):
entry = {
"tag_name": "v1.0.0",
"title": "Release",
"body": "A" * 400,
"published_at": "2025-01-15T10:30:00Z",
"html_url": "https://example.com",
"owner_repo": "owner/repo",
}
xml_str = generate_feed([entry])
root = ET.fromstring(xml_str)
desc = root[0].find("item").find("description").text
assert len(desc) == 301 # 300 chars + 1 ellipsis char
def test_tag_entry_no_body(self):
entry = {
"tag_name": "v1.0.0",
"title": "v1.0.0",
"body": "",
"published_at": "2025-01-15T10:30:00Z",
"html_url": "https://github.com/owner/repo/tags/v1.0.0",
"owner_repo": "owner/repo",
}
xml_str = generate_feed([entry])
root = ET.fromstring(xml_str)
desc = root[0].find("item").find("description").text
assert desc == "Tag v1.0.0"
def test_item_cap(self):
entries = self._entries(100)
xml_str = generate_feed(entries)
root = ET.fromstring(xml_str)
items = root[0].findall("item")
assert len(items) == 50 # RSS_MAX_ITEMS
def test_xml_escaping(self):
entry = {
"tag_name": "v1.0.0",
"title": "Release with <special> & 'chars'",
"body": "Some **bold** text",
"published_at": "2025-01-15T10:30:00Z",
"html_url": "https://example.com?a=1&b=2",
"owner_repo": "owner/repo",
}
xml_str = generate_feed([entry])
# Should parse without error
root = ET.fromstring(xml_str)
assert root is not None
def test_base_url(self):
entries = self._entries(1)
xml_str = generate_feed(entries, base_url="http://example.com:9000")
root = ET.fromstring(xml_str)
assert root[0].find("link").text == "http://example.com:9000/"
+90
View File
@@ -0,0 +1,90 @@
"""Tests for the HTTP server."""
import sqlite3
import threading
import tempfile
import httpx
import pytest
from src.db import _create_tables, add_repo, upsert_entry
from src.server import create_handler
from http.server import HTTPServer
@pytest.fixture
def db_path(tmp_path):
"""Create a temporary database with sample data."""
path = str(tmp_path / "test.db")
conn = __import__("sqlite3").connect(path)
_create_tables(conn)
repo_id = add_repo(conn, "owner/repo")
upsert_entry(conn, repo_id, "release", "v1.0.0", "Release v1.0.0",
"# Changelog\n\n- New feature",
"2025-01-15T10:30:00Z",
"https://github.com/owner/repo/releases/tag/v1.0.0")
conn.close()
return path
@pytest.fixture
def server(db_path):
"""Start a test HTTP server in a background thread."""
handler = create_handler(db_path)
server = HTTPServer(("127.0.0.1", 0), handler)
port = server.server_address[1]
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
yield f"http://127.0.0.1:{port}"
server.shutdown()
class TestFeedEndpoint:
def test_feed_returns_200(self, server):
resp = httpx.get(f"{server}/feed.xml")
assert resp.status_code == 200
def test_feed_content_type(self, server):
resp = httpx.get(f"{server}/feed.xml")
assert "application/rss+xml" in resp.headers["content-type"]
def test_feed_contains_entry(self, server):
resp = httpx.get(f"{server}/feed.xml")
assert "owner/repo" in resp.text
assert "v1.0.0" in resp.text
def test_feed_is_valid_xml(self, server):
import xml.etree.ElementTree as ET
resp = httpx.get(f"{server}/feed.xml")
root = ET.fromstring(resp.text)
assert root.tag == "rss"
class TestIndexEndpoint:
def test_index_returns_200(self, server):
resp = httpx.get(server + "/")
assert resp.status_code == 200
def test_index_content_type(self, server):
resp = httpx.get(server + "/")
assert "text/html" in resp.headers["content-type"]
def test_index_links_to_feed(self, server):
resp = httpx.get(server + "/")
assert "/feed.xml" in resp.text
class TestHealthEndpoint:
def test_health_returns_200(self, server):
resp = httpx.get(f"{server}/health")
assert resp.status_code == 200
def test_health_body(self, server):
resp = httpx.get(f"{server}/health")
assert resp.text == "OK"
class TestNotFound:
def test_unknown_path(self, server):
resp = httpx.get(f"{server}/unknown")
assert resp.status_code == 404