Public Access

Files

T

Ganonmaster 9c5212de05 feat: add emote import from legacy source with dry-run mode

Add POST /manage/import (auth-protected) that fetches emotes from a
legacy JSON endpoint, downloads each image, uploads it to S3, and
inserts it into the DB with the original timestamps preserved.

- Skip emotes whose name already exists (best-effort duplicate detection
  across SQLite and PostgreSQL via error code + message fallback)
- Validate source_url against a configurable host allowlist
  ([import] allowed_hosts in config, default ["smutba.se"])
- dry_run: true previews the import without writing to S3 or DB;
  result statuses are "would_import" / "would_skip" instead of
  "imported" / "skipped"
- Add db.name_exists() for efficient per-name existence checks used
  by dry-run
- Add reqwest (rustls-tls + json) and url dependencies
- Integration tests: auth guard, allowlist rejection, mirror +
  skip-duplicates, dry-run no-persist

2026-05-02 02:48:24 +02:00

1.3 KiB

Raw Permalink Blame History

mikebase

A Rust-based emote database and API.

Importing legacy emotes

Use the protected management endpoint to mirror emotes from a legacy JSON feed.

Endpoint

POST /manage/import

Auth: HTTP Basic (same credentials used for other protected routes)
Content-Type: application/json
Body:

{
  "source_url": "https://smutba.se/emoji/json/"
}

Behavior

Fetches source JSON in the format {"emotes": [{name, url, created, modified}, ...]}
Downloads each image URL and uploads bytes to this app's configured S3 bucket
Inserts emote rows preserving source created and modified timestamps
Skips entries where name already exists locally
Continues processing after per-item failures and returns a batch summary

Example response

{
  "source_url": "https://smutba.se/emoji/json/",
  "total": 2,
  "imported": 1,
  "skipped": 1,
  "failed": 0,
  "results": [
    {"name": "legacy_new", "status": "imported", "reason": null},
    {"name": "legacy_duplicate", "status": "skipped", "reason": "Name already exists"}
  ]
}

Allowlisted hosts

Import is restricted to hosts in configuration:

[import]
allowed_hosts = ["smutba.se"]

Environment override example:

APP__IMPORT__ALLOWED_HOSTS=["smutba.se","legacy.example.org"]

1.3 KiB Raw Permalink Blame History