Ganonmaster
9c5212de05
Add POST /manage/import (auth-protected) that fetches emotes from a legacy JSON endpoint, downloads each image, uploads it to S3, and inserts it into the DB with the original timestamps preserved. - Skip emotes whose name already exists (best-effort duplicate detection across SQLite and PostgreSQL via error code + message fallback) - Validate source_url against a configurable host allowlist ([import] allowed_hosts in config, default ["smutba.se"]) - dry_run: true previews the import without writing to S3 or DB; result statuses are "would_import" / "would_skip" instead of "imported" / "skipped" - Add db.name_exists() for efficient per-name existence checks used by dry-run - Add reqwest (rustls-tls + json) and url dependencies - Integration tests: auth guard, allowlist rejection, mirror + skip-duplicates, dry-run no-persist
58 lines
1.3 KiB
Markdown
58 lines
1.3 KiB
Markdown
# mikebase
|
|
A Rust-based emote database and API.
|
|
|
|
## Importing legacy emotes
|
|
|
|
Use the protected management endpoint to mirror emotes from a legacy JSON feed.
|
|
|
|
### Endpoint
|
|
|
|
POST /manage/import
|
|
|
|
- Auth: HTTP Basic (same credentials used for other protected routes)
|
|
- Content-Type: application/json
|
|
- Body:
|
|
|
|
```json
|
|
{
|
|
"source_url": "https://smutba.se/emoji/json/"
|
|
}
|
|
```
|
|
|
|
### Behavior
|
|
|
|
- Fetches source JSON in the format `{"emotes": [{name, url, created, modified}, ...]}`
|
|
- Downloads each image URL and uploads bytes to this app's configured S3 bucket
|
|
- Inserts emote rows preserving source `created` and `modified` timestamps
|
|
- Skips entries where `name` already exists locally
|
|
- Continues processing after per-item failures and returns a batch summary
|
|
|
|
### Example response
|
|
|
|
```json
|
|
{
|
|
"source_url": "https://smutba.se/emoji/json/",
|
|
"total": 2,
|
|
"imported": 1,
|
|
"skipped": 1,
|
|
"failed": 0,
|
|
"results": [
|
|
{"name": "legacy_new", "status": "imported", "reason": null},
|
|
{"name": "legacy_duplicate", "status": "skipped", "reason": "Name already exists"}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Allowlisted hosts
|
|
|
|
Import is restricted to hosts in configuration:
|
|
|
|
```toml
|
|
[import]
|
|
allowed_hosts = ["smutba.se"]
|
|
```
|
|
|
|
Environment override example:
|
|
|
|
- `APP__IMPORT__ALLOWED_HOSTS=["smutba.se","legacy.example.org"]`
|