No description
  • TypeScript 97.6%
  • Shell 1.3%
  • JavaScript 0.8%
  • Standard ML 0.3%
Find a file
2026-07-13 21:11:06 -05:00
.forgejo/workflows Bugfix: Fixed CI/CD pipeline gates so that it gets deployed using internal tools 2026-07-09 21:56:03 -05:00
.githooks chore(release): pre-push guard — warn when pushing unreleased commits 2026-07-03 20:48:27 -05:00
dist Bugfix: Reinforced steam app ID mapping if it matches the filename's blob 2026-07-13 21:11:06 -05:00
docs Bugfix: Fixed bug where mapped games are treated as not mapped 2026-07-11 23:17:05 -05:00
node_modules Initial Commit 2026-06-30 01:59:00 -05:00
release release: v0.1.17 2026-07-11 23:17:15 -05:00
sandbox chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
.gitattributes fix(release): publish the COMMITTED signed tarball; reproducibility becomes informational 2026-07-03 21:58:32 -05:00
.gitignore chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
_service-test-stubs.ts Bugfix: Fixed bug where mapped games are treated as not mapped 2026-07-11 23:17:05 -05:00
auto-bump.js chore(release): auto-bump patch when releasing an already-released version 2026-07-03 20:37:26 -05:00
BACKLOG.md Feature: Added app id mapping for steam app ID numbers 2026-07-13 11:53:53 -05:00
claim-registry.test.ts fix(ssot): release the claim on a game boundary, not setImmediate 2026-07-05 00:43:30 -05:00
claim-registry.ts fix(ssot): release the claim on a game boundary, not setImmediate 2026-07-05 00:43:30 -05:00
CLAUDE.md chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
configuration.ts Bugfix: Fixed bug where mapped games are treated as not mapped 2026-07-11 23:17:05 -05:00
deploy.sh feat(deploy): build + staged hot-redeploy into the local GameVault stack 2026-07-03 20:27:56 -05:00
dry-run.ts chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
gamevault.d.ts Initial Commit 2026-06-30 01:59:00 -05:00
HANDOFF.md Bugfix: Fixed bug where mapped games are treated as not mapped 2026-07-11 23:17:05 -05:00
helpers.ts Initial Commit 2026-06-30 01:59:00 -05:00
LICENSE chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
matching.test.ts Bugfix: Reinforced steam app ID mapping if it matches the filename's blob 2026-07-13 21:11:06 -05:00
matching.ts Bugfix: Reinforced steam app ID mapping if it matches the filename's blob 2026-07-13 21:11:06 -05:00
models.ts chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
name-normalizer.ts Bugfix: Reinforced steam app ID mapping if it matches the filename's blob 2026-07-13 21:11:06 -05:00
package-lock.json release: v0.1.2 2026-07-03 21:49:22 -05:00
package.json release: v0.1.17 2026-07-11 23:17:15 -05:00
plugin-version.ts release: v0.1.17 2026-07-11 23:17:15 -05:00
postbuild.js Initial Commit 2026-06-30 01:59:00 -05:00
README.md Bugfix: Fixed bug where mapped games are treated as not mapped 2026-07-11 23:17:05 -05:00
release.ts fix(release): publish the COMMITTED signed tarball; reproducibility becomes informational 2026-07-03 21:58:32 -05:00
self-updater.ts feat(update): signed stage-only self-updater v1 + release tooling + CI pipeline 2026-07-03 20:29:30 -05:00
ship.js chore(release): npm run ship — the whole release in one command 2026-07-03 20:39:47 -05:00
status.sh feat(ops): status.sh — one-command channel/disk/running verification 2026-07-03 22:52:15 -05:00
steam-store-client.test.ts Bugfix: Fixed bug where mapped games are treated as not mapped 2026-07-11 23:17:05 -05:00
steam-store-client.ts Bugfix: Fixed bug where mapped games are treated as not mapped 2026-07-11 23:17:05 -05:00
steam-unfiltered.metadata-provider.service.ts Bugfix: Reinforced steam app ID mapping if it matches the filename's blob 2026-07-13 21:11:06 -05:00
steam-unfiltered.plugin.module.ts chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
steam-unfiltered.service.test.ts Bugfix: Reinforced steam app ID mapping if it matches the filename's blob 2026-07-13 21:11:06 -05:00
sync-version.js chore(release): one-command version bumps via the npm version hook 2026-07-03 20:31:34 -05:00
tsconfig.json build(tsconfig): exclude _*.ts probe files so tsc never collides with review-agent scratch 2026-07-04 18:19:44 -05:00
update-crypto.ts feat(release): masked passphrase prompt + three key-protection modes 2026-07-03 21:37:26 -05:00
update-keys.ts chore(keys): pin the new primary release pubkey (plain identity file) 2026-07-03 21:48:15 -05:00
update-state.ts fix(update): per-process temp name for state writes (cross-instance rename race) 2026-07-03 22:53:26 -05:00
update-swap.ts chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
update-tar.ts feat(update): signed stage-only self-updater v1 + release tooling + CI pipeline 2026-07-03 20:29:30 -05:00
updater.test.ts chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
wayback-media-client.ts chore: rebrand prose to LudoCodex; LICENSE; untrack stale sandbox artifacts 2026-07-03 23:53:42 -05:00
websearch-client.test.ts Merge branch 'main' of https://git.lordandradus.dev/LordAndradus/ludonexus-plugin-steamstore into feature/better-normalization 2026-07-09 06:13:01 -05:00
websearch-client.ts Merge branch 'main' of https://git.lordandradus.dev/LordAndradus/ludonexus-plugin-steamstore into feature/better-normalization 2026-07-09 06:13:01 -05:00

Steam Store (Unfiltered) — LudoCodex Backend Plugin

TS port of the SteamStoreMetadataUnfiltered C# Playnite plugin, packaged as a drop-in NestJS module for the LudoCodex backend.

Unlike the official Steam Library plugin, this one bypasses the store age gate, pulls full store data plus community tags (mature-content store tags included), and — when it confidently matches a game — takes over as the single source of truth for that game, unmapping all other providers from it.

Zero third-party runtime dependencies. This plugin requires only @nestjs/common (provided by the backend), so dist/ drops straight into /plugins with no bundled node_modules. The former cheerio and steam-user (PICS) dependencies were removed — see docs/cheerio-and-steam-user.md for what they did and how to reintroduce them if ever needed.

Why "single source of truth"

LudoCodex's merge engine combines metadata from every enabled provider for a given game. For most cases that's great (gap-fill). For Steam-native titles this plugin matches strictly, the result can be: cleaner cover art, fewer wrong tags, and one canonical mapping the user doesn't have to babysit.

When this provider strictly matches a game's title (case-insensitive exact, or alphanumeric-only exact — same happy paths as the original C# plugin's FindMatchingGame), the takeover runs as a deliberate sequence inside getBestMatch, before returning the matched DTO to the platform:

  1. Unmap every provider currently attached to the game (including any stale Steam Unfiltered entry — the upcoming map() call re-attaches a fresh one).
  2. Mark the game claimed in a module-scope Set. A one-time prototype patch on MetadataProvider.getBestMatch makes every other provider throw NotFoundException for the claimed game, which the findMetadata caller treats as "no match" → skip.
  3. Mutate the in-memory game.provider_metadata to empty so the updateMetadata loop's "existing provider metadata?" branch routes through getBestMatch (which the patch will throw from) instead of map-by-existing-id (which would bypass the patch).
  4. Return the match. The platform's metadataService.map(steam-unfiltered, …) runs next, which re-attaches Steam Unfiltered as the only mapping.
  5. The remaining providers in the loop hit the patched getBestMatch and are skipped.
  6. The platform's final merge(gameId) sees only Steam Unfiltered in provider_metadata.

The claim is cleared via setImmediate so it scopes to the current updateMetadata loop — a future loop will re-claim naturally if Steam still strictly matches.

Strict vs. fuzzy claims

Both strict and fuzzy matches claim the game and block other providers when EXCLUSIVE=true, but they differ in title handling:

  • Strict match (case-insensitive exact OR alphanumeric-only equality) → claim + use Steam's title as the canonical title.
  • Fuzzy match (first result fallback when no strict candidate exists) → claim + block others + preserve the original game.title (the user's library title from the filename or initial entry). Steam's title is discarded — that's the one field we explicitly can't trust on a fuzzy match, since a string mismatch is what made it fuzzy in the first place. Cover, tags, screenshots, links, description, etc. are still Steam's — they're almost always correct on a fuzzy match, only the title is suspect.

Fuzzy matches log a WARN with both the original title and Steam's title so admins can spot mispairs and remap manually if needed. The cover and tags landing on the original title is also a visual tell — if they look obviously wrong, the match was bad.

With EXCLUSIVE=false, fuzzy matches behave like any normal provider: their title goes through the standard priority-based merge (which Steam will usually win anyway, by priority).

The Steam Store Map Failed tag

Any game where Steam Unfiltered couldn't produce a confident strict match gets tagged Steam Store Map Failed so the user can sort/filter the library for items that need manual review. The tag fires in two scenarios:

  • Fuzzy match → tag appended to the full Steam metadata (cover, screenshots, etc.). The tag survives the merge because SSOT blocked other providers from contributing tags.
  • No match → instead of throwing NotFoundException, this plugin ships a sentinel mapping with provider_data_id = "unmapped-{gameId}", provider_priority = -1 (lowest), and only the failure tag. SSOT is NOT triggered — other providers can still match and contribute real data.

Transient failures never degrade a mapping

The sentinel — and the recovered-* delisted downgrade — ship only on positive evidence: Steam answered and said "zero results" / "no data for this appid" (HTTP 200 + success:false, or a definitive 404). A rate-limited or erroring Steam (429/5xx/403, timeouts, network failures) is retried, and if it never clears the provider throws instead — the platform keeps the previous mapping and the next scan retries. Cover/background probes follow the same rule: a CDN blip fails that refresh pass (previous art is kept) rather than persisting a blank cover over good art.

recovered-* mappings additionally self-heal: every recache re-checks the appid against the live storefront (a direct appdetails lookup — no re-search, no mis-map risk) and upgrades back to a full live mapping if Steam lists it again. Games downgraded by the historical transient-as-delisted bug repair themselves on their next recache. The unmapped-* sentinel remains sticky by design — unmap the game to re-discover. One EXCLUSIVE=true nuance: self-heal runs from a refresh (no game entity in scope), so it cannot unmap other providers itself — any gap-fill mappings that attached while the game was recovered-* coexist until the next full metadata update re-runs getBestMatch and re-claims.

Tag merge limitation (no-match case)

The platform's merge replaces array fields (like tags) per provider rather than unioning them. For a no-match game where IGDB (or another provider) successfully maps and ships its own tags array, IGDB's tags overwrite ours at merge time and the failure tag is lost. The trade-off was chosen deliberately: better to surface IGDB's real [Action, RPG, Open-World] tags than to clobber them with [Steam Store Map Failed].

The tag survives in the most valuable case — when nobody matched, so the game is genuinely orphaned in the library.

For the "Steam failed but IGDB succeeded" case, two alternative filters work:

  • API: GET /api/games?filter[provider_metadata.provider_data_id][startsWith]=unmapped- lists every game with the sentinel.
  • DB: SELECT … FROM game_metadata WHERE provider_slug = 'steam-unfiltered' AND provider_data_id LIKE 'unmapped-%' is the canonical query.

A backend steam_map_status field would fix this cleanly — parked as future work (backlog phase 999.2 territory).

Known limitation

Direct /api/metadata/map?provider=steam-unfiltered calls (e.g. the user manually picking Steam in the UI) bypass getBestMatch and so don't currently trigger the takeover. Re-trigger metadata update for the game to apply SSOT, or set EXCLUSIVE=false if you'd rather always merge.

Environment configuration

All env vars are prefixed PLUGIN_LORDANDRADUS_STEAM_UNFILTERED_ (mirroring ITAD).

Var Default Notes
ENABLED true Disable to skip provider registration entirely.
PRIORITY 50 Higher than IGDB (30) and ITAD (10) so this runs first in the merge loop. Must be unique across providers.
REQUEST_INTERVAL_MS 750 Throttle between Steam Store API calls. Steam allows ~100 req/min/IP.
RETRY_INTERVAL_MS 2500 Delay between retries of a transiently-failed Steam request (429/5xx/network). Raise it if Steam's rate-limiter needs longer to cool down.
LANGUAGE english Steam language key — german, russian, schinese, brazilian, etc.
BACKGROUND_SOURCE image One of image, store-screenshot, store-background, banner.
VERTICAL_COVERS true Prefer library_600x900_2x.jpg over header.jpg for cover art.
TAG_BLACKLIST (empty) Comma-separated Steam tag IDs (numeric) to drop. E.g. 493,11014.
EXCLUSIVE true The single-source-of-truth switch. Set false to behave like any other gap-fill provider.
AGE_GATE_COOKIES C# defaults Static cookies — exposed so you can refresh them without rebuilding if Steam changes the format.
WEBSEARCH_ENABLED false Opt-in delisted/renamed-game recovery: resolve the Steam appid via web-search when the storefront has no confident match.
WEBSEARCH_PROVIDER auto auto prefers SearXNG (if WEBSEARCH_URL set), then Brave (if WEBSEARCH_API_KEY set), always ending with the keyless DuckDuckGo scrape. Or force one: searxng, brave, duckduckgo.
WEBSEARCH_URL (empty) Base URL of your SearXNG instance (the INTERNAL address, not behind SSO).
WEBSEARCH_API_KEY (empty) API key for hosted providers (Brave).
WEBSEARCH_CONFIDENCE 0.75 Fuzzy-claim confidence gate (0..1) below which recovery escalates to web-search instead of claiming a lookalike.
WEBSEARCH_COOLDOWN_MS 3600000 (1 h) Circuit-breaker: a public engine (DDG, Brave) that fails to respond (CAPTCHA/429) is skipped for this long — re-poking a soft-blocked engine prolongs the block. 0 disables.
WEBSEARCH_SEARXNG_COOLDOWN_MS 300000 (5 min) Same breaker for self-hosted SearXNG — much shorter, since hitting your own box aggravates nothing and skipping it longer only diverts traffic to the public engines. 0 disables.

Install (no clone, no build — the release channel is the installer)

From the directory containing your LudoCodex stack's plugins/:

BASE=https://git.lordandradus.dev/api/packages/LordAndradus/generic/steam-unfiltered/stable
V=$(curl -fsS "$BASE/manifest.json" | grep -o '"version": *"[^"]*"' | cut -d'"' -f4)
mkdir -p ./plugins/gamevault-plugin-steam-unfiltered
curl -fsSL "$BASE/steam-unfiltered-$V.tar.gz" | tar -xz -C ./plugins/gamevault-plugin-steam-unfiltered

Enable via env (PLUGIN_LORDANDRADUS_STEAM_UNFILTERED_ENABLED: "true"), restart the backend. Opt into self-updates (see below) and this is the last manual install that server ever needs. Paranoia check on first install: compare sha256sum of the tarball against the manifest — every LATER update is signature-verified automatically against the key pinned inside the bundle.

Build & deploy

cd SteamStoreMetadataUnfiltered-Plugin-LudoCodex
npm install
npm run build                 # → dist/

Then copy the contents of dist/ into the LudoCodex server's /plugins volume and restart the container. The plugin loader scans for *.plugin.module.js; steam-unfiltered.plugin.module.js will auto-register the provider on boot.

For a one-shot deploy into a docker-compose sandbox:

./deploy.sh                   # rebuilds + restarts the sandbox
./deploy.sh --fresh           # also wipes volumes

Releases & self-update (CI/CD)

Full design + decisions: docs/auto-update-design.md.

One-time (maintainer machine): run npx tsx release.ts init-keys TWICE (primary + cold spare; the spare's key file goes to offline storage), pin both printed public keys in update-keys.ts, commit. Private keys live in ~/.ludonexus-keys/ — outside the repo tree, passphrase-encrypted, never in git or CI.

Per release — ONE command (versions move per RELEASE, never per commit):

npm run ship

That's the whole thing: auto-bumps the patch if you didn't version, builds, tests, prompts for your signing passphrase (that prompt IS the release decision), commits, tags, pushes. For a feature or breaking release, run npm version minor|major --no-git-tag-version first — ship respects it. If the fleet didn't update, the only questions are "did I run ship?" and "is the Actions tab green?" The tag push triggers CI, which holds no keys: it rebuilds, proves the bytes match your signed manifest, boot-tests the bundle in a real backend container, then publishes — bundle first, manifest last — to the generic package registry under a flat stable channel.

One-time (Forgejo instance):

  1. Enable Actions (instance setting, plus the repo's Units tab).
  2. A DEDICATED forgejo-runner registered at the REPO scope (repo Settings → Actions → Runners → Create new runner) with the label ludonexus:docker://node:22-bookworm, and the host docker socket exposed to job containers via its config.yml:
    container:
      docker_host: "automount"
    
    Repo-scoped on purpose: the socket is code-execution on the runner host, so only this repo's workflows may reach it. Full setup + unit file: docs/forgejo-ci-setup.md.
  3. Protect the v* tag pattern and main so only the maintainer can mint tags.
  4. No secrets to configure — that is the point.

Per server (opt-in): set AUTO_UPDATE=true, UPDATE_URL=https://<forgejo>/api/packages/<owner>/generic/steam-unfiltered/stable, and mount ./plugins writable (self-update and the :ro hardening are mutually exclusive). Staged updates apply at the next container restart; UPDATE_PIN=<version> freezes or force-converges a specific version.

Dry-run / sanity check

dry-run.ts exercises every client independently of NestJS:

npx tsx dry-run.ts
npx tsx dry-run.ts "Elden Ring"
npx tsx dry-run.ts "Cyberpunk 2077" 1091500

Field mapping

Steam source GameMetadata field
appdetails.name title
appdetails.about_the_game (cleaned HTML) description
appdetails.release_date.date (locale-parsed) release_date
appdetails.required_age (numeric) age_rating
Reviews → calculateUserScore (or metacritic.score fallback) rating
appdetails.developers developers[]
appdetails.publishers publishers[]
appdetails.genres genres[]
appdetails.categories (features) + scraped store-page community tags tags[] (deduped by name, minus TAG_BLACKLIST)
GetItems library_capsulehero_capsule → 2x variants → legacy library_600x900header (HEAD-probed) cover (Media)
GetItems library_hero + BACKGROUND_SOURCE candidates (HEAD-probed) background (Media)
appdetails.screenshots[].path_full url_screenshots[]
appdetails.movies (split by trailer-keyword) url_trailers[] / url_gameplays[]
Store + community links url_websites[]

Provenance

Initial port grounded by the GAMEVAULT-PORT-BRIEF.md handoff document.